2024-03-30T12:56:41.303 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-30T12:56:41.304 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149 --owner scheduled_mchangir@teuthology --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} --name mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi -- /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/orig.config.yaml 2024-03-30T12:56:41.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T12:56:41.441 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149 branch: wip-mchangir-testing1-main-20240328-072412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} email: mchangir@redhat.com first_in_suite: false job_id: '7629149' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing1-main-20240328-072412 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing1-main-20240328-072412 sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 owner: scheduled_mchangir@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 364 sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-mchangir-testing1-main-20240328-072412 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFY4upZpo7cB0coVquEDOdrwqJg6OHFD4EoV56xP3Skyi4ezLzOlJj9k+KZRYRwcxBjfPZvKAZSZ0i/etF3yKMU= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwDGvp2eo//D/JeOH1AxX2bT9k/mPLPwvuLr2Wx8VimpD9fbDot15X4u0nzdw4lsMuJtCNAREBkkp+FK9HhG0s= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_uninlining teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_13:38:54 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-30T12:56:41.442 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa; will attempt to use it 2024-03-30T12:56:41.442 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks 2024-03-30T12:56:41.443 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-30T12:56:41.444 INFO:teuthology.task.internal:Checking packages... 2024-03-30T12:56:41.471 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7' 2024-03-30T12:56:41.471 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-30T12:56:41.471 INFO:teuthology.packaging:ref: None 2024-03-30T12:56:41.471 INFO:teuthology.packaging:tag: None 2024-03-30T12:56:41.472 INFO:teuthology.packaging:branch: wip-mchangir-testing1-main-20240328-072412 2024-03-30T12:56:41.472 INFO:teuthology.packaging:sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T12:56:41.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing1-main-20240328-072412 2024-03-30T12:56:41.772 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2519.gac171039 2024-03-30T12:56:41.773 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-30T12:56:41.780 INFO:teuthology.task.internal:no buildpackages task found 2024-03-30T12:56:41.781 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-30T12:56:41.830 INFO:teuthology.task.internal:Saving configuration 2024-03-30T12:56:41.849 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-30T12:56:41.855 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T12:56:41.881 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:01.285193', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFY4upZpo7cB0coVquEDOdrwqJg6OHFD4EoV56xP3Skyi4ezLzOlJj9k+KZRYRwcxBjfPZvKAZSZ0i/etF3yKMU='} 2024-03-30T12:56:41.907 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:01.286856', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwDGvp2eo//D/JeOH1AxX2bT9k/mPLPwvuLr2Wx8VimpD9fbDot15X4u0nzdw4lsMuJtCNAREBkkp+FK9HhG0s='} 2024-03-30T12:56:41.907 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-30T12:56:41.914 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-30T12:56:41.914 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-30T12:56:41.914 INFO:teuthology.run_tasks:Running task console_log... 2024-03-30T12:56:41.986 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f26eaf0df70>, signals=[15]) 2024-03-30T12:56:41.987 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-30T12:56:41.993 INFO:teuthology.task.internal:Opening connections... 2024-03-30T12:56:41.993 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T12:56:41.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:56:42.107 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T12:56:42.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:56:42.198 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-30T12:56:42.206 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-03-30T12:56:42.223 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-03-30T12:56:42.224 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-03-30T12:56:42.281 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T12:56:42.282 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-30T12:56:42.283 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-30T12:56:42.283 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-03-30T12:56:42.312 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-03-30T12:56:42.329 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-03-30T12:56:42.329 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-03-30T12:56:42.385 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-30T12:56:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-03-30T12:56:42.387 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T12:56:42.387 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-30T12:56:42.387 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-30T12:56:42.387 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-03-30T12:56:42.412 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-30T12:56:42.422 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-30T12:56:42.429 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-30T12:56:42.430 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-03-30T12:56:42.433 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-03-30T12:56:42.448 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-30T12:56:42.455 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-30T12:56:42.455 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-03-30T12:56:42.490 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-03-30T12:56:42.581 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-30T12:56:42.637 INFO:teuthology.run_tasks:Running task kernel... 2024-03-30T12:56:42.651 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-30T12:56:42.651 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-30T12:56:42.651 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-30T12:56:42.651 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T12:56:42.652 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T12:56:42.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T12:56:42.668 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-30T12:56:42.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T12:56:42.669 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-03-30T12:56:42.725 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-361.el9.x86_64 2024-03-30T12:56:42.726 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-361.el9.x86_64 2024-03-30T12:56:42.726 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-03-30T12:56:42.728 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-361.el9.x86_64 2024-03-30T12:56:42.728 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.14.0-361.el9.x86_64 2024-03-30T12:56:42.728 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum install -y kernel 2024-03-30T12:56:44.830 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-30T12:56:46.029 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 3.9 MB/s | 6.7 MB 00:01 2024-03-30T12:56:46.288 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-30T12:56:47.221 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-30T12:56:47.670 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-03-30T12:56:48.569 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-30T12:56:49.122 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-03-30T12:56:52.528 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 19 MB 00:02 2024-03-30T12:56:54.022 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.1 MB 00:01 2024-03-30T12:56:55.473 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-03-30T12:56:56.861 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-30T12:56:58.104 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-03-30T12:56:59.262 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-30T12:57:00.313 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-03-30T12:57:03.225 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-03-30T12:57:05.254 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-30T12:57:05.540 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-30T12:57:05.540 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-30T12:57:05.654 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-03-30T12:57:05.666 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout:Install 4 Packages 2024-03-30T12:57:05.667 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:05.668 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 76 M 2024-03-30T12:57:05.668 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 123 M 2024-03-30T12:57:05.668 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-03-30T12:57:05.810 INFO:teuthology.orchestra.run.smithi066.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-30T12:57:06.577 INFO:teuthology.orchestra.run.smithi066.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 17 MB/s | 15 MB 00:00 2024-03-30T12:57:07.018 INFO:teuthology.orchestra.run.smithi066.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-03-30T12:57:07.335 INFO:teuthology.orchestra.run.smithi066.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 18 MB/s | 28 MB 00:01 2024-03-30T12:57:07.339 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-03-30T12:57:07.339 INFO:teuthology.orchestra.run.smithi066.stdout:Total 46 MB/s | 76 MB 00:01 2024-03-30T12:57:07.339 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-03-30T12:57:07.544 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-30T12:57:07.544 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-30T12:57:07.638 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-30T12:57:07.649 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-30T12:57:07.649 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-30T12:57:07.649 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-03-30T12:57:07.650 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:07.651 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 76 M 2024-03-30T12:57:07.651 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 123 M 2024-03-30T12:57:07.651 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-30T12:57:07.768 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-03-30T12:57:07.768 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-03-30T12:57:07.797 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-30T12:57:08.464 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 19 MB/s | 15 MB 00:00 2024-03-30T12:57:08.626 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-03-30T12:57:08.626 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-03-30T12:57:09.039 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-03-30T12:57:09.440 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 17 MB/s | 28 MB 00:01 2024-03-30T12:57:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-30T12:57:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:Total 43 MB/s | 76 MB 00:01 2024-03-30T12:57:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-30T12:57:09.840 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-30T12:57:09.841 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-30T12:57:10.703 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-30T12:57:10.703 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-30T12:57:11.099 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-03-30T12:57:11.740 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-30T12:57:11.827 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-30T12:57:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-30T12:57:13.703 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-30T12:57:13.798 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-30T12:57:14.749 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-30T12:57:14.878 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-30T12:57:16.563 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-30T12:57:16.709 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-30T12:57:19.960 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-30T12:57:20.002 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:21.434 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-30T12:57:21.691 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:25.111 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:55.707 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:55.707 INFO:teuthology.orchestra.run.smithi066.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T12:57:55.707 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:55.734 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:55.734 INFO:teuthology.orchestra.run.smithi117.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T12:57:55.734 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:55.755 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:55.777 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:56.090 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:56.090 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-30T12:57:56.090 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-30T12:57:56.090 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-30T12:57:56.145 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:56.145 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-30T12:57:56.145 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-30T12:57:56.145 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.507 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:56.508 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:56.598 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-30T12:57:56.819 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-03-30T12:57:56.909 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-03-30T12:57:56.930 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-30T12:57:56.930 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-30T12:57:56.931 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-30T12:57:56.931 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-30T12:57:56.931 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-30T12:57:56.931 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-30T12:57:56.931 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=None, version=distro) 2024-03-30T12:57:56.931 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-03-30T12:57:57.003 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-30T12:57:57.003 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-30T12:57:57.003 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-30T12:57:57.004 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-30T12:57:57.004 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-30T12:57:57.004 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-30T12:57:57.004 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-03-30T12:57:57.004 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-03-30T12:57:57.014 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-30T12:57:57.014 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-30T12:57:57.014 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -qi grub2-tools 2024-03-30T12:57:57.058 INFO:teuthology.orchestra.run.smithi066.stdout:Name : grub2-tools 2024-03-30T12:57:57.058 INFO:teuthology.orchestra.run.smithi066.stdout:Epoch : 1 2024-03-30T12:57:57.058 INFO:teuthology.orchestra.run.smithi066.stdout:Version : 2.06 2024-03-30T12:57:57.058 INFO:teuthology.orchestra.run.smithi066.stdout:Release : 68.el9 2024-03-30T12:57:57.058 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: x86_64 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Group : Unspecified 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Size : 8264254 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:License : GPLv3+ 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Packager : builder@centos.org 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Vendor : CentOS 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Summary : Support tools for GRUB. 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout:Description : 2024-03-30T12:57:57.059 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:57.060 INFO:teuthology.orchestra.run.smithi066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-30T12:57:57.060 INFO:teuthology.orchestra.run.smithi066.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-30T12:57:57.060 INFO:teuthology.orchestra.run.smithi066.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-30T12:57:57.060 INFO:teuthology.orchestra.run.smithi066.stdout:hardware devices. 2024-03-30T12:57:57.060 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T12:57:57.060 INFO:teuthology.orchestra.run.smithi066.stdout:This subpackage provides tools for support of all platforms. 2024-03-30T12:57:57.061 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-30T12:57:57.061 INFO:teuthology.task.kernel:Updating grub on smithi066 to boot 5.14.0-434.el9.x86_64 2024-03-30T12:57:57.061 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-30T12:57:57.083 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-30T12:57:57.084 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-30T12:57:57.084 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 68.el9 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8264254 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-03-30T12:57:57.118 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T12:57:57.119 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-03-30T12:57:57.120 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-30T12:57:57.120 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-434.el9.x86_64 2024-03-30T12:57:57.120 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-30T12:57:57.642 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-03-30T12:57:57.752 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-03-30T12:57:59.004 INFO:teuthology.orchestra.run.smithi066.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-30T12:57:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-03-30T12:57:59.045 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-03-30T12:57:59.066 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.rC9C7UDg2W 2024-03-30T12:57:59.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rC9C7UDg2W 2024-03-30T12:57:59.068 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-30T12:57:59.099 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-03-30T12:57:59.102 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-30T12:57:59.121 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.GiDt0ZWahf 2024-03-30T12:57:59.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GiDt0ZWahf 2024-03-30T12:57:59.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.rC9C7UDg2W 2024-03-30T12:57:59.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.GiDt0ZWahf 2024-03-30T12:57:59.304 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.rC9C7UDg2W is 6KB 2024-03-30T12:57:59.330 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.rC9C7UDg2W 2024-03-30T12:57:59.335 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.GiDt0ZWahf is 6KB 2024-03-30T12:57:59.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo /bin/ls /boot/loader/entries || true 2024-03-30T12:57:59.357 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.GiDt0ZWahf 2024-03-30T12:57:59.371 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-03-30T12:57:59.416 INFO:teuthology.orchestra.run.smithi066.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-30T12:57:59.416 INFO:teuthology.orchestra.run.smithi066.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-30T12:57:59.416 INFO:teuthology.orchestra.run.smithi066.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-30T12:57:59.416 INFO:teuthology.orchestra.run.smithi066.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-30T12:57:59.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-30T12:57:59.438 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-30T12:57:59.438 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-30T12:57:59.438 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-30T12:57:59.439 INFO:teuthology.orchestra.run.smithi117.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-30T12:57:59.440 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-30T12:57:59.561 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2024-03-30T12:57:59.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-03-30T12:58:29.594 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-30T12:58:29.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T12:58:29.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:58:29.598 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-30T12:58:29.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-30T12:58:29.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:04.095 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-30T12:59:04.095 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T12:59:13.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-30T12:59:13.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:13.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T12:59:13.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:16.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T12:59:28.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T12:59:28.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:31.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T12:59:31.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-30T12:59:43.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-30T12:59:43.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:43.971 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-30T12:59:44.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-30T12:59:44.562 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-434.el9.x86_64"... 2024-03-30T12:59:44.563 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-30T12:59:44.576 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-434.el9.x86_64 2024-03-30T12:59:44.576 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-30T12:59:44.577 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T12:59:44.577 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-30T12:59:45.577 DEBUG:teuthology.parallel:result is None 2024-03-30T12:59:46.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T12:59:46.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:49.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T13:00:07.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T13:00:07.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T13:00:10.370 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-03-30T13:00:31.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T13:00:31.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T13:00:31.743 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-03-30T13:00:32.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T13:00:32.241 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-434.el9.x86_64"... 2024-03-30T13:00:32.242 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-03-30T13:00:32.256 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-434.el9.x86_64 2024-03-30T13:00:32.256 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-30T13:00:32.256 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T13:00:32.256 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-30T13:00:33.258 DEBUG:teuthology.parallel:result is None 2024-03-30T13:00:33.258 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-30T13:00:33.265 INFO:teuthology.task.internal:Creating test directory... 2024-03-30T13:00:33.266 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T13:00:33.268 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T13:00:33.289 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-30T13:00:33.297 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-30T13:00:33.312 INFO:teuthology.task.internal:Creating archive directory... 2024-03-30T13:00:33.312 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T13:00:33.325 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T13:00:33.440 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-30T13:00:33.446 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-30T13:00:33.446 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T13:00:33.449 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T13:00:33.511 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T13:00:33.557 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T13:00:33.560 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T13:00:33.588 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T13:00:33.590 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-30T13:00:33.598 INFO:teuthology.task.internal:Configuring sudo... 2024-03-30T13:00:33.598 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T13:00:33.600 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T13:00:33.630 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-30T13:00:33.642 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-30T13:00:33.642 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T13:00:33.673 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T13:00:33.689 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T13:00:33.769 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T13:00:33.824 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T13:00:33.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T13:00:33.952 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:00:33.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T13:00:34.040 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T13:00:34.084 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T13:00:34.141 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T13:00:34.222 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T13:00:34.269 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:00:34.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T13:00:34.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-03-30T13:00:34.340 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-03-30T13:00:34.394 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T13:00:34.438 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T13:00:34.723 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-30T13:00:34.732 INFO:teuthology.task.internal:Starting timer... 2024-03-30T13:00:34.732 INFO:teuthology.run_tasks:Running task pcp... 2024-03-30T13:00:34.748 INFO:teuthology.run_tasks:Running task selinux... 2024-03-30T13:00:34.757 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-30T13:00:34.758 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-03-30T13:00:34.820 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-03-30T13:00:34.822 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-03-30T13:00:34.884 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-03-30T13:00:34.886 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-30T13:00:34.886 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-03-30T13:00:34.910 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-03-30T13:00:34.910 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-03-30T13:00:34.929 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-03-30T13:00:34.929 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-03-30T13:00:34.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-30T13:00:34.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:00:34.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-30T13:00:35.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:00:35.010 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-30T13:00:35.011 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-03-30T13:00:35.155 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-03-30T13:00:35.190 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-30T13:00:35.200 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-30T13:00:35.222 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-30T13:00:35.223 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-30T13:06:31.111 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-03-30T13:06:31.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T13:06:31.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T13:06:31.237 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-03-30T13:06:31.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-30T13:06:31.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-30T13:06:31.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T13:06:31.407 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-30T13:06:31.486 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-30T13:06:31.486 INFO:teuthology.run_tasks:Running task clock... 2024-03-30T13:06:31.499 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-30T13:06:31.499 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T13:06:31.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:06:31.502 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T13:06:31.502 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:06:31.535 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T13:06:31.555 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T13:06:31.559 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T13:06:31.576 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T13:06:31.592 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-03-30T13:06:31.605 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-03-30T13:06:31.609 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-03-30T13:06:31.620 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-03-30T13:06:31.629 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T13:06:31.635 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T13:06:31.648 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T13:06:31.653 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T13:06:31.702 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-03-30T13:06:31.705 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-03-30T13:06:31.705 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T13:06:31.706 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-03-30T13:06:31.706 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.706 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.706 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.706 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.708 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T13:06:31.708 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-03-30T13:06:31.708 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.708 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.709 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.709 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:06:31.709 INFO:teuthology.run_tasks:Running task install... 2024-03-30T13:06:31.718 DEBUG:teuthology.task.install:project ceph 2024-03-30T13:06:31.718 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}} 2024-03-30T13:06:31.718 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'} 2024-03-30T13:06:31.718 INFO:teuthology.task.install:Using flavor: default 2024-03-30T13:06:31.724 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-30T13:06:31.724 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-30T13:06:31.725 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'tag': None, 'wait_for_package': False} 2024-03-30T13:06:31.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:06:31.727 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'tag': None, 'wait_for_package': False} 2024-03-30T13:06:31.727 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:06:31.874 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/ 2024-03-30T13:06:31.874 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2519.gac171039 2024-03-30T13:06:31.970 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/ 2024-03-30T13:06:31.970 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2519.gac171039 2024-03-30T13:06:31.989 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T13:06:31.989 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:06:31.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T13:06:32.022 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T13:06:32.023 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T13:06:32.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T13:06:32.143 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T13:06:32.143 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:06:32.143 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T13:06:32.171 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T13:06:32.171 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T13:06:32.182 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T13:06:32.211 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-03-30T13:06:32.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-03-30T13:06:32.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T13:06:32.322 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T13:06:32.388 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-03-30T13:06:32.390 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-03-30T13:06:32.475 INFO:teuthology.orchestra.run.smithi066.stdout:74 files removed 2024-03-30T13:06:32.503 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T13:06:32.634 INFO:teuthology.orchestra.run.smithi117.stdout:74 files removed 2024-03-30T13:06:32.660 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T13:06:33.208 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-03-30T13:06:33.441 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 172 kB/s | 83 kB 00:00 2024-03-30T13:06:33.584 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-03-30T13:06:33.925 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-03-30T13:06:34.068 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-03-30T13:06:34.301 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-30T13:06:34.545 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-30T13:06:34.878 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-30T13:06:36.112 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-30T13:06:36.412 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-03-30T13:06:37.411 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-30T13:06:47.452 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 749 kB/s | 8.0 MB 00:10 2024-03-30T13:06:50.070 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-30T13:06:54.637 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 6.1 MB 00:00 2024-03-30T13:06:56.305 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-30T13:06:57.148 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-03-30T13:07:02.323 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-30T13:07:04.962 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-30T13:07:04.962 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-30T13:07:04.977 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-30T13:07:04.978 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-30T13:07:04.979 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-30T13:07:04.980 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-30T13:07:04.982 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-30T13:07:04.985 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-30T13:07:04.986 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-30T13:07:05.074 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-03-30T13:07:05.087 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================ 2024-03-30T13:07:05.087 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-03-30T13:07:05.087 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================ 2024-03-30T13:07:05.087 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-2519.gac171039.el9 ceph 6.6 k 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-2519.gac171039.el9 ceph 5.4 M 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-2519.gac171039.el9 ceph 891 k 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2519.gac171039.el9 ceph 145 k 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-2519.gac171039.el9 ceph 1.5 M 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 141 k 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 3.5 M 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 7.4 M 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 49 k 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-2519.gac171039.el9 ceph 9.8 M 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-2519.gac171039.el9 ceph 48 M 2024-03-30T13:07:05.088 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 264 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 762 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-2519.gac171039.el9 ceph 221 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.0.0-2519.gac171039.el9 ceph 27 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 18 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-2519.gac171039.el9 ceph 32 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-2519.gac171039.el9 ceph 746 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-2519.gac171039.el9 ceph 127 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-30T13:07:05.089 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-2519.gac171039.el9 ceph 162 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-2519.gac171039.el9 ceph 321 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-2519.gac171039.el9 ceph 298 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-2519.gac171039.el9 ceph 99 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-2519.gac171039.el9 ceph 85 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-2519.gac171039.el9 ceph 3.1 M 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-2519.gac171039.el9 ceph 170 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-30T13:07:05.090 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-2519.gac171039.el9 ceph 3.4 M 2024-03-30T13:07:05.091 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-2519.gac171039.el9 ceph 3.1 M 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-30T13:07:05.092 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-2519.gac171039.el9 ceph 21 M 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 34 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-2519.gac171039.el9 ceph 2.4 M 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 249 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-2519.gac171039.el9 ceph 4.7 M 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-2519.gac171039.el9 ceph 17 M 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 16 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-2519.gac171039.el9 ceph 25 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-30T13:07:05.093 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-2519.gac171039.el9 ceph 164 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-2519.gac171039.el9 ceph 500 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-2519.gac171039.el9 ceph 5.1 M 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-30T13:07:05.094 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-30T13:07:05.095 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-2519.gac171039.el9 ceph 45 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-2519.gac171039.el9 ceph 134 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-30T13:07:05.096 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-30T13:07:05.097 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-30T13:07:05.098 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-30T13:07:05.099 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:07:05.100 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-03-30T13:07:05.101 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================ 2024-03-30T13:07:05.101 INFO:teuthology.orchestra.run.smithi066.stdout:Install 150 Packages 2024-03-30T13:07:05.101 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 24 Packages 2024-03-30T13:07:05.101 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:07:05.101 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 226 M 2024-03-30T13:07:05.102 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-03-30T13:07:05.716 INFO:teuthology.orchestra.run.smithi066.stdout:(1/174): ceph-19.0.0-2519.gac171039.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-03-30T13:07:05.925 INFO:teuthology.orchestra.run.smithi066.stdout:(2/174): ceph-fuse-19.0.0-2519.gac171039.el9.x8 4.2 MB/s | 891 kB 00:00 2024-03-30T13:07:05.983 INFO:teuthology.orchestra.run.smithi066.stdout:(3/174): ceph-immutable-object-cache-19.0.0-251 2.4 MB/s | 145 kB 00:00 2024-03-30T13:07:06.275 INFO:teuthology.orchestra.run.smithi066.stdout:(4/174): ceph-mds-19.0.0-2519.gac171039.el9.x86 8.1 MB/s | 2.4 MB 00:00 2024-03-30T13:07:06.465 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 691 kB/s | 19 MB 00:27 2024-03-30T13:07:06.475 INFO:teuthology.orchestra.run.smithi066.stdout:(5/174): ceph-mgr-19.0.0-2519.gac171039.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-03-30T13:07:07.067 INFO:teuthology.orchestra.run.smithi066.stdout:(6/174): ceph-mon-19.0.0-2519.gac171039.el9.x86 7.9 MB/s | 4.7 MB 00:00 2024-03-30T13:07:07.134 INFO:teuthology.orchestra.run.smithi066.stdout:(7/174): ceph-base-19.0.0-2519.gac171039.el9.x8 3.4 MB/s | 5.4 MB 00:01 2024-03-30T13:07:07.436 INFO:teuthology.orchestra.run.smithi066.stdout:(8/174): ceph-common-19.0.0-2519.gac171039.el9. 11 MB/s | 21 MB 00:01 2024-03-30T13:07:07.476 INFO:teuthology.orchestra.run.smithi066.stdout:(9/174): ceph-selinux-19.0.0-2519.gac171039.el9 632 kB/s | 25 kB 00:00 2024-03-30T13:07:08.585 INFO:teuthology.orchestra.run.smithi066.stdout:(10/174): ceph-osd-19.0.0-2519.gac171039.el9.x8 11 MB/s | 17 MB 00:01 2024-03-30T13:07:08.643 INFO:teuthology.orchestra.run.smithi066.stdout:(11/174): cephfs-mirror-19.0.0-2519.gac171039.e 3.7 MB/s | 221 kB 00:00 2024-03-30T13:07:08.702 INFO:teuthology.orchestra.run.smithi066.stdout:(12/174): cephfs-shell-19.0.0-2519.gac171039.el 464 kB/s | 27 kB 00:00 2024-03-30T13:07:08.753 INFO:teuthology.orchestra.run.smithi066.stdout:(13/174): libcephfs-devel-19.0.0-2519.gac171039 635 kB/s | 32 kB 00:00 2024-03-30T13:07:08.853 INFO:teuthology.orchestra.run.smithi066.stdout:(14/174): libcephfs2-19.0.0-2519.gac171039.el9. 7.3 MB/s | 746 kB 00:00 2024-03-30T13:07:08.911 INFO:teuthology.orchestra.run.smithi066.stdout:(15/174): libcephsqlite-19.0.0-2519.gac171039.e 2.7 MB/s | 164 kB 00:00 2024-03-30T13:07:08.978 INFO:teuthology.orchestra.run.smithi066.stdout:(16/174): librados-devel-19.0.0-2519.gac171039. 1.9 MB/s | 127 kB 00:00 2024-03-30T13:07:09.062 INFO:teuthology.orchestra.run.smithi066.stdout:(17/174): libradosstriper1-19.0.0-2519.gac17103 5.9 MB/s | 500 kB 00:00 2024-03-30T13:07:09.270 INFO:teuthology.orchestra.run.smithi066.stdout:(18/174): ceph-radosgw-19.0.0-2519.gac171039.el 4.6 MB/s | 9.8 MB 00:02 2024-03-30T13:07:09.387 INFO:teuthology.orchestra.run.smithi066.stdout:(19/174): python3-ceph-argparse-19.0.0-2519.gac 388 kB/s | 45 kB 00:00 2024-03-30T13:07:09.521 INFO:teuthology.orchestra.run.smithi066.stdout:(20/174): librgw2-19.0.0-2519.gac171039.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-30T13:07:09.546 INFO:teuthology.orchestra.run.smithi066.stdout:(21/174): python3-ceph-common-19.0.0-2519.gac17 847 kB/s | 134 kB 00:00 2024-03-30T13:07:09.580 INFO:teuthology.orchestra.run.smithi066.stdout:(22/174): python3-cephfs-19.0.0-2519.gac171039. 2.7 MB/s | 162 kB 00:00 2024-03-30T13:07:09.655 INFO:teuthology.orchestra.run.smithi066.stdout:(23/174): python3-rados-19.0.0-2519.gac171039.e 2.9 MB/s | 321 kB 00:00 2024-03-30T13:07:09.680 INFO:teuthology.orchestra.run.smithi066.stdout:(24/174): python3-rbd-19.0.0-2519.gac171039.el9 2.9 MB/s | 298 kB 00:00 2024-03-30T13:07:09.739 INFO:teuthology.orchestra.run.smithi066.stdout:(25/174): python3-rgw-19.0.0-2519.gac171039.el9 1.2 MB/s | 99 kB 00:00 2024-03-30T13:07:09.765 INFO:teuthology.orchestra.run.smithi066.stdout:(26/174): rbd-fuse-19.0.0-2519.gac171039.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-30T13:07:09.840 INFO:teuthology.orchestra.run.smithi066.stdout:(27/174): rbd-nbd-19.0.0-2519.gac171039.el9.x86 2.2 MB/s | 170 kB 00:00 2024-03-30T13:07:09.907 INFO:teuthology.orchestra.run.smithi066.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2519.g 508 kB/s | 34 kB 00:00 2024-03-30T13:07:09.974 INFO:teuthology.orchestra.run.smithi066.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2519.gac17103 2.1 MB/s | 141 kB 00:00 2024-03-30T13:07:10.307 INFO:teuthology.orchestra.run.smithi066.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2519.gac171 11 MB/s | 3.5 MB 00:00 2024-03-30T13:07:10.508 INFO:teuthology.orchestra.run.smithi066.stdout:(31/174): rbd-mirror-19.0.0-2519.gac171039.el9. 4.0 MB/s | 3.1 MB 00:00 2024-03-30T13:07:10.941 INFO:teuthology.orchestra.run.smithi066.stdout:(32/174): ceph-test-19.0.0-2519.gac171039.el9.x 14 MB/s | 48 MB 00:03 2024-03-30T13:07:10.966 INFO:teuthology.orchestra.run.smithi066.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2519.gac 543 kB/s | 249 kB 00:00 2024-03-30T13:07:10.992 INFO:teuthology.orchestra.run.smithi066.stdout:(34/174): ceph-mgr-rook-19.0.0-2519.gac171039.e 990 kB/s | 49 kB 00:00 2024-03-30T13:07:11.034 INFO:teuthology.orchestra.run.smithi066.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2519.ga 246 kB/s | 16 kB 00:00 2024-03-30T13:07:11.059 INFO:teuthology.orchestra.run.smithi066.stdout:(36/174): ceph-volume-19.0.0-2519.gac171039.el9 3.8 MB/s | 264 kB 00:00 2024-03-30T13:07:11.059 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.1 MB 00:01 2024-03-30T13:07:11.142 INFO:teuthology.orchestra.run.smithi066.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-03-30T13:07:11.168 INFO:teuthology.orchestra.run.smithi066.stdout:(38/174): cephfs-top-19.0.0-2519.gac171039.el9. 168 kB/s | 18 kB 00:00 2024-03-30T13:07:11.276 INFO:teuthology.orchestra.run.smithi066.stdout:(39/174): cephadm-19.0.0-2519.gac171039.el9.noa 3.1 MB/s | 762 kB 00:00 2024-03-30T13:07:11.368 INFO:teuthology.orchestra.run.smithi066.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 662 kB 00:00 2024-03-30T13:07:11.393 INFO:teuthology.orchestra.run.smithi066.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 917 kB/s | 106 kB 00:00 2024-03-30T13:07:11.479 INFO:teuthology.orchestra.run.smithi066.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 920 kB 00:00 2024-03-30T13:07:11.535 INFO:teuthology.orchestra.run.smithi066.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 307 kB 00:00 2024-03-30T13:07:11.561 INFO:teuthology.orchestra.run.smithi066.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 190 kB/s | 32 kB 00:00 2024-03-30T13:07:11.586 INFO:teuthology.orchestra.run.smithi066.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 345 kB/s | 17 kB 00:00 2024-03-30T13:07:11.661 INFO:teuthology.orchestra.run.smithi066.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-30T13:07:11.686 INFO:teuthology.orchestra.run.smithi066.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 80 kB/s | 10 kB 00:00 2024-03-30T13:07:11.712 INFO:teuthology.orchestra.run.smithi066.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-03-30T13:07:11.728 INFO:teuthology.orchestra.run.smithi066.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-30T13:07:11.754 INFO:teuthology.orchestra.run.smithi066.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-30T13:07:11.779 INFO:teuthology.orchestra.run.smithi066.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 8.4 MB/s | 578 kB 00:00 2024-03-30T13:07:11.796 INFO:teuthology.orchestra.run.smithi066.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-30T13:07:11.821 INFO:teuthology.orchestra.run.smithi066.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-30T13:07:11.846 INFO:teuthology.orchestra.run.smithi066.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-03-30T13:07:11.871 INFO:teuthology.orchestra.run.smithi066.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-30T13:07:11.897 INFO:teuthology.orchestra.run.smithi066.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 458 kB/s | 34 kB 00:00 2024-03-30T13:07:11.997 INFO:teuthology.orchestra.run.smithi066.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-30T13:07:12.048 INFO:teuthology.orchestra.run.smithi066.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 817 kB 00:00 2024-03-30T13:07:12.089 INFO:teuthology.orchestra.run.smithi066.stdout:(59/174): postgresql-13.14-1.el9.x86_64.rpm 8.1 MB/s | 1.5 MB 00:00 2024-03-30T13:07:12.117 INFO:teuthology.orchestra.run.smithi066.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 132 kB 00:00 2024-03-30T13:07:12.315 INFO:teuthology.orchestra.run.smithi066.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-30T13:07:12.348 INFO:teuthology.orchestra.run.smithi066.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.4 MB/s | 1.0 MB 00:00 2024-03-30T13:07:12.373 INFO:teuthology.orchestra.run.smithi066.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 305 kB/s | 78 kB 00:00 2024-03-30T13:07:12.449 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-03-30T13:07:12.516 INFO:teuthology.orchestra.run.smithi066.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-30T13:07:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 150 kB/s | 24 kB 00:00 2024-03-30T13:07:12.566 INFO:teuthology.orchestra.run.smithi066.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 96 kB/s | 18 kB 00:00 2024-03-30T13:07:12.591 INFO:teuthology.orchestra.run.smithi066.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-03-30T13:07:12.616 INFO:teuthology.orchestra.run.smithi066.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-30T13:07:12.733 INFO:teuthology.orchestra.run.smithi066.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-30T13:07:12.758 INFO:teuthology.orchestra.run.smithi066.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-30T13:07:12.784 INFO:teuthology.orchestra.run.smithi066.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-30T13:07:12.809 INFO:teuthology.orchestra.run.smithi066.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 69 kB 00:00 2024-03-30T13:07:12.834 INFO:teuthology.orchestra.run.smithi066.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-03-30T13:07:12.860 INFO:teuthology.orchestra.run.smithi066.stdout:(74/174): python3-prettytable-0.7.2-27.el9.noar 548 kB/s | 41 kB 00:00 2024-03-30T13:07:12.885 INFO:teuthology.orchestra.run.smithi066.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-30T13:07:12.910 INFO:teuthology.orchestra.run.smithi066.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-30T13:07:12.935 INFO:teuthology.orchestra.run.smithi066.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 956 kB/s | 47 kB 00:00 2024-03-30T13:07:12.960 INFO:teuthology.orchestra.run.smithi066.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 574 kB/s | 43 kB 00:00 2024-03-30T13:07:13.327 INFO:teuthology.orchestra.run.smithi066.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-30T13:07:13.352 INFO:teuthology.orchestra.run.smithi066.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 106 kB/s | 41 kB 00:00 2024-03-30T13:07:13.378 INFO:teuthology.orchestra.run.smithi066.stdout:(81/174): python3-toml-0.10.2-6.el9.noarch.rpm 93 kB/s | 39 kB 00:00 2024-03-30T13:07:13.403 INFO:teuthology.orchestra.run.smithi066.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-03-30T13:07:13.428 INFO:teuthology.orchestra.run.smithi066.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 464 kB/s | 35 kB 00:00 2024-03-30T13:07:13.454 INFO:teuthology.orchestra.run.smithi066.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-30T13:07:13.479 INFO:teuthology.orchestra.run.smithi066.stdout:(85/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 615 kB/s | 46 kB 00:00 2024-03-30T13:07:13.504 INFO:teuthology.orchestra.run.smithi066.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 752 kB/s | 56 kB 00:00 2024-03-30T13:07:13.529 INFO:teuthology.orchestra.run.smithi066.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 834 kB/s | 63 kB 00:00 2024-03-30T13:07:13.554 INFO:teuthology.orchestra.run.smithi066.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-30T13:07:13.580 INFO:teuthology.orchestra.run.smithi066.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-30T13:07:13.605 INFO:teuthology.orchestra.run.smithi066.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-30T13:07:13.630 INFO:teuthology.orchestra.run.smithi066.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-30T13:07:13.664 INFO:teuthology.orchestra.run.smithi066.stdout:(92/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-30T13:07:13.691 INFO:teuthology.orchestra.run.smithi066.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-30T13:07:13.716 INFO:teuthology.orchestra.run.smithi066.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-30T13:07:13.842 INFO:teuthology.orchestra.run.smithi066.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-30T13:07:13.867 INFO:teuthology.orchestra.run.smithi066.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-30T13:07:13.892 INFO:teuthology.orchestra.run.smithi066.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-30T13:07:13.934 INFO:teuthology.orchestra.run.smithi066.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-03-30T13:07:14.075 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-30T13:07:14.518 INFO:teuthology.orchestra.run.smithi066.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 33 kB/s | 21 kB 00:00 2024-03-30T13:07:14.551 INFO:teuthology.orchestra.run.smithi066.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 39 kB/s | 25 kB 00:00 2024-03-30T13:07:15.601 INFO:teuthology.orchestra.run.smithi066.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 226 kB/s | 245 kB 00:01 2024-03-30T13:07:15.643 INFO:teuthology.orchestra.run.smithi066.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 227 kB/s | 248 kB 00:01 2024-03-30T13:07:15.735 INFO:teuthology.orchestra.run.smithi066.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-03-30T13:07:15.902 INFO:teuthology.orchestra.run.smithi066.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-03-30T13:07:16.069 INFO:teuthology.orchestra.run.smithi066.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-03-30T13:07:16.127 INFO:teuthology.orchestra.run.smithi066.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-03-30T13:07:16.169 INFO:teuthology.orchestra.run.smithi066.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-03-30T13:07:16.320 INFO:teuthology.orchestra.run.smithi066.stdout:(108/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 131 kB/s | 94 kB 00:00 2024-03-30T13:07:16.422 INFO:teuthology.orchestra.run.smithi066.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-03-30T13:07:16.503 INFO:teuthology.orchestra.run.smithi066.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-03-30T13:07:16.570 INFO:teuthology.orchestra.run.smithi066.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-30T13:07:16.646 INFO:teuthology.orchestra.run.smithi066.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-30T13:07:16.838 INFO:teuthology.orchestra.run.smithi066.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-03-30T13:07:16.896 INFO:teuthology.orchestra.run.smithi066.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-03-30T13:07:16.938 INFO:teuthology.orchestra.run.smithi066.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-30T13:07:16.980 INFO:teuthology.orchestra.run.smithi066.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 775 kB/s | 32 kB 00:00 2024-03-30T13:07:17.022 INFO:teuthology.orchestra.run.smithi066.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 361 kB/s | 15 kB 00:00 2024-03-30T13:07:17.064 INFO:teuthology.orchestra.run.smithi066.stdout:(118/174): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-03-30T13:07:17.089 INFO:teuthology.orchestra.run.smithi066.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-30T13:07:17.156 INFO:teuthology.orchestra.run.smithi066.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-30T13:07:17.198 INFO:teuthology.orchestra.run.smithi066.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-30T13:07:17.223 INFO:teuthology.orchestra.run.smithi066.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-03-30T13:07:17.265 INFO:teuthology.orchestra.run.smithi066.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-30T13:07:17.324 INFO:teuthology.orchestra.run.smithi066.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-30T13:07:17.366 INFO:teuthology.orchestra.run.smithi066.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-03-30T13:07:17.408 INFO:teuthology.orchestra.run.smithi066.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-03-30T13:07:17.450 INFO:teuthology.orchestra.run.smithi066.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 558 kB/s | 23 kB 00:00 2024-03-30T13:07:17.492 INFO:teuthology.orchestra.run.smithi066.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 469 kB/s | 19 kB 00:00 2024-03-30T13:07:17.534 INFO:teuthology.orchestra.run.smithi066.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 472 kB/s | 20 kB 00:00 2024-03-30T13:07:17.692 INFO:teuthology.orchestra.run.smithi066.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-03-30T13:07:17.734 INFO:teuthology.orchestra.run.smithi066.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-30T13:07:17.768 INFO:teuthology.orchestra.run.smithi066.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-03-30T13:07:17.793 INFO:teuthology.orchestra.run.smithi066.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-30T13:07:17.819 INFO:teuthology.orchestra.run.smithi066.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-30T13:07:17.854 INFO:teuthology.orchestra.run.smithi066.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 494 kB/s | 16 kB 00:00 2024-03-30T13:07:17.877 INFO:teuthology.orchestra.run.smithi066.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-30T13:07:17.894 INFO:teuthology.orchestra.run.smithi066.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-03-30T13:07:17.928 INFO:teuthology.orchestra.run.smithi066.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 929 kB/s | 31 kB 00:00 2024-03-30T13:07:17.945 INFO:teuthology.orchestra.run.smithi066.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-03-30T13:07:17.987 INFO:teuthology.orchestra.run.smithi066.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-03-30T13:07:18.004 INFO:teuthology.orchestra.run.smithi066.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-30T13:07:18.021 INFO:teuthology.orchestra.run.smithi066.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-03-30T13:07:18.038 INFO:teuthology.orchestra.run.smithi066.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-03-30T13:07:18.079 INFO:teuthology.orchestra.run.smithi066.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-03-30T13:07:18.105 INFO:teuthology.orchestra.run.smithi066.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-30T13:07:18.147 INFO:teuthology.orchestra.run.smithi066.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 481 kB/s | 20 kB 00:00 2024-03-30T13:07:18.189 INFO:teuthology.orchestra.run.smithi066.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-03-30T13:07:18.231 INFO:teuthology.orchestra.run.smithi066.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-30T13:07:18.439 INFO:teuthology.orchestra.run.smithi066.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-30T13:07:18.514 INFO:teuthology.orchestra.run.smithi066.stdout:(150/174): librados2-19.0.0-2519.gac171039.el9. 12 MB/s | 3.4 MB 00:00 2024-03-30T13:07:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 18 MB/s | 2.3 MB 00:00 2024-03-30T13:07:18.732 INFO:teuthology.orchestra.run.smithi066.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-03-30T13:07:18.765 INFO:teuthology.orchestra.run.smithi066.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-30T13:07:18.807 INFO:teuthology.orchestra.run.smithi066.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-30T13:07:18.858 INFO:teuthology.orchestra.run.smithi066.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-30T13:07:18.941 INFO:teuthology.orchestra.run.smithi066.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 856 kB/s | 71 kB 00:00 2024-03-30T13:07:18.983 INFO:teuthology.orchestra.run.smithi066.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 879 kB/s | 36 kB 00:00 2024-03-30T13:07:19.025 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-30T13:07:19.034 INFO:teuthology.orchestra.run.smithi066.stdout:(158/174): librbd1-19.0.0-2519.gac171039.el9.x8 5.3 MB/s | 3.1 MB 00:00 2024-03-30T13:07:19.076 INFO:teuthology.orchestra.run.smithi066.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-30T13:07:19.118 INFO:teuthology.orchestra.run.smithi066.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-03-30T13:07:19.143 INFO:teuthology.orchestra.run.smithi066.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-30T13:07:19.168 INFO:teuthology.orchestra.run.smithi066.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-30T13:07:19.194 INFO:teuthology.orchestra.run.smithi066.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-30T13:07:19.218 INFO:teuthology.orchestra.run.smithi066.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-30T13:07:19.244 INFO:teuthology.orchestra.run.smithi066.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-30T13:07:19.269 INFO:teuthology.orchestra.run.smithi066.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-03-30T13:07:19.328 INFO:teuthology.orchestra.run.smithi066.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-03-30T13:07:19.411 INFO:teuthology.orchestra.run.smithi066.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 27 MB/s | 2.3 MB 00:00 2024-03-30T13:07:19.453 INFO:teuthology.orchestra.run.smithi066.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-03-30T13:07:19.478 INFO:teuthology.orchestra.run.smithi066.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-30T13:07:19.504 INFO:teuthology.orchestra.run.smithi066.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 621 kB/s | 31 kB 00:00 2024-03-30T13:07:19.537 INFO:teuthology.orchestra.run.smithi066.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-30T13:07:20.263 INFO:teuthology.orchestra.run.smithi066.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 98 kB/s | 74 kB 00:00 2024-03-30T13:07:20.521 INFO:teuthology.orchestra.run.smithi066.stdout:(174/174): openssl-devel-3.0.7-27.el9.x86_64.rp 642 kB/s | 4.1 MB 00:06 2024-03-30T13:07:20.527 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-03-30T13:07:20.527 INFO:teuthology.orchestra.run.smithi066.stdout:Total 15 MB/s | 226 MB 00:15 2024-03-30T13:07:20.922 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-03-30T13:07:21.221 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-03-30T13:07:21.221 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-03-30T13:07:21.542 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-30T13:07:21.543 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-30T13:07:21.557 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-30T13:07:21.558 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-30T13:07:21.559 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-30T13:07:21.560 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-30T13:07:21.562 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-30T13:07:21.565 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-30T13:07:21.566 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-30T13:07:21.644 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-03-30T13:07:21.662 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-03-30T13:07:21.662 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.0.0-2519.gac171039.el9 ceph 6.6 k 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.0.0-2519.gac171039.el9 ceph 5.4 M 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.0.0-2519.gac171039.el9 ceph 891 k 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2519.gac171039.el9 ceph 145 k 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.0.0-2519.gac171039.el9 ceph 1.5 M 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 141 k 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 3.5 M 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 7.4 M 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 49 k 2024-03-30T13:07:21.663 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.0.0-2519.gac171039.el9 ceph 9.8 M 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.0.0-2519.gac171039.el9 ceph 48 M 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 264 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 762 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.0.0-2519.gac171039.el9 ceph 221 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.0.0-2519.gac171039.el9 ceph 27 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 18 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.0.0-2519.gac171039.el9 ceph 32 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.0.0-2519.gac171039.el9 ceph 746 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.0.0-2519.gac171039.el9 ceph 127 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-30T13:07:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.0.0-2519.gac171039.el9 ceph 162 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.0.0-2519.gac171039.el9 ceph 321 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.0.0-2519.gac171039.el9 ceph 298 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.0.0-2519.gac171039.el9 ceph 99 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.0.0-2519.gac171039.el9 ceph 85 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.0.0-2519.gac171039.el9 ceph 3.1 M 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.0.0-2519.gac171039.el9 ceph 170 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-30T13:07:21.665 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.0.0-2519.gac171039.el9 ceph 3.4 M 2024-03-30T13:07:21.666 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.0.0-2519.gac171039.el9 ceph 3.1 M 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-03-30T13:07:21.667 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.0.0-2519.gac171039.el9 ceph 21 M 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 34 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.0.0-2519.gac171039.el9 ceph 2.4 M 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 249 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.0.0-2519.gac171039.el9 ceph 4.7 M 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.0.0-2519.gac171039.el9 ceph 17 M 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2519.gac171039.el9 ceph-noarch 16 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.0.0-2519.gac171039.el9 ceph 25 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-30T13:07:21.668 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.0.0-2519.gac171039.el9 ceph 164 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.0.0-2519.gac171039.el9 ceph 500 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.0.0-2519.gac171039.el9 ceph 5.1 M 2024-03-30T13:07:21.669 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-30T13:07:21.670 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.0.0-2519.gac171039.el9 ceph 45 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.0.0-2519.gac171039.el9 ceph 134 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-30T13:07:21.671 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-30T13:07:21.672 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-30T13:07:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-30T13:07:21.674 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-30T13:07:21.675 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout:Install 150 Packages 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 24 Packages 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 226 M 2024-03-30T13:07:21.676 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-03-30T13:07:22.843 INFO:teuthology.orchestra.run.smithi117.stdout:(1/174): ceph-19.0.0-2519.gac171039.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-03-30T13:07:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:(2/174): ceph-base-19.0.0-2519.gac171039.el9.x8 11 MB/s | 5.4 MB 00:00 2024-03-30T13:07:23.026 INFO:teuthology.orchestra.run.smithi117.stdout:(3/174): ceph-immutable-object-cache-19.0.0-251 2.8 MB/s | 145 kB 00:00 2024-03-30T13:07:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:(4/174): ceph-mds-19.0.0-2519.gac171039.el9.x86 16 MB/s | 2.4 MB 00:00 2024-03-30T13:07:23.277 INFO:teuthology.orchestra.run.smithi117.stdout:(5/174): ceph-mgr-19.0.0-2519.gac171039.el9.x86 15 MB/s | 1.5 MB 00:00 2024-03-30T13:07:23.363 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-03-30T13:07:23.363 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-03-30T13:07:23.552 INFO:teuthology.orchestra.run.smithi117.stdout:(6/174): ceph-mon-19.0.0-2519.gac171039.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-30T13:07:23.686 INFO:teuthology.orchestra.run.smithi117.stdout:(7/174): ceph-fuse-19.0.0-2519.gac171039.el9.x8 1.0 MB/s | 891 kB 00:00 2024-03-30T13:07:24.494 INFO:teuthology.orchestra.run.smithi117.stdout:(8/174): ceph-osd-19.0.0-2519.gac171039.el9.x86 18 MB/s | 17 MB 00:00 2024-03-30T13:07:24.536 INFO:teuthology.orchestra.run.smithi117.stdout:(9/174): ceph-selinux-19.0.0-2519.gac171039.el9 606 kB/s | 25 kB 00:00 2024-03-30T13:07:26.261 INFO:teuthology.orchestra.run.smithi117.stdout:(10/174): ceph-radosgw-19.0.0-2519.gac171039.el 3.8 MB/s | 9.8 MB 00:02 2024-03-30T13:07:26.369 INFO:teuthology.orchestra.run.smithi117.stdout:(11/174): cephfs-mirror-19.0.0-2519.gac171039.e 2.0 MB/s | 221 kB 00:00 2024-03-30T13:07:26.478 INFO:teuthology.orchestra.run.smithi117.stdout:(12/174): cephfs-shell-19.0.0-2519.gac171039.el 250 kB/s | 27 kB 00:00 2024-03-30T13:07:26.587 INFO:teuthology.orchestra.run.smithi117.stdout:(13/174): libcephfs-devel-19.0.0-2519.gac171039 293 kB/s | 32 kB 00:00 2024-03-30T13:07:26.962 INFO:teuthology.orchestra.run.smithi117.stdout:(14/174): ceph-common-19.0.0-2519.gac171039.el9 4.6 MB/s | 21 MB 00:04 2024-03-30T13:07:26.996 INFO:teuthology.orchestra.run.smithi117.stdout:(15/174): libcephfs2-19.0.0-2519.gac171039.el9. 1.8 MB/s | 746 kB 00:00 2024-03-30T13:07:27.037 INFO:teuthology.orchestra.run.smithi117.stdout:(16/174): libcephsqlite-19.0.0-2519.gac171039.e 2.1 MB/s | 164 kB 00:00 2024-03-30T13:07:27.471 INFO:teuthology.orchestra.run.smithi117.stdout:(17/174): ceph-test-19.0.0-2519.gac171039.el9.x 16 MB/s | 48 MB 00:02 2024-03-30T13:07:27.488 INFO:teuthology.orchestra.run.smithi117.stdout:(18/174): librados-devel-19.0.0-2519.gac171039. 257 kB/s | 127 kB 00:00 2024-03-30T13:07:27.521 INFO:teuthology.orchestra.run.smithi117.stdout:(19/174): libradosstriper1-19.0.0-2519.gac17103 1.0 MB/s | 500 kB 00:00 2024-03-30T13:07:27.563 INFO:teuthology.orchestra.run.smithi117.stdout:(20/174): python3-ceph-argparse-19.0.0-2519.gac 600 kB/s | 45 kB 00:00 2024-03-30T13:07:27.614 INFO:teuthology.orchestra.run.smithi117.stdout:(21/174): python3-ceph-common-19.0.0-2519.gac17 1.4 MB/s | 134 kB 00:00 2024-03-30T13:07:27.681 INFO:teuthology.orchestra.run.smithi117.stdout:(22/174): python3-cephfs-19.0.0-2519.gac171039. 1.4 MB/s | 162 kB 00:00 2024-03-30T13:07:27.773 INFO:teuthology.orchestra.run.smithi117.stdout:(23/174): librgw2-19.0.0-2519.gac171039.el9.x86 17 MB/s | 5.1 MB 00:00 2024-03-30T13:07:27.798 INFO:teuthology.orchestra.run.smithi117.stdout:(24/174): python3-rados-19.0.0-2519.gac171039.e 1.7 MB/s | 321 kB 00:00 2024-03-30T13:07:27.823 INFO:teuthology.orchestra.run.smithi117.stdout:(25/174): python3-rgw-19.0.0-2519.gac171039.el9 1.9 MB/s | 99 kB 00:00 2024-03-30T13:07:27.849 INFO:teuthology.orchestra.run.smithi117.stdout:(26/174): python3-rbd-19.0.0-2519.gac171039.el9 1.7 MB/s | 298 kB 00:00 2024-03-30T13:07:27.874 INFO:teuthology.orchestra.run.smithi117.stdout:(27/174): rbd-fuse-19.0.0-2519.gac171039.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-30T13:07:27.933 INFO:teuthology.orchestra.run.smithi117.stdout:(28/174): rbd-nbd-19.0.0-2519.gac171039.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-30T13:07:27.958 INFO:teuthology.orchestra.run.smithi117.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2519.g 406 kB/s | 34 kB 00:00 2024-03-30T13:07:28.008 INFO:teuthology.orchestra.run.smithi117.stdout:(30/174): rbd-mirror-19.0.0-2519.gac171039.el9. 17 MB/s | 3.1 MB 00:00 2024-03-30T13:07:28.033 INFO:teuthology.orchestra.run.smithi117.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2519.gac17103 1.4 MB/s | 141 kB 00:00 2024-03-30T13:07:28.142 INFO:teuthology.orchestra.run.smithi117.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2519.gac 2.2 MB/s | 249 kB 00:00 2024-03-30T13:07:28.225 INFO:teuthology.orchestra.run.smithi117.stdout:(33/174): ceph-mgr-rook-19.0.0-2519.gac171039.e 592 kB/s | 49 kB 00:00 2024-03-30T13:07:28.309 INFO:teuthology.orchestra.run.smithi117.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2519.ga 198 kB/s | 16 kB 00:00 2024-03-30T13:07:28.434 INFO:teuthology.orchestra.run.smithi117.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-03-30T13:07:28.459 INFO:teuthology.orchestra.run.smithi117.stdout:(36/174): ceph-volume-19.0.0-2519.gac171039.el9 1.7 MB/s | 264 kB 00:00 2024-03-30T13:07:28.510 INFO:teuthology.orchestra.run.smithi117.stdout:(37/174): cephadm-19.0.0-2519.gac171039.el9.noa 9.9 MB/s | 762 kB 00:00 2024-03-30T13:07:28.535 INFO:teuthology.orchestra.run.smithi117.stdout:(38/174): cephfs-top-19.0.0-2519.gac171039.el9. 242 kB/s | 18 kB 00:00 2024-03-30T13:07:28.594 INFO:teuthology.orchestra.run.smithi117.stdout:(39/174): ceph-mgr-dashboard-19.0.0-2519.gac171 5.6 MB/s | 3.5 MB 00:00 2024-03-30T13:07:28.777 INFO:teuthology.orchestra.run.smithi117.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 106 kB 00:00 2024-03-30T13:07:28.802 INFO:teuthology.orchestra.run.smithi117.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-30T13:07:28.837 INFO:teuthology.orchestra.run.smithi117.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2024-03-30T13:07:28.861 INFO:teuthology.orchestra.run.smithi117.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 307 kB 00:00 2024-03-30T13:07:28.886 INFO:teuthology.orchestra.run.smithi117.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 377 kB/s | 32 kB 00:00 2024-03-30T13:07:28.953 INFO:teuthology.orchestra.run.smithi117.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-30T13:07:28.979 INFO:teuthology.orchestra.run.smithi117.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 191 kB/s | 17 kB 00:00 2024-03-30T13:07:29.004 INFO:teuthology.orchestra.run.smithi117.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 88 kB/s | 10 kB 00:00 2024-03-30T13:07:29.029 INFO:teuthology.orchestra.run.smithi117.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-30T13:07:29.054 INFO:teuthology.orchestra.run.smithi117.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 889 kB/s | 44 kB 00:00 2024-03-30T13:07:29.079 INFO:teuthology.orchestra.run.smithi117.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-30T13:07:29.113 INFO:teuthology.orchestra.run.smithi117.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-30T13:07:29.138 INFO:teuthology.orchestra.run.smithi117.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 805 kB/s | 46 kB 00:00 2024-03-30T13:07:29.163 INFO:teuthology.orchestra.run.smithi117.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-30T13:07:29.189 INFO:teuthology.orchestra.run.smithi117.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-30T13:07:29.214 INFO:teuthology.orchestra.run.smithi117.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-30T13:07:29.239 INFO:teuthology.orchestra.run.smithi117.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-30T13:07:29.381 INFO:teuthology.orchestra.run.smithi117.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-30T13:07:29.432 INFO:teuthology.orchestra.run.smithi117.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.5 MB 00:00 2024-03-30T13:07:29.465 INFO:teuthology.orchestra.run.smithi117.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 817 kB 00:00 2024-03-30T13:07:29.491 INFO:teuthology.orchestra.run.smithi117.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 132 kB 00:00 2024-03-30T13:07:29.657 INFO:teuthology.orchestra.run.smithi117.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-03-30T13:07:29.674 INFO:teuthology.orchestra.run.smithi117.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 430 kB/s | 78 kB 00:00 2024-03-30T13:07:29.708 INFO:teuthology.orchestra.run.smithi117.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-03-30T13:07:29.875 INFO:teuthology.orchestra.run.smithi117.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-30T13:07:29.891 INFO:teuthology.orchestra.run.smithi117.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 114 kB/s | 24 kB 00:00 2024-03-30T13:07:29.909 INFO:teuthology.orchestra.run.smithi117.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 91 kB/s | 18 kB 00:00 2024-03-30T13:07:29.933 INFO:teuthology.orchestra.run.smithi117.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-30T13:07:30.026 INFO:teuthology.orchestra.run.smithi117.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 271 kB/s | 32 kB 00:00 2024-03-30T13:07:30.092 INFO:teuthology.orchestra.run.smithi117.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-30T13:07:30.118 INFO:teuthology.orchestra.run.smithi117.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-30T13:07:30.146 INFO:teuthology.orchestra.run.smithi117.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 174 kB 00:00 2024-03-30T13:07:30.168 INFO:teuthology.orchestra.run.smithi117.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 828 kB/s | 41 kB 00:00 2024-03-30T13:07:30.193 INFO:teuthology.orchestra.run.smithi117.stdout:(73/174): python3-packaging-20.9-5.el9.noarch.r 913 kB/s | 69 kB 00:00 2024-03-30T13:07:30.218 INFO:teuthology.orchestra.run.smithi117.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-30T13:07:30.244 INFO:teuthology.orchestra.run.smithi117.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-30T13:07:30.269 INFO:teuthology.orchestra.run.smithi117.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-30T13:07:30.294 INFO:teuthology.orchestra.run.smithi117.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 630 kB/s | 47 kB 00:00 2024-03-30T13:07:30.320 INFO:teuthology.orchestra.run.smithi117.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 572 kB/s | 43 kB 00:00 2024-03-30T13:07:30.420 INFO:teuthology.orchestra.run.smithi117.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 309 kB/s | 39 kB 00:00 2024-03-30T13:07:30.487 INFO:teuthology.orchestra.run.smithi117.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 248 kB/s | 41 kB 00:00 2024-03-30T13:07:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-03-30T13:07:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-30T13:07:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-30T13:07:30.729 INFO:teuthology.orchestra.run.smithi117.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-03-30T13:07:30.745 INFO:teuthology.orchestra.run.smithi117.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 680 kB/s | 220 kB 00:00 2024-03-30T13:07:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-30T13:07:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-30T13:07:30.771 INFO:teuthology.orchestra.run.smithi117.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 123 kB/s | 35 kB 00:00 2024-03-30T13:07:30.796 INFO:teuthology.orchestra.run.smithi117.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 698 kB/s | 46 kB 00:00 2024-03-30T13:07:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-30T13:07:30.821 INFO:teuthology.orchestra.run.smithi117.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-30T13:07:30.846 INFO:teuthology.orchestra.run.smithi117.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-30T13:07:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-30T13:07:30.871 INFO:teuthology.orchestra.run.smithi117.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 828 kB/s | 63 kB 00:00 2024-03-30T13:07:30.897 INFO:teuthology.orchestra.run.smithi117.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-03-30T13:07:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-30T13:07:30.913 INFO:teuthology.orchestra.run.smithi117.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-30T13:07:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-30T13:07:30.939 INFO:teuthology.orchestra.run.smithi117.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-30T13:07:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-30T13:07:30.989 INFO:teuthology.orchestra.run.smithi117.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-30T13:07:31.023 INFO:teuthology.orchestra.run.smithi117.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-30T13:07:31.048 INFO:teuthology.orchestra.run.smithi117.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-03-30T13:07:31.074 INFO:teuthology.orchestra.run.smithi117.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-30T13:07:31.274 INFO:teuthology.orchestra.run.smithi117.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 832 kB/s | 187 kB 00:00 2024-03-30T13:07:31.299 INFO:teuthology.orchestra.run.smithi117.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 867 kB/s | 195 kB 00:00 2024-03-30T13:07:31.327 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-30T13:07:31.332 INFO:teuthology.orchestra.run.smithi117.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-30T13:07:31.375 INFO:teuthology.orchestra.run.smithi117.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-03-30T13:07:31.380 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-2519.gac171039.el9.x86_64 10/198 2024-03-30T13:07:31.400 INFO:teuthology.orchestra.run.smithi117.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-03-30T13:07:31.442 INFO:teuthology.orchestra.run.smithi117.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 233 kB/s | 25 kB 00:00 2024-03-30T13:07:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-2519.gac171039.el9.x86_64 10/198 2024-03-30T13:07:31.517 INFO:teuthology.orchestra.run.smithi117.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.1 MB/s | 245 kB 00:00 2024-03-30T13:07:31.548 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 11/198 2024-03-30T13:07:31.559 INFO:teuthology.orchestra.run.smithi117.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.1 MB/s | 248 kB 00:00 2024-03-30T13:07:31.576 INFO:teuthology.orchestra.run.smithi117.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-30T13:07:31.640 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 11/198 2024-03-30T13:07:31.668 INFO:teuthology.orchestra.run.smithi117.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-30T13:07:31.673 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-2519.gac171039.el9.x86_64 12/198 2024-03-30T13:07:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-30T13:07:31.793 INFO:teuthology.orchestra.run.smithi117.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.9 MB/s | 4.1 MB 00:00 2024-03-30T13:07:31.835 INFO:teuthology.orchestra.run.smithi117.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-03-30T13:07:31.905 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-30T13:07:32.026 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-30T13:07:32.053 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 16/198 2024-03-30T13:07:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 16/198 2024-03-30T13:07:32.252 INFO:teuthology.orchestra.run.smithi117.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 940 kB/s | 548 kB 00:00 2024-03-30T13:07:32.303 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-2519.gac171039.el9.x86 17/198 2024-03-30T13:07:32.369 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2519.gac171039.el9.x86 17/198 2024-03-30T13:07:32.394 INFO:teuthology.orchestra.run.smithi117.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 516 kB/s | 309 kB 00:00 2024-03-30T13:07:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 34 kB/s | 19 kB 00:00 2024-03-30T13:07:32.416 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-30T13:07:32.439 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-30T13:07:32.456 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-30T13:07:32.607 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-30T13:07:32.637 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-30T13:07:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-30T13:07:32.859 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-30T13:07:32.995 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-30T13:07:33.047 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-30T13:07:33.143 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-30T13:07:33.198 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-30T13:07:33.261 INFO:teuthology.orchestra.run.smithi117.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-03-30T13:07:33.286 INFO:teuthology.orchestra.run.smithi117.stdout:(111/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 29 kB/s | 26 kB 00:00 2024-03-30T13:07:33.311 INFO:teuthology.orchestra.run.smithi117.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 54 kB/s | 49 kB 00:00 2024-03-30T13:07:33.337 INFO:teuthology.orchestra.run.smithi117.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-03-30T13:07:33.362 INFO:teuthology.orchestra.run.smithi117.stdout:(114/174): libunwind-1.6.2-1.el9.x86_64.rpm 895 kB/s | 67 kB 00:00 2024-03-30T13:07:33.425 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-30T13:07:33.462 INFO:teuthology.orchestra.run.smithi117.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 838 kB 00:00 2024-03-30T13:07:33.529 INFO:teuthology.orchestra.run.smithi117.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-03-30T13:07:33.554 INFO:teuthology.orchestra.run.smithi117.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 226 kB/s | 43 kB 00:00 2024-03-30T13:07:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-30T13:07:33.580 INFO:teuthology.orchestra.run.smithi117.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-03-30T13:07:33.605 INFO:teuthology.orchestra.run.smithi117.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-30T13:07:33.630 INFO:teuthology.orchestra.run.smithi117.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-03-30T13:07:33.651 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-30T13:07:33.664 INFO:teuthology.orchestra.run.smithi117.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-30T13:07:33.686 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-30T13:07:33.689 INFO:teuthology.orchestra.run.smithi117.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-30T13:07:33.713 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-30T13:07:33.722 INFO:teuthology.orchestra.run.smithi117.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-03-30T13:07:33.748 INFO:teuthology.orchestra.run.smithi117.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-30T13:07:33.757 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-30T13:07:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-30T13:07:33.886 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-30T13:07:33.923 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-30T13:07:33.966 INFO:teuthology.orchestra.run.smithi117.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-03-30T13:07:33.990 INFO:teuthology.orchestra.run.smithi117.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-03-30T13:07:33.999 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-30T13:07:34.007 INFO:teuthology.orchestra.run.smithi117.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 41 kB/s | 11 kB 00:00 2024-03-30T13:07:34.032 INFO:teuthology.orchestra.run.smithi117.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-03-30T13:07:34.039 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-30T13:07:34.074 INFO:teuthology.orchestra.run.smithi117.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 348 kB/s | 23 kB 00:00 2024-03-30T13:07:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-30T13:07:34.091 INFO:teuthology.orchestra.run.smithi117.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-03-30T13:07:34.116 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-30T13:07:34.116 INFO:teuthology.orchestra.run.smithi117.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-03-30T13:07:34.143 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-30T13:07:34.192 INFO:teuthology.orchestra.run.smithi117.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-03-30T13:07:34.216 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-30T13:07:34.216 INFO:teuthology.orchestra.run.smithi117.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-03-30T13:07:34.242 INFO:teuthology.orchestra.run.smithi117.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 627 kB/s | 79 kB 00:00 2024-03-30T13:07:34.247 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-2519.gac171039.el9. 39/198 2024-03-30T13:07:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 866 kB/s | 58 kB 00:00 2024-03-30T13:07:34.291 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-2519.gac171039.el 40/198 2024-03-30T13:07:34.292 INFO:teuthology.orchestra.run.smithi117.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-30T13:07:34.309 INFO:teuthology.orchestra.run.smithi117.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-03-30T13:07:34.319 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-2519.gac171039.el9.x86_6 41/198 2024-03-30T13:07:34.334 INFO:teuthology.orchestra.run.smithi117.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-30T13:07:34.351 INFO:teuthology.orchestra.run.smithi117.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-03-30T13:07:34.368 INFO:teuthology.orchestra.run.smithi117.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 927 kB/s | 31 kB 00:00 2024-03-30T13:07:34.393 INFO:teuthology.orchestra.run.smithi117.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-03-30T13:07:34.394 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-30T13:07:34.410 INFO:teuthology.orchestra.run.smithi117.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-30T13:07:34.430 INFO:teuthology.orchestra.run.smithi117.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-03-30T13:07:34.442 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-30T13:07:34.444 INFO:teuthology.orchestra.run.smithi117.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-30T13:07:34.478 INFO:teuthology.orchestra.run.smithi117.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-30T13:07:34.503 INFO:teuthology.orchestra.run.smithi117.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-30T13:07:34.520 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-30T13:07:34.528 INFO:teuthology.orchestra.run.smithi117.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-30T13:07:34.583 INFO:teuthology.orchestra.run.smithi117.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 192 kB/s | 20 kB 00:00 2024-03-30T13:07:34.612 INFO:teuthology.orchestra.run.smithi117.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-30T13:07:34.667 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-30T13:07:34.687 INFO:teuthology.orchestra.run.smithi117.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-30T13:07:34.722 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-30T13:07:34.804 INFO:teuthology.orchestra.run.smithi117.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-03-30T13:07:34.879 INFO:teuthology.orchestra.run.smithi117.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-30T13:07:34.913 INFO:teuthology.orchestra.run.smithi117.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-30T13:07:34.955 INFO:teuthology.orchestra.run.smithi117.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-30T13:07:35.014 INFO:teuthology.orchestra.run.smithi117.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-30T13:07:35.189 INFO:teuthology.orchestra.run.smithi117.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 408 kB/s | 71 kB 00:00 2024-03-30T13:07:35.248 INFO:teuthology.orchestra.run.smithi117.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 621 kB/s | 36 kB 00:00 2024-03-30T13:07:35.275 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-30T13:07:35.351 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-30T13:07:35.435 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-30T13:07:35.439 INFO:teuthology.orchestra.run.smithi117.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.0 MB/s | 192 kB 00:00 2024-03-30T13:07:35.493 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-30T13:07:35.503 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-30T13:07:35.543 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-30T13:07:35.591 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-30T13:07:35.628 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-2519.gac171039.el9.x86_6 53/198 2024-03-30T13:07:35.640 INFO:teuthology.orchestra.run.smithi117.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.2 MB/s | 256 kB 00:00 2024-03-30T13:07:35.654 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-30T13:07:35.707 INFO:teuthology.orchestra.run.smithi117.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 345 kB/s | 23 kB 00:00 2024-03-30T13:07:35.857 INFO:teuthology.orchestra.run.smithi117.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 723 kB/s | 108 kB 00:00 2024-03-30T13:07:35.940 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-30T13:07:35.982 INFO:teuthology.orchestra.run.smithi117.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-03-30T13:07:36.024 INFO:teuthology.orchestra.run.smithi117.stdout:(163/174): librbd1-19.0.0-2519.gac171039.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-03-30T13:07:36.074 INFO:teuthology.orchestra.run.smithi117.stdout:(164/174): librados2-19.0.0-2519.gac171039.el9. 2.3 MB/s | 3.4 MB 00:01 2024-03-30T13:07:36.158 INFO:teuthology.orchestra.run.smithi117.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 773 kB/s | 136 kB 00:00 2024-03-30T13:07:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-30T13:07:36.350 INFO:teuthology.orchestra.run.smithi117.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 196 kB/s | 64 kB 00:00 2024-03-30T13:07:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-30T13:07:36.440 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-30T13:07:36.458 INFO:teuthology.orchestra.run.smithi117.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 75 kB/s | 29 kB 00:00 2024-03-30T13:07:36.586 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-30T13:07:36.649 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-30T13:07:36.705 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-30T13:07:36.734 INFO:teuthology.orchestra.run.smithi117.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 2.1 MB/s | 1.2 MB 00:00 2024-03-30T13:07:36.777 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-30T13:07:36.807 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-30T13:07:36.836 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-30T13:07:36.884 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-30T13:07:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-30T13:07:36.934 INFO:teuthology.orchestra.run.smithi117.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 2.3 MB/s | 465 kB 00:00 2024-03-30T13:07:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-30T13:07:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-30T13:07:37.017 INFO:teuthology.orchestra.run.smithi117.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 373 kB/s | 31 kB 00:00 2024-03-30T13:07:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-30T13:07:37.481 INFO:teuthology.orchestra.run.smithi117.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.6 MB/s | 1.1 MB 00:00 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-30T13:07:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-30T13:07:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-30T13:07:37.560 INFO:teuthology.orchestra.run.smithi117.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 596 kB/s | 74 kB 00:00 2024-03-30T13:07:37.593 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-30T13:07:37.725 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-30T13:07:37.842 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-30T13:07:37.888 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-30T13:07:37.947 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-30T13:07:38.010 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-30T13:07:38.224 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-30T13:07:38.267 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-30T13:07:38.409 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-30T13:07:38.448 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-30T13:07:38.482 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-30T13:07:38.511 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-30T13:07:38.539 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-30T13:07:38.573 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-30T13:07:38.613 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-30T13:07:38.626 INFO:teuthology.orchestra.run.smithi117.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 970 kB/s | 2.2 MB 00:02 2024-03-30T13:07:38.644 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-30T13:07:38.674 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-30T13:07:38.746 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-30T13:07:38.776 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-30T13:07:38.801 INFO:teuthology.orchestra.run.smithi117.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 998 kB/s | 2.3 MB 00:02 2024-03-30T13:07:38.807 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-30T13:07:38.807 INFO:teuthology.orchestra.run.smithi117.stdout:Total 13 MB/s | 226 MB 00:17 2024-03-30T13:07:38.809 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-30T13:07:38.851 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-30T13:07:38.885 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-30T13:07:38.979 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-30T13:07:39.067 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-30T13:07:39.125 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-30T13:07:39.155 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-30T13:07:39.171 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-03-30T13:07:39.203 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-30T13:07:39.312 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-30T13:07:39.375 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-30T13:07:39.445 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-03-30T13:07:39.446 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-03-30T13:07:40.139 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-30T13:07:40.189 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-30T13:07:40.223 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-30T13:07:40.271 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-30T13:07:40.362 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-30T13:07:40.398 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-30T13:07:40.580 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-30T13:07:40.652 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-30T13:07:40.835 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-30T13:07:41.415 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-03-30T13:07:41.416 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-03-30T13:07:41.729 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-30T13:07:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-30T13:07:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-30T13:07:43.220 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-30T13:07:43.251 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-30T13:07:43.282 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-30T13:07:43.309 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-30T13:07:43.641 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-30T13:07:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-2519.gac171039.el9.x86_64 124/198 2024-03-30T13:07:43.772 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-2519.gac171039.el9.x86_64 124/198 2024-03-30T13:07:43.802 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-2519.gac171039.el9.x86_64 125/198 2024-03-30T13:07:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-30T13:07:44.323 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-30T13:07:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-2519.gac171039.el9.x86_64 128/198 2024-03-30T13:07:44.412 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-2519.gac171039.el9.x86_64 128/198 2024-03-30T13:07:44.435 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-2519.gac171039.el9.x86_64 129/198 2024-03-30T13:07:46.804 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 130/198 2024-03-30T13:07:46.840 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 130/198 2024-03-30T13:07:47.408 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 130/198 2024-03-30T13:07:47.451 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 131/198 2024-03-30T13:07:47.526 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 131/198 2024-03-30T13:07:47.526 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-30T13:07:47.526 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-30T13:07:47.526 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:07:47.569 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 132/198 2024-03-30T13:07:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-03-30T13:07:48.446 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-30T13:07:48.481 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-30T13:07:48.519 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-30T13:07:48.528 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-30T13:07:48.569 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-30T13:07:48.633 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-30T13:07:48.673 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-30T13:07:48.718 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-30T13:07:48.769 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-30T13:07:49.098 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-30T13:07:49.144 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.0.0-2519.gac171039.el9.x86_64 10/198 2024-03-30T13:07:49.264 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.0.0-2519.gac171039.el9.x86_64 10/198 2024-03-30T13:07:49.287 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 11/198 2024-03-30T13:07:49.377 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 11/198 2024-03-30T13:07:49.410 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.0.0-2519.gac171039.el9.x86_64 12/198 2024-03-30T13:07:49.445 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-30T13:07:49.640 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-30T13:07:49.887 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-30T13:07:49.925 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 16/198 2024-03-30T13:07:50.012 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 16/198 2024-03-30T13:07:50.034 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.0.0-2519.gac171039.el9.x86 17/198 2024-03-30T13:07:50.099 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2519.gac171039.el9.x86 17/198 2024-03-30T13:07:50.138 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-30T13:07:50.161 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-30T13:07:50.176 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-30T13:07:50.369 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-30T13:07:50.401 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-30T13:07:50.448 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-30T13:07:50.640 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-30T13:07:50.795 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-30T13:07:50.855 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-30T13:07:50.963 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-30T13:07:51.023 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-30T13:07:51.233 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-30T13:07:51.335 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-30T13:07:51.439 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-30T13:07:51.478 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-30T13:07:51.505 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-30T13:07:51.548 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-30T13:07:51.622 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-30T13:07:51.684 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-30T13:07:51.723 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-30T13:07:51.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-30T13:07:51.822 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-30T13:07:51.865 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-30T13:07:51.899 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-30T13:07:51.926 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-30T13:07:51.995 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-30T13:07:52.031 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.0.0-2519.gac171039.el9. 39/198 2024-03-30T13:07:52.074 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.0.0-2519.gac171039.el 40/198 2024-03-30T13:07:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.0.0-2519.gac171039.el9.x86_6 41/198 2024-03-30T13:07:52.171 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-30T13:07:52.217 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-30T13:07:52.307 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-30T13:07:52.440 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-30T13:07:52.486 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-30T13:07:53.005 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-30T13:07:53.090 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-30T13:07:53.184 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-30T13:07:53.235 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-30T13:07:53.248 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-30T13:07:53.285 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-30T13:07:53.338 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-30T13:07:53.370 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.0.0-2519.gac171039.el9.x86_6 53/198 2024-03-30T13:07:53.410 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-30T13:07:53.668 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-30T13:07:54.021 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-30T13:07:54.103 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-30T13:07:54.147 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-30T13:07:54.288 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-30T13:07:54.341 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-30T13:07:54.396 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-30T13:07:54.469 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-30T13:07:54.501 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-30T13:07:54.537 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-30T13:07:54.583 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-30T13:07:54.619 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-30T13:07:54.662 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-30T13:07:54.704 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-30T13:07:54.844 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-30T13:07:54.893 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-30T13:07:54.923 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-30T13:07:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-30T13:07:55.067 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-30T13:07:55.106 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-30T13:07:55.156 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-30T13:07:55.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-30T13:07:55.206 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-30T13:07:55.288 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-30T13:07:55.413 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-30T13:07:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-30T13:07:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-30T13:07:55.612 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-30T13:07:55.667 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-30T13:07:55.848 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-30T13:07:55.893 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-30T13:07:56.020 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-30T13:07:56.066 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-30T13:07:56.098 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-30T13:07:56.129 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-30T13:07:56.157 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-30T13:07:56.191 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-30T13:07:56.230 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-30T13:07:56.262 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-30T13:07:56.292 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-30T13:07:56.360 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-30T13:07:56.402 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-30T13:07:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-30T13:07:56.477 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-30T13:07:56.512 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-30T13:07:56.600 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-30T13:07:56.683 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-30T13:07:56.726 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-30T13:07:56.766 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-30T13:07:56.821 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-30T13:07:56.919 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-30T13:07:56.967 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-30T13:07:57.700 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-30T13:07:57.747 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-30T13:07:57.781 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-30T13:07:57.829 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-30T13:07:57.917 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-30T13:07:57.949 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-30T13:07:58.136 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-30T13:07:58.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-30T13:07:58.260 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-30T13:07:58.904 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-30T13:07:59.012 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 132/198 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-03-30T13:07:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-03-30T13:07:59.859 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-03-30T13:07:59.859 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:00.021 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 133/198 2024-03-30T13:08:00.037 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 133/198 2024-03-30T13:08:00.038 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:00.038 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-30T13:08:00.038 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:00.222 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-30T13:08:00.306 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-30T13:08:00.337 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-30T13:08:00.369 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-30T13:08:00.396 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-30T13:08:00.712 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-30T13:08:00.743 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.0.0-2519.gac171039.el9.x86_64 124/198 2024-03-30T13:08:00.832 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.0.0-2519.gac171039.el9.x86_64 124/198 2024-03-30T13:08:00.864 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.0.0-2519.gac171039.el9.x86_64 125/198 2024-03-30T13:08:00.900 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-30T13:08:01.360 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-30T13:08:01.404 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.0.0-2519.gac171039.el9.x86_64 128/198 2024-03-30T13:08:01.475 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.0.0-2519.gac171039.el9.x86_64 128/198 2024-03-30T13:08:01.506 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.0.0-2519.gac171039.el9.x86_64 129/198 2024-03-30T13:08:01.566 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 134/198 2024-03-30T13:08:01.594 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 134/198 2024-03-30T13:08:01.594 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:01.594 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-30T13:08:01.594 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-30T13:08:01.594 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-30T13:08:01.595 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:01.835 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 135/198 2024-03-30T13:08:01.859 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 135/198 2024-03-30T13:08:01.859 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:01.860 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-30T13:08:01.860 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-30T13:08:01.860 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-30T13:08:01.860 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:02.328 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 136/198 2024-03-30T13:08:02.352 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 136/198 2024-03-30T13:08:02.352 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:02.352 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-30T13:08:02.352 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-30T13:08:02.352 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-30T13:08:02.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:02.446 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-30T13:08:02.847 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-2519.gac171039.el9.noarch 138/198 2024-03-30T13:08:02.878 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-2519.gac171039.el9.noarch 138/198 2024-03-30T13:08:02.910 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2519.gac171039.e 139/198 2024-03-30T13:08:02.975 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2519.gac171039. 140/198 2024-03-30T13:08:03.000 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noa 141/198 2024-03-30T13:08:03.702 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 130/198 2024-03-30T13:08:03.735 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 130/198 2024-03-30T13:08:04.092 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noa 141/198 2024-03-30T13:08:04.127 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.n 142/198 2024-03-30T13:08:04.287 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 130/198 2024-03-30T13:08:04.330 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 131/198 2024-03-30T13:08:04.422 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 131/198 2024-03-30T13:08:04.422 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-30T13:08:04.422 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-30T13:08:04.422 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:04.456 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 132/198 2024-03-30T13:08:05.140 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.n 142/198 2024-03-30T13:08:05.169 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2519.gac1 143/198 2024-03-30T13:08:05.275 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2519.gac1 143/198 2024-03-30T13:08:05.448 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2519.gac171039.el 144/198 2024-03-30T13:08:05.479 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 145/198 2024-03-30T13:08:05.504 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 145/198 2024-03-30T13:08:05.504 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:05.504 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-30T13:08:05.504 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-30T13:08:05.504 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-30T13:08:05.504 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:05.555 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 146/198 2024-03-30T13:08:05.566 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 146/198 2024-03-30T13:08:05.619 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-2519.gac171039.el9.x86_64 147/198 2024-03-30T13:08:06.538 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-30T13:08:06.565 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 149/198 2024-03-30T13:08:06.589 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 149/198 2024-03-30T13:08:06.589 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:06.589 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-30T13:08:06.589 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:08:06.589 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:08:06.589 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:06.632 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2519.gac171 150/198 2024-03-30T13:08:06.656 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2519.gac171 150/198 2024-03-30T13:08:06.656 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:06.656 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-30T13:08:06.656 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:06.717 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 151/198 2024-03-30T13:08:06.740 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 151/198 2024-03-30T13:08:06.741 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:06.741 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-30T13:08:06.741 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:06.741 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:06.741 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:07.059 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 152/198 2024-03-30T13:08:07.083 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 152/198 2024-03-30T13:08:07.083 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:07.083 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-30T13:08:07.083 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-30T13:08:07.083 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-30T13:08:07.083 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:11.797 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-2519.gac171039.el9.x86_64 153/198 2024-03-30T13:08:11.838 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-2519.gac171039.el9.x86_64 154/198 2024-03-30T13:08:11.868 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-2519.gac171039.el9.x86_64 155/198 2024-03-30T13:08:11.896 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-30T13:08:12.086 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.0.0-2519.gac171039.el9.x86_64 157/198 2024-03-30T13:08:12.199 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-30T13:08:12.244 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-2519.gac171039.el9.x86_64 159/198 2024-03-30T13:08:12.288 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-30T13:08:12.343 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-30T13:08:12.386 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-30T13:08:12.418 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-2519.gac171039.el9.x86_ 163/198 2024-03-30T13:08:12.451 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-30T13:08:12.480 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-30T13:08:12.538 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-30T13:08:12.599 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-30T13:08:12.628 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-30T13:08:13.209 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-30T13:08:13.251 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-30T13:08:13.272 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-30T13:08:14.151 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-2519.gac171039.el9.noarch 170/198 2024-03-30T13:08:14.246 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-30T13:08:14.289 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-30T13:08:14.387 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-30T13:08:14.416 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-30T13:08:14.441 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-30T13:08:14.466 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-30T13:08:14.493 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-30T13:08:14.493 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-30T13:08:14.581 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-30T13:08:14.610 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-30T13:08:14.611 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-30T13:08:14.695 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-30T13:08:14.721 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-30T13:08:14.754 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-30T13:08:14.780 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-30T13:08:14.797 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-30T13:08:14.831 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-30T13:08:14.869 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-30T13:08:14.917 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-30T13:08:14.943 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-30T13:08:14.968 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-30T13:08:14.985 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-30T13:08:15.010 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-30T13:08:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-30T13:08:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-30T13:08:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-30T13:08:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-30T13:08:15.296 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-30T13:08:15.321 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-30T13:08:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-30T13:08:15.380 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-30T13:08:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-30T13:08:15.474 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-30T13:08:15.827 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 132/198 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-03-30T13:08:15.828 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:16.002 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 133/198 2024-03-30T13:08:16.017 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 133/198 2024-03-30T13:08:16.017 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:16.017 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-30T13:08:16.017 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:17.480 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 134/198 2024-03-30T13:08:17.505 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 134/198 2024-03-30T13:08:17.506 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:17.506 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-30T13:08:17.506 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-30T13:08:17.506 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-30T13:08:17.506 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:17.731 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 135/198 2024-03-30T13:08:17.753 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 135/198 2024-03-30T13:08:17.753 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:17.753 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-30T13:08:17.753 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-30T13:08:17.753 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-30T13:08:17.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:18.190 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 136/198 2024-03-30T13:08:18.212 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 136/198 2024-03-30T13:08:18.212 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:18.212 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-30T13:08:18.212 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-30T13:08:18.212 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-30T13:08:18.213 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:18.308 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-30T13:08:18.690 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.0.0-2519.gac171039.el9.noarch 138/198 2024-03-30T13:08:18.715 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.0.0-2519.gac171039.el9.noarch 138/198 2024-03-30T13:08:18.747 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2519.gac171039.e 139/198 2024-03-30T13:08:18.808 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2519.gac171039. 140/198 2024-03-30T13:08:18.838 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noa 141/198 2024-03-30T13:08:19.872 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noa 141/198 2024-03-30T13:08:19.914 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.n 142/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-2519.gac171039.el9.x86_64 1/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 2/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 3/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-2519.gac171039.el9.x86_64 4/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2519.gac171 5/198 2024-03-30T13:08:20.878 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 6/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 7/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 8/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 9/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 10/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 11/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-2519.gac171039.el9.x86_64 12/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 13/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.0.0-2519.gac171039.el9.x86_64 14/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-2519.gac171039.el9.x86_ 15/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 16/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 17/198 2024-03-30T13:08:20.879 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-2519.gac171039.el9.x86_6 18/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-2519.gac171039.el9.x86 19/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-2519.gac171039.el9.x86_64 20/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2519.gac171039.el 21/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-2519.gac171039.el9. 22/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-2519.gac171039.el9.x86_6 23/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-2519.gac171039.el9.x86_64 24/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-2519.gac171039.el9.x86_64 25/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-2519.gac171039.el9.x86_64 26/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-2519.gac171039.el9.x86_64 27/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 28/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-2519.gac171039.el9.x86_64 29/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2519.gac171039. 30/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noa 31/198 2024-03-30T13:08:20.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.n 32/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2519.gac1 33/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2519.gac171039.el 34/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 35/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2519.gac171039.e 36/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 37/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-2519.gac171039.el9.noarch 38/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-2519.gac171039.el9.noarch 39/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-30T13:08:20.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-30T13:08:20.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-30T13:08:20.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-30T13:08:20.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-30T13:08:20.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-30T13:08:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-30T13:08:20.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-30T13:08:20.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-2519.gac171039.el9.x86_64 151/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-2519.gac171039.el9.x86_64 153/198 2024-03-30T13:08:20.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-30T13:08:20.890 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-30T13:08:20.891 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-30T13:08:20.892 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-30T13:08:20.893 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-30T13:08:20.893 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-30T13:08:20.893 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-30T13:08:20.893 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.n 142/198 2024-03-30T13:08:20.923 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2519.gac1 143/198 2024-03-30T13:08:21.025 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2519.gac1 143/198 2024-03-30T13:08:21.195 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2519.gac171039.el 144/198 2024-03-30T13:08:21.233 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 145/198 2024-03-30T13:08:21.256 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 145/198 2024-03-30T13:08:21.256 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:21.256 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-30T13:08:21.256 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-30T13:08:21.256 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-30T13:08:21.256 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:21.301 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 146/198 2024-03-30T13:08:21.311 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 146/198 2024-03-30T13:08:21.362 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.0.0-2519.gac171039.el9.x86_64 147/198 2024-03-30T13:08:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-30T13:08:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 149/198 2024-03-30T13:08:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 149/198 2024-03-30T13:08:22.292 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:22.292 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-30T13:08:22.292 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:08:22.292 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:08:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:22.337 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2519.gac171 150/198 2024-03-30T13:08:22.358 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2519.gac171 150/198 2024-03-30T13:08:22.358 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:22.359 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-30T13:08:22.359 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:22.412 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 151/198 2024-03-30T13:08:22.434 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 151/198 2024-03-30T13:08:22.434 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:22.434 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-30T13:08:22.434 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:22.435 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:22.435 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 152/198 2024-03-30T13:08:22.768 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 152/198 2024-03-30T13:08:22.769 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-30T13:08:22.769 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-30T13:08:22.769 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-30T13:08:22.769 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-30T13:08:22.769 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:23.710 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-30T13:08:23.710 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: acl-2.3.1-4.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-30T13:08:23.711 INFO:teuthology.orchestra.run.smithi066.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.712 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.713 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.714 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-30T13:08:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: jq-1.6-16.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-30T13:08:23.716 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-30T13:08:23.717 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.14-1.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-30T13:08:23.718 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-30T13:08:23.719 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-30T13:08:23.720 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-30T13:08:23.721 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-30T13:08:23.722 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-30T13:08:23.723 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.724 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-30T13:08:23.725 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-30T13:08:23.725 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-30T13:08:23.725 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-30T13:08:23.726 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-30T13:08:23.727 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-30T13:08:23.727 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-30T13:08:23.727 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-30T13:08:23.728 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-30T13:08:23.728 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-30T13:08:23.729 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-30T13:08:23.729 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-30T13:08:23.729 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-30T13:08:23.729 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.729 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.729 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:23.730 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-03-30T13:08:23.730 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-30T13:08:23.730 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-30T13:08:23.730 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-30T13:08:23.730 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-30T13:08:23.731 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-30T13:08:23.733 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-30T13:08:23.733 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:23.733 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-03-30T13:08:24.211 DEBUG:teuthology.parallel:result is None 2024-03-30T13:08:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.0.0-2519.gac171039.el9.x86_64 153/198 2024-03-30T13:08:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.0.0-2519.gac171039.el9.x86_64 154/198 2024-03-30T13:08:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.0.0-2519.gac171039.el9.x86_64 155/198 2024-03-30T13:08:27.341 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-30T13:08:27.527 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.0.0-2519.gac171039.el9.x86_64 157/198 2024-03-30T13:08:27.640 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-30T13:08:27.680 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.0.0-2519.gac171039.el9.x86_64 159/198 2024-03-30T13:08:27.738 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-30T13:08:27.789 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-30T13:08:27.824 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-30T13:08:27.848 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.0.0-2519.gac171039.el9.x86_ 163/198 2024-03-30T13:08:27.889 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-30T13:08:27.918 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-30T13:08:27.988 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-30T13:08:28.049 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-30T13:08:28.091 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-30T13:08:28.666 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-30T13:08:28.706 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-30T13:08:28.726 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-30T13:08:29.539 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.0.0-2519.gac171039.el9.noarch 170/198 2024-03-30T13:08:29.611 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-30T13:08:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-30T13:08:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-30T13:08:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-30T13:08:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-30T13:08:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-30T13:08:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-30T13:08:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-30T13:08:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-30T13:08:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-30T13:08:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-30T13:08:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-30T13:08:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-30T13:08:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-30T13:08:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-30T13:08:30.179 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-30T13:08:30.213 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-30T13:08:30.260 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-30T13:08:30.300 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-30T13:08:30.326 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-30T13:08:30.351 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-30T13:08:30.376 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-30T13:08:30.401 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-30T13:08:30.407 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-30T13:08:30.408 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-30T13:08:30.587 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-30T13:08:30.587 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-30T13:08:30.662 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-30T13:08:30.687 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-30T13:08:30.718 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-30T13:08:30.746 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-30T13:08:30.771 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-30T13:08:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-30T13:08:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.0.0-2519.gac171039.el9.x86_64 1/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 2/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 3/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.0.0-2519.gac171039.el9.x86_64 4/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2519.gac171 5/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 6/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 7/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 8/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 9/198 2024-03-30T13:08:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 10/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 11/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.0.0-2519.gac171039.el9.x86_64 12/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 13/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.0.0-2519.gac171039.el9.x86_64 14/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.0.0-2519.gac171039.el9.x86_ 15/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 16/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 17/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.0.0-2519.gac171039.el9.x86_6 18/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.0.0-2519.gac171039.el9.x86 19/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.0.0-2519.gac171039.el9.x86_64 20/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2519.gac171039.el 21/198 2024-03-30T13:08:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.0.0-2519.gac171039.el9. 22/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.0.0-2519.gac171039.el9.x86_6 23/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.0.0-2519.gac171039.el9.x86_64 24/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.0.0-2519.gac171039.el9.x86_64 25/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.0.0-2519.gac171039.el9.x86_64 26/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.0.0-2519.gac171039.el9.x86_64 27/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 28/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.0.0-2519.gac171039.el9.x86_64 29/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2519.gac171039. 30/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noa 31/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.n 32/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2519.gac1 33/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2519.gac171039.el 34/198 2024-03-30T13:08:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 35/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2519.gac171039.e 36/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 37/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.0.0-2519.gac171039.el9.noarch 38/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.0.0-2519.gac171039.el9.noarch 39/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-30T13:08:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-30T13:08:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-30T13:08:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-30T13:08:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-30T13:08:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-30T13:08:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-30T13:08:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-30T13:08:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.0.0-2519.gac171039.el9.x86_64 151/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.0.0-2519.gac171039.el9.x86_64 153/198 2024-03-30T13:08:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-30T13:08:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-30T13:08:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-30T13:08:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-30T13:08:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: acl-2.3.1-4.el9.x86_64 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-30T13:08:39.512 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.513 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.514 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.515 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.0.0-2519.gac171039.el9.noarch 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-30T13:08:39.516 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.6-16.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-30T13:08:39.517 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-30T13:08:39.518 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.14-1.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-30T13:08:39.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-30T13:08:39.520 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-30T13:08:39.521 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-30T13:08:39.522 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-30T13:08:39.523 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.0.0-2519.gac171039.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-30T13:08:39.524 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-30T13:08:39.525 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-30T13:08:39.525 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-30T13:08:39.525 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-30T13:08:39.525 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-30T13:08:39.525 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:39.525 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-03-30T13:08:39.985 DEBUG:teuthology.parallel:result is None 2024-03-30T13:08:39.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:08:40.131 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T13:08:40.158 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-2519.gac171039.el9 2024-03-30T13:08:40.159 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2519.gac171039.el9 2024-03-30T13:08:40.159 INFO:teuthology.task.install:The correct ceph version 19.0.0-2519.gac171039 is installed. 2024-03-30T13:08:40.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:08:40.378 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T13:08:40.400 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-2519.gac171039.el9 2024-03-30T13:08:40.400 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2519.gac171039.el9 2024-03-30T13:08:40.401 INFO:teuthology.task.install:The correct ceph version 19.0.0-2519.gac171039 is installed. 2024-03-30T13:08:40.402 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-30T13:08:40.402 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:40.403 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T13:08:40.442 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:40.443 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T13:08:40.472 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-30T13:08:40.473 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:40.473 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T13:08:40.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T13:08:40.591 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:40.591 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T13:08:40.618 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T13:08:40.685 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-30T13:08:40.685 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:40.685 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T13:08:40.719 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T13:08:40.791 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:40.791 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T13:08:40.818 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T13:08:40.885 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-30T13:08:40.885 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:40.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T13:08:40.916 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T13:08:40.988 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:40.988 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T13:08:41.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T13:08:41.081 INFO:teuthology.run_tasks:Running task ceph... 2024-03-30T13:08:41.172 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-30T13:08:41.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-03-30T13:08:41.175 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2024-03-30T13:08:41.203 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-30T13:08:41.204 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T13:08:41.246 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T13:08:41.275 INFO:tasks.ceph:Creating extra log directories... 2024-03-30T13:08:41.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T13:08:41.317 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T13:08:41.352 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-30T13:08:41.353 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-30T13:08:41.353 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-30T13:08:41.353 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:41.353 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T13:08:41.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:41.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:41.561 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:41.561 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T13:08:41.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:41.696 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:41.728 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-30T13:08:41.728 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-30T13:08:41.729 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149', 'branch': 'wip-mchangir-testing1-main-20240328-072412', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7629149', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing1-main-20240328-072412'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing1-main-20240328-072412', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 364, 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-mchangir-testing1-main-20240328-072412', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFY4upZpo7cB0coVquEDOdrwqJg6OHFD4EoV56xP3Skyi4ezLzOlJj9k+KZRYRwcxBjfPZvKAZSZ0i/etF3yKMU=', 'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwDGvp2eo//D/JeOH1AxX2bT9k/mPLPwvuLr2Wx8VimpD9fbDot15X4u0nzdw4lsMuJtCNAREBkkp+FK9HhG0s='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_uninlining']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_13:38:54', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-30T13:08:41.729 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T13:08:41.731 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T13:08:41.778 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T13:08:41.786 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T13:08:41.845 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:41.845 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T13:08:41.870 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T13:08:41.870 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-03-30T13:08:41.926 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T13:08:41.926 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:41.926 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-30T13:08:41.926 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:41.926 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:41.927 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 13:08:15.806916192 +0000 2024-03-30T13:08:41.927 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 13:04:32.253319279 +0000 2024-03-30T13:08:41.927 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 13:04:32.253319279 +0000 2024-03-30T13:08:41.927 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 13:04:32.253319279 +0000 2024-03-30T13:08:41.927 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T13:08:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T13:08:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T13:08:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000204471 s, 2.5 MB/s 2024-03-30T13:08:41.998 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T13:08:42.054 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-03-30T13:08:42.110 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T13:08:42.110 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:42.110 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-30T13:08:42.110 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:42.110 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:42.111 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 13:08:15.807916182 +0000 2024-03-30T13:08:42.111 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 13:04:32.595316252 +0000 2024-03-30T13:08:42.111 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 13:04:32.595316252 +0000 2024-03-30T13:08:42.111 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 13:04:32.595316252 +0000 2024-03-30T13:08:42.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T13:08:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T13:08:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T13:08:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000172939 s, 3.0 MB/s 2024-03-30T13:08:42.180 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T13:08:42.236 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-03-30T13:08:42.291 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T13:08:42.291 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:42.291 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-30T13:08:42.292 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:42.292 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:42.292 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 13:08:15.807916182 +0000 2024-03-30T13:08:42.292 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 13:04:32.957313048 +0000 2024-03-30T13:08:42.292 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 13:04:32.957313048 +0000 2024-03-30T13:08:42.292 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 13:04:32.957313048 +0000 2024-03-30T13:08:42.292 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T13:08:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T13:08:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T13:08:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000224819 s, 2.3 MB/s 2024-03-30T13:08:42.362 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T13:08:42.419 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-03-30T13:08:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T13:08:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:42.474 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-30T13:08:42.474 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:42.474 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:42.474 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-30 13:08:15.808916173 +0000 2024-03-30T13:08:42.475 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-30 13:04:33.305309968 +0000 2024-03-30T13:08:42.475 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-30 13:04:33.305309968 +0000 2024-03-30T13:08:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-30 13:04:33.305309968 +0000 2024-03-30T13:08:42.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T13:08:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-30T13:08:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-30T13:08:42.544 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000187459 s, 2.7 MB/s 2024-03-30T13:08:42.545 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T13:08:42.602 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:08:42.603 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:42.603 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T13:08:42.621 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T13:08:42.621 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-03-30T13:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T13:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-30T13:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-30 13:08:31.206669951 +0000 2024-03-30T13:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-30 13:06:09.358815543 +0000 2024-03-30T13:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-30 13:06:09.358815543 +0000 2024-03-30T13:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-30 13:06:09.358815543 +0000 2024-03-30T13:08:42.679 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T13:08:42.748 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-30T13:08:42.748 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-30T13:08:42.748 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000275682 s, 1.9 MB/s 2024-03-30T13:08:42.749 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T13:08:42.807 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-30 13:08:31.206669951 +0000 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-30 13:06:09.700808980 +0000 2024-03-30T13:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-30 13:06:09.700808980 +0000 2024-03-30T13:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-30 13:06:09.700808980 +0000 2024-03-30T13:08:42.865 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T13:08:42.932 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-30T13:08:42.932 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-30T13:08:42.932 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000278443 s, 1.8 MB/s 2024-03-30T13:08:42.934 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T13:08:42.991 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-03-30T13:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-30 13:08:31.207669932 +0000 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-30 13:06:10.068801917 +0000 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-30 13:06:10.068801917 +0000 2024-03-30T13:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-30 13:06:10.068801917 +0000 2024-03-30T13:08:43.048 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T13:08:43.115 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-30T13:08:43.115 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-30T13:08:43.115 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000230229 s, 2.2 MB/s 2024-03-30T13:08:43.116 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T13:08:43.173 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-30 13:08:31.207669932 +0000 2024-03-30T13:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-30 13:06:10.413795295 +0000 2024-03-30T13:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-30 13:06:10.413795295 +0000 2024-03-30T13:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-03-30 13:06:10.413795295 +0000 2024-03-30T13:08:43.230 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T13:08:43.298 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-30T13:08:43.298 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-30T13:08:43.298 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000165838 s, 3.1 MB/s 2024-03-30T13:08:43.298 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T13:08:43.354 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:08:43.354 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T13:08:43.354 INFO:tasks.ceph:Generating config... 2024-03-30T13:08:43.355 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-30T13:08:43.355 INFO:tasks.ceph:[client] debug client = 20 2024-03-30T13:08:43.355 INFO:tasks.ceph:[client] debug ms = 1 2024-03-30T13:08:43.355 INFO:tasks.ceph:[client] pid file = 2024-03-30T13:08:43.355 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-30T13:08:43.355 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-30T13:08:43.355 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] mds_bal_replicate_threshold = 1 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-30T13:08:43.356 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-30T13:08:43.357 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-30T13:08:43.357 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-30T13:08:43.357 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-30T13:08:43.357 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-30T13:08:43.357 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-30T13:08:43.357 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-30T13:08:43.358 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-30T13:08:43.358 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-30T13:08:43.358 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-30T13:08:43.358 INFO:tasks.ceph:Setting up mon.a... 2024-03-30T13:08:43.358 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-30T13:08:43.422 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.keyring 2024-03-30T13:08:43.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-30T13:08:43.476 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T13:08:43.508 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.66'), ('mon.b', '172.21.15.117'), ('mon.c', '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]')] 2024-03-30T13:08:43.508 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.66,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-30T13:08:43.509 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:43.509 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T13:08:43.564 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.66 --add b 172.21.15.117 --addv c '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:43.650 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:43.650 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: generated fsid 03992909-781c-43b4-940b-d3d6d831de0b 2024-03-30T13:08:43.650 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = quincy 2024-03-30T13:08:43.650 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 0 2024-03-30T13:08:43.650 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 03992909-781c-43b4-940b-d3d6d831de0b 2024-03-30T13:08:43.650 INFO:teuthology.orchestra.run.smithi066.stdout:last_changed 2024-03-30T13:08:43.648037+0000 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:created 2024-03-30T13:08:43.648037+0000 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:min_mon_release 17 (quincy) 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:election_strategy: 1 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:2: [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] mon.c 2024-03-30T13:08:43.651 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-30T13:08:43.652 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T13:08:43.667 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 03992909-781c-43b4-940b-d3d6d831de0b... 2024-03-30T13:08:43.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T13:08:43.710 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-03-30T13:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-03-30T13:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.66,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = 03992909-781c-43b4-940b-d3d6d831de0b 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-03-30T13:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-03-30T13:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 20 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 20 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 10 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-03-30T13:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-03-30T13:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: debug finisher = 20 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: mon down mkfs grace = 300 2024-03-30T13:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-03-30T13:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-03-30T13:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = True 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: mds_bal_replicate_threshold = 1 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T13:08:43.775 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.776 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-03-30T13:08:43.777 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-03-30T13:08:43.778 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on skipped message = False 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 03992909-781c-43b4-940b-d3d6d831de0b 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.779 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.780 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-03-30T13:08:43.781 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:08:43.782 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: debug finisher = 20 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:08:43.783 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-03-30T13:08:43.784 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-03-30T13:08:43.785 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: mds_bal_replicate_threshold = 1 2024-03-30T13:08:43.786 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-03-30T13:08:43.787 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-03-30T13:08:43.787 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-03-30T13:08:43.788 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-30T13:08:43.788 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-30T13:08:43.891 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-30T13:08:43.891 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:43.891 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-30T13:08:43.906 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:43.906 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-30T13:08:43.961 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:08:43.961 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:43.961 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T13:08:43.961 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T13:08:44.048 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:08:44.048 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:44.103 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:08:44.104 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:44.104 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T13:08:44.104 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T13:08:44.145 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:08:44.145 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:44.200 INFO:tasks.ceph:Setting up mon nodes... 2024-03-30T13:08:44.264 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-30T13:08:44.264 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T13:08:44.328 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T13:08:44.331 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T13:08:44.391 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T13:08:44.394 INFO:tasks.ceph:Setting up mds nodes... 2024-03-30T13:08:44.395 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T13:08:44.457 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T13:08:44.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-30T13:08:44.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T13:08:44.594 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T13:08:44.597 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-30T13:08:44.627 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T13:08:44.682 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T13:08:44.685 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-30T13:08:44.713 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T13:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T13:08:44.810 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-30T13:08:44.836 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-30T13:08:44.837 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-30T13:08:44.884 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-30T13:08:44.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-30T13:08:44.950 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-30T13:08:44.968 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-30T13:08:45.010 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-30T13:08:45.025 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-30T13:08:45.108 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-30T13:08:45.123 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-30T13:08:45.123 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T13:08:45.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-30T13:08:45.154 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:08:45.155 INFO:tasks.ceph:role: osd.0 2024-03-30T13:08:45.155 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:08:45.155 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T13:08:45.233 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:08:45.233 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:08:45.234 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:08:45.737 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T13:08:45.739 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T13:08:45.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-30T13:08:45.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-30T13:08:45.922 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-30T13:08:45.995 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:08:45.995 INFO:tasks.ceph:role: osd.1 2024-03-30T13:08:45.995 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:08:45.996 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:08:46.075 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:08:46.523 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T13:08:46.525 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T13:08:46.525 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-30T13:08:46.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-30T13:08:46.656 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-30T13:08:46.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:08:46.728 INFO:tasks.ceph:role: osd.2 2024-03-30T13:08:46.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:08:46.728 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:08:46.807 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:08:46.808 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:08:46.808 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:08:47.288 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T13:08:47.290 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T13:08:47.290 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-30T13:08:47.430 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-30T13:08:47.468 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-30T13:08:47.539 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:08:47.539 INFO:tasks.ceph:role: osd.3 2024-03-30T13:08:47.539 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:08:47.539 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T13:08:47.617 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:08:47.617 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:08:47.617 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:08:47.618 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:08:48.128 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-03-30T13:08:48.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-03-30T13:08:48.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-30T13:08:48.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-30T13:08:48.294 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:48.395+0000 7fbfb1801640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-30T13:08:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:48.395+0000 7fbfb1801640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-30T13:08:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:48.395+0000 7fbfb1801640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T13:08:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:48.395+0000 7fbfb1801640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T13:08:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:48.395+0000 7fbfb1801640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T13:08:48.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:48.396+0000 7fbfb1801640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-30T13:08:51.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-30T13:08:51.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:51.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:51.375+0000 7f6869dac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-30T13:08:51.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:51.376+0000 7f6869dac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-30T13:08:51.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:51.376+0000 7f6869dac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T13:08:51.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:51.376+0000 7f6869dac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T13:08:51.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:51.376+0000 7f6869dac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T13:08:51.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:51.376+0000 7f6869dac640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-30T13:08:54.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-30T13:08:54.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:54.359+0000 7f8a56e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-30T13:08:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:54.360+0000 7f8a56e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-30T13:08:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:54.360+0000 7f8a56e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T13:08:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:54.360+0000 7f8a56e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T13:08:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:54.360+0000 7f8a56e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T13:08:54.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:54.360+0000 7f8a56e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-30T13:08:57.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-30T13:08:57.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:08:57.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:57.340+0000 7f0d45013640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-30T13:08:57.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:57.340+0000 7f0d45013640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-30T13:08:57.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:57.340+0000 7f0d45013640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T13:08:57.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:57.340+0000 7f0d45013640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T13:08:57.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:57.340+0000 7f0d45013640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T13:08:57.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:08:57.340+0000 7f0d45013640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-30T13:09:00.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-30T13:09:00.233 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-30T13:09:00.265 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:00.265 INFO:tasks.ceph:role: osd.4 2024-03-30T13:09:00.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:09:00.265 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:00.343 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:00.849 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-30T13:09:00.851 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-30T13:09:00.851 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-30T13:09:00.987 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-30T13:09:01.026 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-30T13:09:01.096 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:01.097 INFO:tasks.ceph:role: osd.5 2024-03-30T13:09:01.097 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:09:01.097 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:01.173 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:01.617 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-30T13:09:01.619 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-30T13:09:01.619 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-30T13:09:01.722 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-30T13:09:01.752 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-30T13:09:01.820 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:01.820 INFO:tasks.ceph:role: osd.6 2024-03-30T13:09:01.820 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:09:01.820 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T13:09:01.895 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:01.896 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:02.391 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-30T13:09:02.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-30T13:09:02.393 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-30T13:09:02.541 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-30T13:09:02.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-30T13:09:02.641 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:02.642 INFO:tasks.ceph:role: osd.7 2024-03-30T13:09:02.642 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:09:02.642 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T13:09:02.715 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:02.716 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:03.246 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-03-30T13:09:03.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-03-30T13:09:03.248 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-30T13:09:03.391 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-30T13:09:03.426 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:03.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:03.521+0000 7f6c97624640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-30T13:09:03.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:03.521+0000 7f6c97624640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-30T13:09:03.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:03.521+0000 7f6c97624640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T13:09:03.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:03.521+0000 7f6c97624640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T13:09:03.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:03.521+0000 7f6c97624640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T13:09:03.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:03.522+0000 7f6c97624640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-30T13:09:06.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-30T13:09:06.400 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:06.485+0000 7faef6c8b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-30T13:09:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:06.486+0000 7faef6c8b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-30T13:09:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:06.486+0000 7faef6c8b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T13:09:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:06.486+0000 7faef6c8b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T13:09:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:06.486+0000 7faef6c8b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T13:09:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:06.486+0000 7faef6c8b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-30T13:09:09.342 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-30T13:09:09.373 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:09.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:09.459+0000 7fa43440e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-30T13:09:09.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:09.460+0000 7fa43440e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-30T13:09:09.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:09.460+0000 7fa43440e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T13:09:09.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:09.460+0000 7fa43440e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T13:09:09.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:09.460+0000 7fa43440e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T13:09:09.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:09.460+0000 7fa43440e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-30T13:09:11.730 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:11.733 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:12.326 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-30T13:09:12.356 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:12.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:12.440+0000 7f3d4b738640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-30T13:09:12.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:12.441+0000 7f3d4b738640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-30T13:09:12.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:12.441+0000 7f3d4b738640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T13:09:12.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:12.441+0000 7f3d4b738640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T13:09:12.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:12.441+0000 7f3d4b738640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T13:09:12.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:12.441+0000 7f3d4b738640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-30T13:09:15.294 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-30T13:09:15.322 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-30T13:09:15.322 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.323 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-30T13:09:15.360 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-30T13:09:15.434 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-30T13:09:15.507 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-30T13:09:15.579 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.579 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-30T13:09:15.651 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-30T13:09:15.722 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:15.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-30T13:09:15.793 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:15.793 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-30T13:09:15.824 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:15.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-30T13:09:15.894 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:15.894 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-30T13:09:15.963 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:15.963 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-30T13:09:16.031 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:16.031 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-30T13:09:16.100 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:16.100 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-30T13:09:16.167 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:16.167 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-30T13:09:16.234 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:16.234 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-30T13:09:16.250 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:16.250 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-30T13:09:16.306 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:16.306 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-30T13:09:16.320 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:16.321 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-30T13:09:16.376 INFO:tasks.ceph:Adding keys to all mons... 2024-03-30T13:09:16.376 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T13:09:16.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.y] 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghm2nh1ExAA/llAPiVUrYlS3mxLR+rPZQ== 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghmDX8pGxAAXUG3Yk8OMBMyc01hkoZZUA== 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghm84NPIxAAca1wZeauGlQMHS54SzhqIw== 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDgDghmz9qhFxAA358Jcrx/J+fQZ7i9GGHxsw== 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDjDghmpXB1FhAASU2j0mAAH7HASAgaR0ZhFg== 2024-03-30T13:09:16.420 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDmDghmZnV/FRAA+G0QiFaHxEweK3xSIr1MYQ== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDpDghma3JUFBAAD9kkiYh+/ew0XBFnsOPAJQ== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghm8/M0FxAA8he7CSb1rjXQZy2E4x/3IQ== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghmpUGNKBAA+EXwkCzjb8uf4IZ1U5QtYg== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghm210LMBAALVzbm4DxE7Nkdt8Xy3Cj4A== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDvDghmyvoYHxAARaVqrylp8Xl1Sw72nVydWw== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDyDghmgkkKHRAAIhxMaiQYTe2MlrvNZH/5uA== 2024-03-30T13:09:16.421 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQD1DghmRRx/GxAAWlX4LBu1lvjj0y8I5jmFjw== 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQD4DghmM0RbGhAAX4q8eHJEih+NUBxEoPlXkw== 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghmupudNBAAFaQaghMGgLisibrdNbX0TQ== 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout:[client.2] 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghmbu2BOBAAvr83E6ExRX1dKKXuanZj6w== 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout:[client.1] 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDdDghmYAp9ABAANUw0ahZSYbh8lGdQvtXebA== 2024-03-30T13:09:16.422 INFO:teuthology.orchestra.run.smithi066.stdout:[client.3] 2024-03-30T13:09:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDdDghmI35QBhAA26SuLDVaBa9wDDqx3JW1Og== 2024-03-30T13:09:16.441 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.y] 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghm2nh1ExAA/llAPiVUrYlS3mxLR+rPZQ== 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghmDX8pGxAAXUG3Yk8OMBMyc01hkoZZUA== 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghm84NPIxAAca1wZeauGlQMHS54SzhqIw== 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDgDghmz9qhFxAA358Jcrx/J+fQZ7i9GGHxsw== 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDjDghmpXB1FhAASU2j0mAAH7HASAgaR0ZhFg== 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2024-03-30T13:09:16.442 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDmDghmZnV/FRAA+G0QiFaHxEweK3xSIr1MYQ== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDpDghma3JUFBAAD9kkiYh+/ew0XBFnsOPAJQ== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghm8/M0FxAA8he7CSb1rjXQZy2E4x/3IQ== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghmpUGNKBAA+EXwkCzjb8uf4IZ1U5QtYg== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghm210LMBAALVzbm4DxE7Nkdt8Xy3Cj4A== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.4] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDvDghmyvoYHxAARaVqrylp8Xl1Sw72nVydWw== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.5] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDyDghmgkkKHRAAIhxMaiQYTe2MlrvNZH/5uA== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.6] 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD1DghmRRx/GxAAWlX4LBu1lvjj0y8I5jmFjw== 2024-03-30T13:09:16.443 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.7] 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQD4DghmM0RbGhAAX4q8eHJEih+NUBxEoPlXkw== 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghmupudNBAAFaQaghMGgLisibrdNbX0TQ== 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout:[client.2] 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDcDghmbu2BOBAAvr83E6ExRX1dKKXuanZj6w== 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDdDghmYAp9ABAANUw0ahZSYbh8lGdQvtXebA== 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout:[client.3] 2024-03-30T13:09:16.444 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDdDghmI35QBhAA26SuLDVaBa9wDDqx3JW1Og== 2024-03-30T13:09:16.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:09:16.467 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:09:16.537 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.539 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.596 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.598 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.656 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.658 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.712 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.714 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.770 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:16.880 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:09:16.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:09:16.937 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.939 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.995 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:16.998 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:09:17.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.170 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.224 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:09:17.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.339 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.341 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.399 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.401 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.461 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:09:17.518 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-30T13:09:17.518 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-30T13:09:17.549 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T13:09:17.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-30T13:09:17.683 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-30T13:09:17.710 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T13:09:17.807 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-30T13:09:17.834 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-30T13:09:17.901 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T13:09:17.999 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-30T13:09:18.025 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:18.027 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:18.081 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-30T13:09:18.082 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-30T13:09:18.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-30T13:09:18.085 INFO:tasks.ceph.mon.a:Started 2024-03-30T13:09:18.085 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-30T13:09:18.085 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-30T13:09:18.124 INFO:tasks.ceph.mon.b:Started 2024-03-30T13:09:18.124 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-30T13:09:18.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-30T13:09:18.126 INFO:tasks.ceph.mon.c:Started 2024-03-30T13:09:18.126 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-30T13:09:18.126 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-30T13:09:18.126 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-30T13:09:18.129 INFO:tasks.ceph.mgr.y:Started 2024-03-30T13:09:18.129 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-30T13:09:18.129 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-30T13:09:18.131 INFO:tasks.ceph.mgr.x:Started 2024-03-30T13:09:18.131 INFO:tasks.ceph:Setting crush tunables to default 2024-03-30T13:09:18.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd crush tunables default 2024-03-30T13:09:18.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.236+0000 7fa10a352640 1 Processor -- start 2024-03-30T13:09:18.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.236+0000 7fa10a352640 1 -- start start 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa10a352640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa1040b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa10a352640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 0x7fa1040b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa10a352640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040b66e0 con 0x7fa1040ba090 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa10a352640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa1040d2d90 con 0x7fa1040b73d0 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa10a352640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040d2ff0 con 0x7fa1040b5dc0 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1037fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa1040ba090 0x7fa1040d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46244/0 (socket says 172.21.15.66:46244) 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1037fe640 1 -- 172.21.15.66:0/1306464610 learned_addr learned my addr 172.21.15.66:0/1306464610 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa103fff640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa1040b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1088ef640 1 --2- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 0x7fa1040b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa103fff640 1 -- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 msgr2=0x7fa1040b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa103fff640 1 -- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 msgr2=0x7fa1040b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa103fff640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa1040b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1088ef640 1 -- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 msgr2=0x7fa1040b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1088ef640 1 -- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 msgr2=0x7fa1040b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1088ef640 1 --2- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 0x7fa1040b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa103fff640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa1040b61a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T13:09:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.237+0000 7fa1088ef640 1 --2- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 0x7fa1040b7880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T13:09:18.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026662873 0 0) 0x7fa1040b66e0 con 0x7fa1040ba090 2024-03-30T13:09:18.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0ec003520 con 0x7fa1040ba090 2024-03-30T13:09:18.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1613142484 0 0) 0x7fa0ec003520 con 0x7fa1040ba090 2024-03-30T13:09:18.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 msgr2=0x7fa1040b61a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:09:18.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa1040b61a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 msgr2=0x7fa1040b7880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 --2- 172.21.15.66:0/1306464610 >> v2:172.21.15.117:3300/0 conn(0x7fa1040b73d0 0x7fa1040b7880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040d32d0 con 0x7fa1040ba090 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7fa0f8001670 con 0x7fa1040ba090 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 >> v1:172.21.15.66:6789/0 conn(0x7fa1040ba090 legacy=0x7fa1040d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa0ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0ec003fe0 0x7fa0ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec004d40 0x7fa0ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d2d90 con 0x7fa0ec004d40 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d2ff0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.407+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec00bac0 con 0x7fa1040b5dc0 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1037fe640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0ec003fe0 0x7fa0ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1088ef640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec004d40 0x7fa0ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa103fff640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa0ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1037fe640 1 -- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 msgr2=0x7fa0ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1037fe640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1040b5dc0 0x7fa0ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1037fe640 1 -- 172.21.15.66:0/1306464610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec004d40 msgr2=0x7fa0ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1037fe640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec004d40 0x7fa0ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1037fe640 1 -- 172.21.15.66:0/1306464610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec003520 con 0x7fa0ec003fe0 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.408+0000 7fa1088ef640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec004d40 0x7fa0ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.409+0000 7fa1037fe640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0ec003fe0 0x7fa0ec004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1040ba4b0 tx=0x7fa0ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c3843f2d332d68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.409+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8007d50 con 0x7fa0ec003fe0 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.409+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f8030070 con 0x7fa0ec003fe0 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.409+0000 7fa102ffd640 1 -- 172.21.15.66:0/1306464610 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8005d10 con 0x7fa0ec003fe0 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa10a352640 1 -- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0ec003fe0 msgr2=0x7fa0ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa10a352640 1 --2- 172.21.15.66:0/1306464610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0ec003fe0 0x7fa0ec004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1040ba4b0 tx=0x7fa0ec005830 comp rx=0 tx=0).stop 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa1088ef640 1 -- 172.21.15.66:0/1306464610 reap_dead start 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa10a352640 1 -- 172.21.15.66:0/1306464610 shutdown_connections 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa10a352640 1 -- 172.21.15.66:0/1306464610 >> 172.21.15.66:0/1306464610 conn(0x7fa1040678f0 msgr2=0x7fa1040d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:18.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa10a352640 1 -- 172.21.15.66:0/1306464610 shutdown_connections 2024-03-30T13:09:18.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.410+0000 7fa10a352640 1 -- 172.21.15.66:0/1306464610 wait complete. 2024-03-30T13:09:18.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.411+0000 7fa10a352640 1 Processor -- start 2024-03-30T13:09:18.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 -- start start 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 0x7fa1041517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040cc460 0x7fa1040cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1041f43d0 0x7fa1041f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d3b70 con 0x7fa0ec003fe0 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d3890 con 0x7fa1040cc460 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa10a352640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040d4090 con 0x7fa1041f43d0 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa103fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 0x7fa1041517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa103fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 0x7fa1041517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59104/0 (socket says 172.21.15.66:59104) 2024-03-30T13:09:18.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa103fff640 1 -- 172.21.15.66:0/41672371 learned_addr learned my addr 172.21.15.66:0/41672371 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:18.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.412+0000 7fa1037fe640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040cc460 0x7fa1040cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa1088ef640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1041f43d0 0x7fa1041f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:18.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa103fff640 1 -- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1041f43d0 msgr2=0x7fa1041f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:18.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa103fff640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1041f43d0 0x7fa1041f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa103fff640 1 -- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040cc460 msgr2=0x7fa1040cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa103fff640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040cc460 0x7fa1040cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa103fff640 1 -- 172.21.15.66:0/41672371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10415abe0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa1037fe640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040cc460 0x7fa1040cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa103fff640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 0x7fa1041517b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40108d0 tx=0x7fa0f4010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f401a020 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f400e9c0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1041614a0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f400edb0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1041606e0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.413+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa0f401d020 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.414+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa0f4014810 con 0x7fa0ec003fe0 2024-03-30T13:09:18.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.414+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa104075890 con 0x7fa0ec003fe0 2024-03-30T13:09:18.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.416+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa0f4024020 con 0x7fa0ec003fe0 2024-03-30T13:09:18.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.483+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa1040b52b0 con 0x7fa0ec003fe0 2024-03-30T13:09:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.485+0000 7fa100ff9640 1 -- 172.21.15.66:0/41672371 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fa0f4014a10 con 0x7fa0ec003fe0 2024-03-30T13:09:18.488 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 msgr2=0x7fa1041517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 0x7fa1041517b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40108d0 tx=0x7fa0f4010d90 comp rx=0 tx=0).stop 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 shutdown_connections 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1041f43d0 0x7fa1041f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1040cc460 0x7fa1040cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 --2- 172.21.15.66:0/41672371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0ec003fe0 0x7fa1041517b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 >> 172.21.15.66:0/41672371 conn(0x7fa1040678f0 msgr2=0x7fa1040d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 shutdown_connections 2024-03-30T13:09:18.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:18.486+0000 7fa10a352640 1 -- 172.21.15.66:0/41672371 wait complete. 2024-03-30T13:09:18.500 INFO:tasks.ceph:check_enable_crimson: False 2024-03-30T13:09:18.501 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-30T13:09:18.501 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:18.501 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-30T13:09:18.532 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:18.532 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-30T13:09:18.603 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:18.603 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-30T13:09:18.674 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:18.674 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-30T13:09:18.745 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:18.745 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-30T13:09:18.772 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:18.773 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-30T13:09:18.841 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:18.841 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-30T13:09:18.907 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:09:18.907 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-30T13:09:18.909 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:18.905+0000 7f9f89a93040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T13:09:18.965 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:18.962+0000 7f9f89a93040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T13:09:18.974 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new ed397a5b-f6cc-47da-901b-1503da0fb2fb 0 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 Processor -- start 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 -- start start 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f59780cd380 0x7f597806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f597806bac0 0x7f597815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59780cf110 con 0x7f5978160200 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5978160710 con 0x7f59780cd380 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.112+0000 7f597df9d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f59781609f0 con 0x7f597806bac0 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f597806bac0 0x7f597815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f597806bac0 0x7f597815fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34896/0 (socket says 172.21.15.117:34896) 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 -- 172.21.15.117:0/454773167 learned_addr learned my addr 172.21.15.117:0/454773167 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f597cf9b640 1 --2- 172.21.15.117:0/454773167 >> v2:172.21.15.66:3300/0 conn(0x7f59780cd380 0x7f597806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 -- 172.21.15.117:0/454773167 >> v2:172.21.15.66:3300/0 conn(0x7f59780cd380 msgr2=0x7f597806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 --2- 172.21.15.117:0/454773167 >> v2:172.21.15.66:3300/0 conn(0x7f59780cd380 0x7f597806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 -- 172.21.15.117:0/454773167 >> v1:172.21.15.66:6789/0 conn(0x7f5978160200 legacy=0x7f59780cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f5977fff640 1 -- 172.21.15.117:0/454773167 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59780cf3b0 con 0x7f597806bac0 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.113+0000 7f597cf9b640 1 --2- 172.21.15.117:0/454773167 >> v2:172.21.15.66:3300/0 conn(0x7f59780cd380 0x7f597806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f5977fff640 1 --2- 172.21.15.117:0/454773167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597806bac0 0x7f597815fcc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59780cf2e0 tx=0x7f596402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d388707176fdfec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f59777fe640 1 -- 172.21.15.117:0/454773167 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964034070 con 0x7f597806bac0 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f59777fe640 1 -- 172.21.15.117:0/454773167 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5964030860 con 0x7f597806bac0 2024-03-30T13:09:19.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f59777fe640 1 -- 172.21.15.117:0/454773167 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964030c30 con 0x7f597806bac0 2024-03-30T13:09:19.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 -- 172.21.15.117:0/454773167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597806bac0 msgr2=0x7f597815fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 --2- 172.21.15.117:0/454773167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597806bac0 0x7f597815fcc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59780cf2e0 tx=0x7f596402b9a0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 -- 172.21.15.117:0/454773167 shutdown_connections 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 --2- 172.21.15.117:0/454773167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597806bac0 0x7f597815fcc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 --2- 172.21.15.117:0/454773167 >> v2:172.21.15.66:3300/0 conn(0x7f59780cd380 0x7f597806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 -- 172.21.15.117:0/454773167 >> 172.21.15.117:0/454773167 conn(0x7f59780d0b20 msgr2=0x7f59780d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.114+0000 7f597df9d640 1 -- 172.21.15.117:0/454773167 shutdown_connections 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.115+0000 7f597df9d640 1 -- 172.21.15.117:0/454773167 wait complete. 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.116+0000 7f597df9d640 1 Processor -- start 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.116+0000 7f597df9d640 1 -- start start 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.116+0000 7f597df9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 0x7f597813d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59780cd380 0x7f597813dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5978160200 0x7f59780b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59780ccb20 con 0x7f5978160200 2024-03-30T13:09:19.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59780cc9a0 con 0x7f59780cd380 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f59780d02c0 con 0x7f597806bac0 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597cf9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 0x7f597813d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59780cd380 0x7f597813dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597cf9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 0x7f597813d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41746/0 (socket says 172.21.15.117:41746) 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597cf9b640 1 -- 172.21.15.117:0/1237471902 learned_addr learned my addr 172.21.15.117:0/1237471902 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597d79c640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5978160200 0x7f59780b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 -- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 msgr2=0x7f597813d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 0x7f597813d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 -- 172.21.15.117:0/1237471902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5978160200 msgr2=0x7f59780b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5978160200 0x7f59780b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 -- 172.21.15.117:0/1237471902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597813e570 con 0x7f59780cd380 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597cf9b640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 0x7f597813d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:19.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597d79c640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5978160200 0x7f59780b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:19.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f5977fff640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59780cd380 0x7f597813dcd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5964033000 tx=0x7f5964004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964048070 con 0x7f59780cd380 2024-03-30T13:09:19.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5964030660 con 0x7f59780cd380 2024-03-30T13:09:19.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964034040 con 0x7f59780cd380 2024-03-30T13:09:19.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59780b42e0 con 0x7f59780cd380 2024-03-30T13:09:19.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.117+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59780b48a0 con 0x7f59780cd380 2024-03-30T13:09:19.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.118+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f596404b070 con 0x7f59780cd380 2024-03-30T13:09:19.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.118+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f596404dc90 con 0x7f59780cd380 2024-03-30T13:09:19.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.118+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597806eef0 con 0x7f59780cd380 2024-03-30T13:09:19.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.120+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5964052070 con 0x7f59780cd380 2024-03-30T13:09:19.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.193+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ed397a5b-f6cc-47da-901b-1503da0fb2fb", "id": 0} v 0) -- 0x7f59781559b0 con 0x7f59780cd380 2024-03-30T13:09:19.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.196+0000 7f59757fa640 1 -- 172.21.15.117:0/1237471902 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed397a5b-f6cc-47da-901b-1503da0fb2fb", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f597813e570 con 0x7f59780cd380 2024-03-30T13:09:19.199 INFO:teuthology.orchestra.run.smithi117.stdout:0 2024-03-30T13:09:19.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.196+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59780cd380 msgr2=0x7f597813dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59780cd380 0x7f597813dcd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5964033000 tx=0x7f5964004730 comp rx=0 tx=0).stop 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 shutdown_connections 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5978160200 0x7f59780b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59780cd380 0x7f597813dcd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 --2- 172.21.15.117:0/1237471902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f597806bac0 0x7f597813d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 >> 172.21.15.117:0/1237471902 conn(0x7f59780d0b20 msgr2=0x7f597813e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 shutdown_connections 2024-03-30T13:09:19.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.197+0000 7f597df9d640 1 -- 172.21.15.117:0/1237471902 wait complete. 2024-03-30T13:09:19.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 7d89d47e-4ac4-4b2d-9197-96f97a2229e6 1 2024-03-30T13:09:19.288 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:19.284+0000 7fd4a2c10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T13:09:19.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.308+0000 7fdd6c359640 1 Processor -- start 2024-03-30T13:09:19.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.308+0000 7fdd6c359640 1 -- start start 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6c359640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd64155b10 0x7fdd640b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6c359640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd640d03c0 0x7fdd640d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6c359640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd640b5ff0 con 0x7fdd6415fae0 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6c359640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fdd64155ef0 con 0x7fdd640d03c0 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6c359640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdd640d0d60 con 0x7fdd64155b10 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd640d03c0 0x7fdd640d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a0f5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd64155b10 0x7fdd640b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd640d03c0 0x7fdd640d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34908/0 (socket says 172.21.15.117:34908) 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/4278763661 learned_addr learned my addr 172.21.15.117:0/4278763661 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd64155b10 msgr2=0x7fdd640b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 --2- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd64155b10 0x7fdd640b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/4278763661 >> v1:172.21.15.66:6789/0 conn(0x7fdd6415fae0 legacy=0x7fdd640b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.309+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/4278763661 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd640b41b0 con 0x7fdd640d03c0 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.310+0000 7fdd6a8f6640 1 --2- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd640d03c0 0x7fdd640d0820 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009e50 tx=0x7fdd5c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e46feefb5362f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.310+0000 7fdd690f3640 1 -- 172.21.15.117:0/4278763661 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c02cb00 con 0x7fdd640d03c0 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.310+0000 7fdd690f3640 1 -- 172.21.15.117:0/4278763661 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5c02cca0 con 0x7fdd640d03c0 2024-03-30T13:09:19.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.310+0000 7fdd690f3640 1 -- 172.21.15.117:0/4278763661 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c034610 con 0x7fdd640d03c0 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 -- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd640d03c0 msgr2=0x7fdd640d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 --2- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd640d03c0 0x7fdd640d0820 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009e50 tx=0x7fdd5c02bf60 comp rx=0 tx=0).stop 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 -- 172.21.15.117:0/4278763661 shutdown_connections 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 --2- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd640d03c0 0x7fdd640d0820 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 --2- 172.21.15.117:0/4278763661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd64155b10 0x7fdd640b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 -- 172.21.15.117:0/4278763661 >> 172.21.15.117:0/4278763661 conn(0x7fdd640d1160 msgr2=0x7fdd640d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 -- 172.21.15.117:0/4278763661 shutdown_connections 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.311+0000 7fdd6c359640 1 -- 172.21.15.117:0/4278763661 wait complete. 2024-03-30T13:09:19.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.312+0000 7fdd6c359640 1 Processor -- start 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.312+0000 7fdd6c359640 1 -- start start 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd640d03c0 0x7fdd64151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd64155b10 0x7fdd640b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 0x7fdd640727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640b62d0 con 0x7fdd640d03c0 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640b4770 con 0x7fdd64155b10 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6415b9d0 con 0x7fdd6415fae0 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 0x7fdd640727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd698f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd64155b10 0x7fdd640b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 0x7fdd640727e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41764/0 (socket says 172.21.15.117:41764) 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/809746670 learned_addr learned my addr 172.21.15.117:0/809746670 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a0f5640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd640d03c0 0x7fdd64151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd64155b10 msgr2=0x7fdd640b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd64155b10 0x7fdd640b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/809746670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd640d03c0 msgr2=0x7fdd64151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd640d03c0 0x7fdd64151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 -- 172.21.15.117:0/809746670 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd640b1a20 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd698f4640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd64155b10 0x7fdd640b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6a8f6640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 0x7fdd640727e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009e50 tx=0x7fdd5c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c043020 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5c02ce50 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c009510 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.313+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd640b1d00 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.314+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd640b22c0 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.314+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd5c046020 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.314+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fdd5c0487b0 con 0x7fdd6415fae0 2024-03-30T13:09:19.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.314+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64075890 con 0x7fdd6415fae0 2024-03-30T13:09:19.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.318+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdd5c04d030 con 0x7fdd6415fae0 2024-03-30T13:09:19.346 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:19.343+0000 7fd4a2c10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T13:09:19.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.391+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7d89d47e-4ac4-4b2d-9197-96f97a2229e6", "id": 1} v 0) -- 0x7fdd64153210 con 0x7fdd6415fae0 2024-03-30T13:09:19.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.396+0000 7fdd5affd640 1 -- 172.21.15.117:0/809746670 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d89d47e-4ac4-4b2d-9197-96f97a2229e6", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fdd5c03edc0 con 0x7fdd6415fae0 2024-03-30T13:09:19.400 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-30T13:09:19.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 msgr2=0x7fdd640727e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 0x7fdd640727e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009e50 tx=0x7fdd5c004a40 comp rx=0 tx=0).stop 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 shutdown_connections 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd6415fae0 0x7fdd640727e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd64155b10 0x7fdd640b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 --2- 172.21.15.117:0/809746670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd640d03c0 0x7fdd64151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 >> 172.21.15.117:0/809746670 conn(0x7fdd640d1160 msgr2=0x7fdd6406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 shutdown_connections 2024-03-30T13:09:19.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.397+0000 7fdd6c359640 1 -- 172.21.15.117:0/809746670 wait complete. 2024-03-30T13:09:19.412 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new f1273686-748a-40d1-9fab-ef964bbe5595 2 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.508+0000 7f6842fe2640 1 Processor -- start 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.508+0000 7f6842fe2640 1 -- start start 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f6842fe2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 0x7f683c15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f6842fe2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f683c0d03c0 0x7f683c0d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f6842fe2640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f683c0b5ff0 con 0x7f683c155b10 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f6842fe2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f683c155ef0 con 0x7f683c0d03c0 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f6842fe2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f683c0b41b0 con 0x7f683c0b4c90 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 0x7f683c15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 0x7f683c15fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41770/0 (socket says 172.21.15.117:41770) 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 -- 172.21.15.117:0/3282784673 learned_addr learned my addr 172.21.15.117:0/3282784673 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f684157f640 1 --2- 172.21.15.117:0/3282784673 >> v2:172.21.15.66:3300/0 conn(0x7f683c0d03c0 0x7f683c0d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 -- 172.21.15.117:0/3282784673 >> v2:172.21.15.66:3300/0 conn(0x7f683c0d03c0 msgr2=0x7f683c0d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 --2- 172.21.15.117:0/3282784673 >> v2:172.21.15.66:3300/0 conn(0x7f683c0d03c0 0x7f683c0d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 -- 172.21.15.117:0/3282784673 >> v1:172.21.15.66:6789/0 conn(0x7f683c155b10 legacy=0x7f683c0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.509+0000 7f683bfff640 1 -- 172.21.15.117:0/3282784673 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c0b4490 con 0x7f683c0b4c90 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.510+0000 7f683bfff640 1 --2- 172.21.15.117:0/3282784673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 0x7f683c15fae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f682800bb10 tx=0x7f682802e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138fb302b0245bf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.510+0000 7f683b7fe640 1 -- 172.21.15.117:0/3282784673 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682803b070 con 0x7f683c0b4c90 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.510+0000 7f683b7fe640 1 -- 172.21.15.117:0/3282784673 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828036440 con 0x7f683c0b4c90 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.510+0000 7f683b7fe640 1 -- 172.21.15.117:0/3282784673 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828036810 con 0x7f683c0b4c90 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.510+0000 7f6842fe2640 1 -- 172.21.15.117:0/3282784673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 msgr2=0x7f683c15fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.510+0000 7f6842fe2640 1 --2- 172.21.15.117:0/3282784673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 0x7f683c15fae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f682800bb10 tx=0x7f682802e150 comp rx=0 tx=0).stop 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.511+0000 7f6842fe2640 1 -- 172.21.15.117:0/3282784673 shutdown_connections 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.511+0000 7f6842fe2640 1 --2- 172.21.15.117:0/3282784673 >> v2:172.21.15.66:3300/0 conn(0x7f683c0d03c0 0x7f683c0d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.511+0000 7f6842fe2640 1 --2- 172.21.15.117:0/3282784673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c0b4c90 0x7f683c15fae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.511+0000 7f6842fe2640 1 -- 172.21.15.117:0/3282784673 >> 172.21.15.117:0/3282784673 conn(0x7f683c0d1160 msgr2=0x7f683c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.511+0000 7f6842fe2640 1 -- 172.21.15.117:0/3282784673 shutdown_connections 2024-03-30T13:09:19.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.511+0000 7f6842fe2640 1 -- 172.21.15.117:0/3282784673 wait complete. 2024-03-30T13:09:19.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.512+0000 7f6842fe2640 1 Processor -- start 2024-03-30T13:09:19.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.512+0000 7f6842fe2640 1 -- start start 2024-03-30T13:09:19.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c0b4c90 0x7f683c1391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683c0d03c0 0x7f683c14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 0x7f683c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f683c15b9d0 con 0x7f683c0d03c0 2024-03-30T13:09:19.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f683c0b62d0 con 0x7f683c0b4c90 2024-03-30T13:09:19.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f683c15bb50 con 0x7f683c155b10 2024-03-30T13:09:19.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 0x7f683c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 0x7f683c0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41774/0 (socket says 172.21.15.117:41774) 2024-03-30T13:09:19.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6840d7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c0b4c90 0x7f683c1391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 -- 172.21.15.117:0/1459699199 learned_addr learned my addr 172.21.15.117:0/1459699199 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f683bfff640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683c0d03c0 0x7f683c14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 -- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c0b4c90 msgr2=0x7f683c1391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c0b4c90 0x7f683c1391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 -- 172.21.15.117:0/1459699199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683c0d03c0 msgr2=0x7f683c14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683c0d03c0 0x7f683c14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 -- 172.21.15.117:0/1459699199 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c139720 con 0x7f683c155b10 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6840d7e640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c0b4c90 0x7f683c1391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f684157f640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 0x7f683c0cd8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6834009dd0 tx=0x7f683400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834018020 con 0x7f683c155b10 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6834014480 con 0x7f683c155b10 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834014870 con 0x7f683c155b10 2024-03-30T13:09:19.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c150600 con 0x7f683c155b10 2024-03-30T13:09:19.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.513+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c150bc0 con 0x7f683c155b10 2024-03-30T13:09:19.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.514+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6834005d10 con 0x7f683c155b10 2024-03-30T13:09:19.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.514+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6834013800 con 0x7f683c155b10 2024-03-30T13:09:19.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.514+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c075890 con 0x7f683c155b10 2024-03-30T13:09:19.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.516+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6834022070 con 0x7f683c155b10 2024-03-30T13:09:19.535 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.532+0000 7f9f89a93040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T13:09:19.592 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.589+0000 7f9f89a93040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T13:09:19.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.588+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f1273686-748a-40d1-9fab-ef964bbe5595", "id": 2} v 0) -- 0x7f683c154ff0 con 0x7f683c155b10 2024-03-30T13:09:19.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.592+0000 7f68397fa640 1 -- 172.21.15.117:0/1459699199 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f1273686-748a-40d1-9fab-ef964bbe5595", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6834010040 con 0x7f683c155b10 2024-03-30T13:09:19.595 INFO:teuthology.orchestra.run.smithi117.stdout:2 2024-03-30T13:09:19.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.592+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 msgr2=0x7f683c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.592+0000 7f6842fe2640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 0x7f683c0cd8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6834009dd0 tx=0x7f683400c490 comp rx=0 tx=0).stop 2024-03-30T13:09:19.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 shutdown_connections 2024-03-30T13:09:19.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f683c155b10 0x7f683c0cd8a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f683c0d03c0 0x7f683c14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 --2- 172.21.15.117:0/1459699199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683c0b4c90 0x7f683c1391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 >> 172.21.15.117:0/1459699199 conn(0x7f683c0d1160 msgr2=0x7f683c06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 shutdown_connections 2024-03-30T13:09:19.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.593+0000 7f6842fe2640 1 -- 172.21.15.117:0/1459699199 wait complete. 2024-03-30T13:09:19.607 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 13f783a3-d401-4d87-bb74-7ba74809c37d 3 2024-03-30T13:09:19.640 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.636+0000 7f9f89a93040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T13:09:19.696 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.693+0000 7f9f89a93040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T13:09:19.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.704+0000 7fe4a5a2a640 1 Processor -- start 2024-03-30T13:09:19.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.704+0000 7fe4a5a2a640 1 -- start start 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe4a5a2a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe4a00b3f20 0x7fe4a015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe4a5a2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 0x7fe4a00d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe4a5a2a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a00d2670 con 0x7fe4a00b4300 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe4a5a2a640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a00cc8e0 con 0x7fe4a00b3f20 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe4a5a2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00ccbc0 con 0x7fe4a00d1ca0 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 0x7fe4a00d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 0x7fe4a00d2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41780/0 (socket says 172.21.15.117:41780) 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe49f7fe640 1 -- 172.21.15.117:0/484431537 learned_addr learned my addr 172.21.15.117:0/484431537 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe49effd640 1 --2- 172.21.15.117:0/484431537 >> v2:172.21.15.66:3300/0 conn(0x7fe4a00b3f20 0x7fe4a015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe49f7fe640 1 -- 172.21.15.117:0/484431537 >> v2:172.21.15.66:3300/0 conn(0x7fe4a00b3f20 msgr2=0x7fe4a015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:19.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.705+0000 7fe49f7fe640 1 --2- 172.21.15.117:0/484431537 >> v2:172.21.15.66:3300/0 conn(0x7fe4a00b3f20 0x7fe4a015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.706+0000 7fe49f7fe640 1 -- 172.21.15.117:0/484431537 >> v1:172.21.15.66:6789/0 conn(0x7fe4a00b4300 legacy=0x7fe4a00ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.706+0000 7fe49f7fe640 1 -- 172.21.15.117:0/484431537 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a015cac0 con 0x7fe4a00d1ca0 2024-03-30T13:09:19.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.706+0000 7fe49f7fe640 1 --2- 172.21.15.117:0/484431537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 0x7fe4a00d2130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe498009e50 tx=0x7fe49802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce69f740642ade5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.706+0000 7fe49dffb640 1 -- 172.21.15.117:0/484431537 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49802ca70 con 0x7fe4a00d1ca0 2024-03-30T13:09:19.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.706+0000 7fe49dffb640 1 -- 172.21.15.117:0/484431537 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49802cc10 con 0x7fe4a00d1ca0 2024-03-30T13:09:19.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.706+0000 7fe49dffb640 1 -- 172.21.15.117:0/484431537 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980343f0 con 0x7fe4a00d1ca0 2024-03-30T13:09:19.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/484431537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 msgr2=0x7fe4a00d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/484431537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 0x7fe4a00d2130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe498009e50 tx=0x7fe49802bed0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/484431537 shutdown_connections 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/484431537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00d1ca0 0x7fe4a00d2130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/484431537 >> v2:172.21.15.66:3300/0 conn(0x7fe4a00b3f20 0x7fe4a015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/484431537 >> 172.21.15.117:0/484431537 conn(0x7fe4a00672c0 msgr2=0x7fe4a00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/484431537 shutdown_connections 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.707+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/484431537 wait complete. 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.708+0000 7fe4a5a2a640 1 Processor -- start 2024-03-30T13:09:19.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.708+0000 7fe4a5a2a640 1 -- start start 2024-03-30T13:09:19.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe4a5a2a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4a00b3f20 0x7fe4a0150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe4a5a2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 0x7fe4a01e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe4a5a2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a00d1ca0 0x7fe4a0153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe4a5a2a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00cd020 con 0x7fe4a00b3f20 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe4a5a2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00ccea0 con 0x7fe4a00d1ca0 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe4a5a2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a015d800 con 0x7fe4a00b4300 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 0x7fe4a01e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 0x7fe4a01e6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41790/0 (socket says 172.21.15.117:41790) 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 -- 172.21.15.117:0/983742805 learned_addr learned my addr 172.21.15.117:0/983742805 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49f7fe640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a00d1ca0 0x7fe4a0153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49effd640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4a00b3f20 0x7fe4a0150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 -- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a00d1ca0 msgr2=0x7fe4a0153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a00d1ca0 0x7fe4a0153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 -- 172.21.15.117:0/983742805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4a00b3f20 msgr2=0x7fe4a0150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4a00b3f20 0x7fe4a0150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49e7fc640 1 -- 172.21.15.117:0/983742805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a01526d0 con 0x7fe4a00b4300 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.709+0000 7fe49effd640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4a00b3f20 0x7fe4a0150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:19.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe49e7fc640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 0x7fe4a01e6620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c460 tx=0x7fe49000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4900052d0 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe490012630 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a01529b0 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4900054b0 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a0152f70 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe490012a20 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.710+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe49001b8e0 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.711+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a0075890 con 0x7fe4a00b4300 2024-03-30T13:09:19.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.713+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe4a00d2670 con 0x7fe4a00b4300 2024-03-30T13:09:19.741 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.737+0000 7f9f89a93040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T13:09:19.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.784+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "13f783a3-d401-4d87-bb74-7ba74809c37d", "id": 3} v 0) -- 0x7fe4a00d1a80 con 0x7fe4a00b4300 2024-03-30T13:09:19.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.788+0000 7fe487fff640 1 -- 172.21.15.117:0/983742805 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13f783a3-d401-4d87-bb74-7ba74809c37d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe49001bb80 con 0x7fe4a00b4300 2024-03-30T13:09:19.792 INFO:teuthology.orchestra.run.smithi117.stdout:3 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 msgr2=0x7fe4a01e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 0x7fe4a01e6620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c460 tx=0x7fe49000c920 comp rx=0 tx=0).stop 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 shutdown_connections 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a00d1ca0 0x7fe4a0153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4a00b4300 0x7fe4a01e6620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 --2- 172.21.15.117:0/983742805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4a00b3f20 0x7fe4a0150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 >> 172.21.15.117:0/983742805 conn(0x7fe4a00672c0 msgr2=0x7fe4a00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 shutdown_connections 2024-03-30T13:09:19.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.789+0000 7fe4a5a2a640 1 -- 172.21.15.117:0/983742805 wait complete. 2024-03-30T13:09:19.803 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new dbc2a753-dc2b-4e4f-affd-3819d14fe865 4 2024-03-30T13:09:19.864 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.861+0000 7f9f89a93040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T13:09:19.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.900+0000 7fe23c8d9640 1 Processor -- start 2024-03-30T13:09:19.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.900+0000 7fe23c8d9640 1 -- start start 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe23c8d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2380cf890 0x7fe2380d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe23c8d9640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2380b49f0 con 0x7fe238154fb0 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe23c8d9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2380cead0 con 0x7fe23815ec60 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe23c8d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe238070720 con 0x7fe2380cf890 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe236d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2380cf890 0x7fe2380d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe22ffff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe23815ec60 0x7fe2380af4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34580/0 (socket says 172.21.15.117:34580) 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe22ffff640 1 -- 172.21.15.117:0/3067586714 learned_addr learned my addr 172.21.15.117:0/3067586714 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-03-30T13:09:19.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe235d9b640 1 -- 172.21.15.117:0/3067586714 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708622322 0 0) 0x7fe2380cead0 con 0x7fe23815ec60 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe236d9d640 1 -- 172.21.15.117:0/3067586714 >> v1:172.21.15.117:6789/0 conn(0x7fe23815ec60 legacy=0x7fe2380af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe236d9d640 1 -- 172.21.15.117:0/3067586714 >> v1:172.21.15.66:6789/0 conn(0x7fe238154fb0 legacy=0x7fe23815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.901+0000 7fe236d9d640 1 -- 172.21.15.117:0/3067586714 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2380cedb0 con 0x7fe2380cf890 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.902+0000 7fe236d9d640 1 --2- 172.21.15.117:0/3067586714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2380cf890 0x7fe2380d01c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe23000aed0 tx=0x7fe23002d020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8557f0968bb4f1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.902+0000 7fe235d9b640 1 -- 172.21.15.117:0/3067586714 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23002dbc0 con 0x7fe2380cf890 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.902+0000 7fe235d9b640 1 -- 172.21.15.117:0/3067586714 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe23002dd60 con 0x7fe2380cf890 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.902+0000 7fe235d9b640 1 -- 172.21.15.117:0/3067586714 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230008660 con 0x7fe2380cf890 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 -- 172.21.15.117:0/3067586714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2380cf890 msgr2=0x7fe2380d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 --2- 172.21.15.117:0/3067586714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2380cf890 0x7fe2380d01c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe23000aed0 tx=0x7fe23002d020 comp rx=0 tx=0).stop 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 -- 172.21.15.117:0/3067586714 shutdown_connections 2024-03-30T13:09:19.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 --2- 172.21.15.117:0/3067586714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2380cf890 0x7fe2380d01c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.910 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fd4a2c10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 -- 172.21.15.117:0/3067586714 >> 172.21.15.117:0/3067586714 conn(0x7fe2380cd310 msgr2=0x7fe2380cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 -- 172.21.15.117:0/3067586714 shutdown_connections 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.903+0000 7fe23c8d9640 1 -- 172.21.15.117:0/3067586714 wait complete. 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.905+0000 7fe23c8d9640 1 Processor -- start 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.905+0000 7fe23c8d9640 1 -- start start 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.906+0000 7fe23c8d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2380cf890 0x7fe238147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.906+0000 7fe23c8d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe238154fb0 0x7fe2380b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.906+0000 7fe23c8d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe23815ec60 0x7fe2380b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:19.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.906+0000 7fe23c8d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe238155390 con 0x7fe238154fb0 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.906+0000 7fe23c8d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2380cf370 con 0x7fe2380cf890 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.906+0000 7fe23c8d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe2380b4280 con 0x7fe23815ec60 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe23659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2380cf890 0x7fe238147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe23815ec60 0x7fe2380b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe23659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2380cf890 0x7fe238147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34952/0 (socket says 172.21.15.117:34952) 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe23659c640 1 -- 172.21.15.117:0/126902140 learned_addr learned my addr 172.21.15.117:0/126902140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe22ffff640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe238154fb0 0x7fe2380b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 -- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2380cf890 msgr2=0x7fe238147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2380cf890 0x7fe238147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 -- 172.21.15.117:0/126902140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe238154fb0 msgr2=0x7fe2380b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe238154fb0 0x7fe2380b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 -- 172.21.15.117:0/126902140 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe238147860 con 0x7fe23815ec60 2024-03-30T13:09:19.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe236d9d640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe23815ec60 0x7fe2380b3ae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe230033000 tx=0x7fe230004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230003990 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe230003b30 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230004030 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe238158350 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe238158910 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe230045070 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.907+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fe2300047c0 con 0x7fe23815ec60 2024-03-30T13:09:19.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.908+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe238075890 con 0x7fe23815ec60 2024-03-30T13:09:19.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.910+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe238147860 con 0x7fe23815ec60 2024-03-30T13:09:19.938 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.934+0000 7f9f89a93040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T13:09:19.965 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:19.961+0000 7fd4a2c10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T13:09:19.987 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:19.982+0000 7f9f89a93040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T13:09:19.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.983+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dbc2a753-dc2b-4e4f-affd-3819d14fe865", "id": 4} v 0) -- 0x7fe2380b1e90 con 0x7fe23815ec60 2024-03-30T13:09:19.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.988+0000 7fe22effd640 1 -- 172.21.15.117:0/126902140 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dbc2a753-dc2b-4e4f-affd-3819d14fe865", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fe23002dc90 con 0x7fe23815ec60 2024-03-30T13:09:19.992 INFO:teuthology.orchestra.run.smithi117.stdout:4 2024-03-30T13:09:19.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe23815ec60 msgr2=0x7fe2380b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:19.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe23815ec60 0x7fe2380b3ae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe230033000 tx=0x7fe230004a40 comp rx=0 tx=0).stop 2024-03-30T13:09:19.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 shutdown_connections 2024-03-30T13:09:19.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe23815ec60 0x7fe2380b3ae0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe238154fb0 0x7fe2380b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 --2- 172.21.15.117:0/126902140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2380cf890 0x7fe238147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:19.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 >> 172.21.15.117:0/126902140 conn(0x7fe2380cd310 msgr2=0x7fe23806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:19.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 shutdown_connections 2024-03-30T13:09:19.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:19.989+0000 7fe23c8d9640 1 -- 172.21.15.117:0/126902140 wait complete. 2024-03-30T13:09:20.003 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 7c3189d7-05b3-4e03-8797-ce35e70a9db1 5 2024-03-30T13:09:20.013 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.009+0000 7fd4a2c10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T13:09:20.041 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.038+0000 7f9f89a93040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T13:09:20.070 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.066+0000 7fd4a2c10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T13:09:20.091 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.087+0000 7f9f89a93040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T13:09:20.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.099+0000 7f699198d640 1 Processor -- start 2024-03-30T13:09:20.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f699198d640 1 -- start start 2024-03-30T13:09:20.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f699198d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 0x7f698c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f699198d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f698c0b73d0 0x7f698c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f699198d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c0d2ff0 con 0x7f698c0b5dc0 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f699198d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f698c0d32d0 con 0x7f698c0b73d0 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f699198d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f698c0d35b0 con 0x7f698c0ba090 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f698b7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f698c0b73d0 0x7f698c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f698a7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 0x7f698c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698b7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f698c0b73d0 0x7f698c0b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34960/0 (socket says 172.21.15.117:34960) 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698a7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 0x7f698c0d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41824/0 (socket says 172.21.15.117:41824) 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698b7fe640 1 -- 172.21.15.117:0/623438125 learned_addr learned my addr 172.21.15.117:0/623438125 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.100+0000 7f698affd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f698c0b5dc0 0x7f698c0b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34590/0 (socket says 172.21.15.117:34590) 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f6989ffb640 1 -- 172.21.15.117:0/623438125 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852856954 0 0) 0x7f698c0d2ff0 con 0x7f698c0b5dc0 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f6989ffb640 1 -- 172.21.15.117:0/623438125 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6974003520 con 0x7f698c0b5dc0 2024-03-30T13:09:20.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698b7fe640 1 -- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 msgr2=0x7f698c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698b7fe640 1 --2- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 0x7f698c0d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698b7fe640 1 -- 172.21.15.117:0/623438125 >> v1:172.21.15.117:6789/0 conn(0x7f698c0b5dc0 legacy=0x7f698c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698b7fe640 1 -- 172.21.15.117:0/623438125 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c0d3890 con 0x7f698c0b73d0 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.101+0000 7f698a7fc640 1 --2- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 0x7f698c0d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f698b7fe640 1 --2- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b73d0 0x7f698c0b7880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6984009e50 tx=0x7f698402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d67f2a3316fe08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f6989ffb640 1 -- 172.21.15.117:0/623438125 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698402cb00 con 0x7f698c0b73d0 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f6989ffb640 1 -- 172.21.15.117:0/623438125 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698402cca0 con 0x7f698c0b73d0 2024-03-30T13:09:20.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f6989ffb640 1 -- 172.21.15.117:0/623438125 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984034610 con 0x7f698c0b73d0 2024-03-30T13:09:20.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 -- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b73d0 msgr2=0x7f698c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 --2- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b73d0 0x7f698c0b7880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6984009e50 tx=0x7f698402bf60 comp rx=0 tx=0).stop 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 -- 172.21.15.117:0/623438125 shutdown_connections 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 --2- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b73d0 0x7f698c0b7880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 --2- 172.21.15.117:0/623438125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0ba090 0x7f698c0d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 -- 172.21.15.117:0/623438125 >> 172.21.15.117:0/623438125 conn(0x7f698c0678f0 msgr2=0x7f698c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.102+0000 7f699198d640 1 -- 172.21.15.117:0/623438125 shutdown_connections 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.103+0000 7f699198d640 1 -- 172.21.15.117:0/623438125 wait complete. 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.104+0000 7f699198d640 1 Processor -- start 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.104+0000 7f699198d640 1 -- start start 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b5dc0 0x7f698c1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 0x7f698c0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f698c0ba090 0x7f698c160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15e880 con 0x7f698c0ba090 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f698c0d4090 con 0x7f698c0b5dc0 2024-03-30T13:09:20.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f698c15ea00 con 0x7f698c0b73d0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 0x7f698c0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b5dc0 0x7f698c1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 0x7f698c0cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41830/0 (socket says 172.21.15.117:41830) 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 -- 172.21.15.117:0/1713867532 learned_addr learned my addr 172.21.15.117:0/1713867532 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698b7fe640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f698c0ba090 0x7f698c160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 -- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b5dc0 msgr2=0x7f698c1f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b5dc0 0x7f698c1f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 -- 172.21.15.117:0/1713867532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f698c0ba090 msgr2=0x7f698c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f698c0ba090 0x7f698c160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 -- 172.21.15.117:0/1713867532 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c1f4680 con 0x7f698c0b73d0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698b7fe640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f698c0ba090 0x7f698c160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f698a7fc640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 0x7f698c0cb410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f697c00e930 tx=0x7f697c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c017070 con 0x7f698c0b73d0 2024-03-30T13:09:20.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c00ca00 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c00cdf0 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c1f4960 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.105+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1f4f20 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.106+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f697c01a070 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.106+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f697c013470 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.106+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c075890 con 0x7f698c0b73d0 2024-03-30T13:09:20.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.108+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f697c009360 con 0x7f698c0b73d0 2024-03-30T13:09:20.116 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.112+0000 7fd4a2c10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T13:09:20.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.181+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c3189d7-05b3-4e03-8797-ce35e70a9db1", "id": 5} v 0) -- 0x7f698c0b6520 con 0x7f698c0b73d0 2024-03-30T13:09:20.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.186+0000 7f6973fff640 1 -- 172.21.15.117:0/1713867532 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c3189d7-05b3-4e03-8797-ce35e70a9db1", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f697c005990 con 0x7f698c0b73d0 2024-03-30T13:09:20.190 INFO:teuthology.orchestra.run.smithi117.stdout:5 2024-03-30T13:09:20.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 msgr2=0x7f698c0cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 0x7f698c0cb410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f697c00e930 tx=0x7f697c00edf0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 shutdown_connections 2024-03-30T13:09:20.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f698c0ba090 0x7f698c160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f698c0b73d0 0x7f698c0cb410 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 --2- 172.21.15.117:0/1713867532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f698c0b5dc0 0x7f698c1f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 >> 172.21.15.117:0/1713867532 conn(0x7f698c0678f0 msgr2=0x7f698c0b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 shutdown_connections 2024-03-30T13:09:20.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.187+0000 7f699198d640 1 -- 172.21.15.117:0/1713867532 wait complete. 2024-03-30T13:09:20.201 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new b3a28f78-833e-4970-b45f-8527734c119d 6 2024-03-30T13:09:20.238 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.234+0000 7fd4a2c10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T13:09:20.312 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.308+0000 7fd4a2c10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T13:09:20.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.334+0000 7f3d73688640 1 Processor -- start 2024-03-30T13:09:20.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.334+0000 7f3d73688640 1 -- start start 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d73688640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d6c0b3f20 0x7f3d6c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d73688640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3d6c0b4300 0x7f3d6c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d73688640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0cc8e0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d73688640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c0ccbc0 con 0x7f3d6c0b4300 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d73688640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c15cac0 con 0x7f3d6c0b3f20 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d6c0b3f20 0x7f3d6c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d6c0b3f20 0x7f3d6c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34986/0 (socket says 172.21.15.117:34986) 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 -- 172.21.15.117:0/2670235866 learned_addr learned my addr 172.21.15.117:0/2670235866 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d70c23640 1 --2- 172.21.15.117:0/2670235866 >> v2:172.21.15.66:3300/0 conn(0x7f3d6c0b4300 0x7f3d6c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 -- 172.21.15.117:0/2670235866 >> v2:172.21.15.66:3300/0 conn(0x7f3d6c0b4300 msgr2=0x7f3d6c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:20.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 --2- 172.21.15.117:0/2670235866 >> v2:172.21.15.66:3300/0 conn(0x7f3d6c0b4300 0x7f3d6c0ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 -- 172.21.15.117:0/2670235866 >> v1:172.21.15.66:6789/0 conn(0x7f3d6c0ce9d0 legacy=0x7f3d6c0d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.335+0000 7f3d71424640 1 -- 172.21.15.117:0/2670235866 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0ccea0 con 0x7f3d6c0b3f20 2024-03-30T13:09:20.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.336+0000 7f3d71424640 1 --2- 172.21.15.117:0/2670235866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0b3f20 0x7f3d6c15d2c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c067f70 tx=0x7f3d6802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd5c7b7b8feef546 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.336+0000 7f3d63fff640 1 -- 172.21.15.117:0/2670235866 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68035020 con 0x7f3d6c0b3f20 2024-03-30T13:09:20.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.336+0000 7f3d63fff640 1 -- 172.21.15.117:0/2670235866 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d68031870 con 0x7f3d6c0b3f20 2024-03-30T13:09:20.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.336+0000 7f3d63fff640 1 -- 172.21.15.117:0/2670235866 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68031c40 con 0x7f3d6c0b3f20 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 -- 172.21.15.117:0/2670235866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0b3f20 msgr2=0x7f3d6c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 --2- 172.21.15.117:0/2670235866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0b3f20 0x7f3d6c15d2c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c067f70 tx=0x7f3d6802b910 comp rx=0 tx=0).stop 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 -- 172.21.15.117:0/2670235866 shutdown_connections 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 --2- 172.21.15.117:0/2670235866 >> v2:172.21.15.66:3300/0 conn(0x7f3d6c0b4300 0x7f3d6c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 --2- 172.21.15.117:0/2670235866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0b3f20 0x7f3d6c15d2c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 -- 172.21.15.117:0/2670235866 >> 172.21.15.117:0/2670235866 conn(0x7f3d6c0672c0 msgr2=0x7f3d6c0676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 -- 172.21.15.117:0/2670235866 shutdown_connections 2024-03-30T13:09:20.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.337+0000 7f3d73688640 1 -- 172.21.15.117:0/2670235866 wait complete. 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.338+0000 7f3d73688640 1 Processor -- start 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.338+0000 7f3d73688640 1 -- start start 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 0x7f3d6c0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4300 0x7f3d6c1e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0ce9d0 0x7f3d6c0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0d2530 con 0x7f3d6c0b4300 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0d23b0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c0d26b0 con 0x7f3d6c0b3f20 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71424640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 0x7f3d6c0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0ce9d0 0x7f3d6c0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71424640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 0x7f3d6c0ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41846/0 (socket says 172.21.15.117:41846) 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71424640 1 -- 172.21.15.117:0/3433499098 learned_addr learned my addr 172.21.15.117:0/3433499098 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d70c23640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4300 0x7f3d6c1e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 -- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 msgr2=0x7f3d6c0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 0x7f3d6c0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 -- 172.21.15.117:0/3433499098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4300 msgr2=0x7f3d6c1e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4300 0x7f3d6c1e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 -- 172.21.15.117:0/3433499098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c1e71e0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71424640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 0x7f3d6c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d71c25640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0ce9d0 0x7f3d6c0b0e90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c160fb0 tx=0x7f3d6400cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64003b90 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d64003d50 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c1e74c0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.339+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64005310 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.340+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c1f2930 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.340+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d640054b0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.340+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3d64012ab0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.340+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c075890 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.343+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3d64012d90 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.361 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.357+0000 7fd4a2c10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T13:09:20.416 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.412+0000 7fd4a2c10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T13:09:20.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.416+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b3a28f78-833e-4970-b45f-8527734c119d", "id": 6} v 0) -- 0x7f3d6c0d1a80 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.428 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.424+0000 7f3d61ffb640 1 -- 172.21.15.117:0/3433499098 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3a28f78-833e-4970-b45f-8527734c119d", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3d640124c0 con 0x7f3d6c0ce9d0 2024-03-30T13:09:20.428 INFO:teuthology.orchestra.run.smithi117.stdout:6 2024-03-30T13:09:20.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0ce9d0 msgr2=0x7f3d6c0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0ce9d0 0x7f3d6c0b0e90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c160fb0 tx=0x7f3d6400cb90 comp rx=0 tx=0).stop 2024-03-30T13:09:20.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 shutdown_connections 2024-03-30T13:09:20.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d6c0ce9d0 0x7f3d6c0b0e90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4300 0x7f3d6c1e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 --2- 172.21.15.117:0/3433499098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d6c0b3f20 0x7f3d6c0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 >> 172.21.15.117:0/3433499098 conn(0x7f3d6c0672c0 msgr2=0x7f3d6c0d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 shutdown_connections 2024-03-30T13:09:20.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.425+0000 7f3d73688640 1 -- 172.21.15.117:0/3433499098 wait complete. 2024-03-30T13:09:20.440 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 92d584f2-2882-4eaf-962f-045287f91202 7 2024-03-30T13:09:20.467 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.463+0000 7fd4a2c10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T13:09:20.493 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.490+0000 7f9f89a93040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T13:09:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.536+0000 7fe3c89a8640 1 Processor -- start 2024-03-30T13:09:20.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.536+0000 7fe3c89a8640 1 -- start start 2024-03-30T13:09:20.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c89a8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3c40d03f0 0x7fe3c415f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c89a8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c40b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c89a8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c4162ac0 con 0x7fe3c41625b0 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c89a8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c415eae0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c89a8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c4163080 con 0x7fe3c4162150 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3c40d03f0 0x7fe3c415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c2d9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe3c41625b0 0x7fe3c415ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:34600/0 (socket says 172.21.15.117:34600) 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.537+0000 7fe3c1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c40b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c2d9d640 1 -- 172.21.15.117:0/4194923105 learned_addr learned my addr 172.21.15.117:0/4194923105 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c259c640 1 -- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 msgr2=0x7fe3c40b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c259c640 1 --2- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c40b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c259c640 1 -- 172.21.15.117:0/4194923105 >> v1:172.21.15.117:6789/0 conn(0x7fe3c41625b0 legacy=0x7fe3c415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c159a640 1 -- 172.21.15.117:0/4194923105 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768242699 0 0) 0x7fe3c4162ac0 con 0x7fe3c41625b0 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c259c640 1 -- 172.21.15.117:0/4194923105 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c4162da0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.538+0000 7fe3c1d9b640 1 --2- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c40b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c259c640 1 --2- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c415f500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80097a0 tx=0x7fe3b802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e63b0480c06d1c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c159a640 1 -- 172.21.15.117:0/4194923105 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80315a0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c159a640 1 -- 172.21.15.117:0/4194923105 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b8037070 con 0x7fe3c40d03f0 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c159a640 1 -- 172.21.15.117:0/4194923105 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8031960 con 0x7fe3c40d03f0 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 -- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 msgr2=0x7fe3c415f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c415f500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80097a0 tx=0x7fe3b802b840 comp rx=0 tx=0).stop 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 -- 172.21.15.117:0/4194923105 shutdown_connections 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c40b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/4194923105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c415f500 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 -- 172.21.15.117:0/4194923105 >> 172.21.15.117:0/4194923105 conn(0x7fe3c40672c0 msgr2=0x7fe3c40676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 -- 172.21.15.117:0/4194923105 shutdown_connections 2024-03-30T13:09:20.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.539+0000 7fe3c89a8640 1 -- 172.21.15.117:0/4194923105 wait complete. 2024-03-30T13:09:20.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 Processor -- start 2024-03-30T13:09:20.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 -- start start 2024-03-30T13:09:20.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c41464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c41625b0 0x7fe3c40b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c415fa40 con 0x7fe3c41625b0 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c415f070 con 0x7fe3c40d03f0 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.541+0000 7fe3c89a8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c40cf980 con 0x7fe3c4162150 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c41464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c40c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35020/0 (socket says 172.21.15.117:35020) 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c41464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41868/0 (socket says 172.21.15.117:41868) 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 -- 172.21.15.117:0/530920346 learned_addr learned my addr 172.21.15.117:0/530920346 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c2d9d640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c41625b0 0x7fe3c40b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 -- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 msgr2=0x7fe3c41464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c41464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 -- 172.21.15.117:0/530920346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c41625b0 msgr2=0x7fe3c40b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c41625b0 0x7fe3c40b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 -- 172.21.15.117:0/530920346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c41469e0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c2d9d640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c41625b0 0x7fe3c40b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c259c640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c40c70d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80098c0 tx=0x7fe3b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8032630 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b80327d0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8032bc0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c40b5c90 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c40c79d0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3b804e7c0 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.542+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fe3b802f030 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.543+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c4075890 con 0x7fe3c40d03f0 2024-03-30T13:09:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.545+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe3b8040070 con 0x7fe3c40d03f0 2024-03-30T13:09:20.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.618+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "92d584f2-2882-4eaf-962f-045287f91202", "id": 7} v 0) -- 0x7fe3c40b8170 con 0x7fe3c40d03f0 2024-03-30T13:09:20.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.629+0000 7fe3b2ffd640 1 -- 172.21.15.117:0/530920346 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92d584f2-2882-4eaf-962f-045287f91202", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fe3b8052020 con 0x7fe3c40d03f0 2024-03-30T13:09:20.633 INFO:teuthology.orchestra.run.smithi117.stdout:7 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 msgr2=0x7fe3c40c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c40c70d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80098c0 tx=0x7fe3b80046c0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 shutdown_connections 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3c41625b0 0x7fe3c40b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3c4162150 0x7fe3c41464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 --2- 172.21.15.117:0/530920346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3c40d03f0 0x7fe3c40c70d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 >> 172.21.15.117:0/530920346 conn(0x7fe3c40672c0 msgr2=0x7fe3c40b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 shutdown_connections 2024-03-30T13:09:20.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:09:20.630+0000 7fe3c89a8640 1 -- 172.21.15.117:0/530920346 wait complete. 2024-03-30T13:09:20.635 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.632+0000 7f9f89a93040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T13:09:20.644 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-30T13:09:20.644 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-30T13:09:20.646 INFO:tasks.ceph.osd.0:Started 2024-03-30T13:09:20.646 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-30T13:09:20.647 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-30T13:09:20.651 INFO:tasks.ceph.osd.1:Started 2024-03-30T13:09:20.651 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-30T13:09:20.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-30T13:09:20.653 INFO:tasks.ceph.osd.2:Started 2024-03-30T13:09:20.653 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-30T13:09:20.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-30T13:09:20.655 INFO:tasks.ceph.osd.3:Started 2024-03-30T13:09:20.655 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-30T13:09:20.655 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-30T13:09:20.687 INFO:tasks.ceph.osd.4:Started 2024-03-30T13:09:20.687 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-30T13:09:20.687 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-30T13:09:20.689 INFO:tasks.ceph.osd.5:Started 2024-03-30T13:09:20.689 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-30T13:09:20.689 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-30T13:09:20.691 INFO:tasks.ceph.osd.6:Started 2024-03-30T13:09:20.691 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-30T13:09:20.692 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-30T13:09:20.694 INFO:tasks.ceph.osd.7:Started 2024-03-30T13:09:20.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:20.798 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.795+0000 7f9f89a93040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T13:09:20.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.810+0000 7fabd7ec8640 1 Processor -- start 2024-03-30T13:09:20.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.810+0000 7fabd7ec8640 1 -- start start 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd7ec8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd7ec8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd00d36f0 con 0x7fabd00d30d0 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd7ec8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd00d39d0 con 0x7fabd00d48c0 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd7ec8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fabd00d3cb0 con 0x7fabd015e950 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd6465640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fabd00d30d0 0x7fabd00b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46262/0 (socket says 172.21.15.66:46262) 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd6465640 1 -- 172.21.15.66:0/1453806144 learned_addr learned my addr 172.21.15.66:0/1453806144 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd5c64640 1 --2- 172.21.15.66:0/1453806144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd4c62640 1 -- 172.21.15.66:0/1453806144 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977072524 0 0) 0x7fabd00d36f0 con 0x7fabd00d30d0 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd4c62640 1 -- 172.21.15.66:0/1453806144 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabbc003520 con 0x7fabd00d30d0 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd4c62640 1 -- 172.21.15.66:0/1453806144 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1691433852 0 0) 0x7fabd00d39d0 con 0x7fabd00d48c0 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd5c64640 1 -- 172.21.15.66:0/1453806144 >> v1:172.21.15.117:6789/0 conn(0x7fabd00d48c0 legacy=0x7fabd00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd5c64640 1 -- 172.21.15.66:0/1453806144 >> v1:172.21.15.66:6789/0 conn(0x7fabd00d30d0 legacy=0x7fabd00b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.811+0000 7fabd5c64640 1 -- 172.21.15.66:0/1453806144 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd00b7ac0 con 0x7fabd015e950 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.812+0000 7fabd5c64640 1 --2- 172.21.15.66:0/1453806144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00d4380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0097d0 tx=0x7fabcc02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc391ce94250f852 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.812+0000 7fabd4c62640 1 -- 172.21.15.66:0/1453806144 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc034070 con 0x7fabd015e950 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.812+0000 7fabd4c62640 1 -- 172.21.15.66:0/1453806144 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabcc030880 con 0x7fabd015e950 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.812+0000 7fabd4c62640 1 -- 172.21.15.66:0/1453806144 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc030c50 con 0x7fabd015e950 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 -- 172.21.15.66:0/1453806144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 msgr2=0x7fabd00d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 --2- 172.21.15.66:0/1453806144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00d4380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0097d0 tx=0x7fabcc02b9c0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 -- 172.21.15.66:0/1453806144 shutdown_connections 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 --2- 172.21.15.66:0/1453806144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00d4380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 -- 172.21.15.66:0/1453806144 >> 172.21.15.66:0/1453806144 conn(0x7fabd00676e0 msgr2=0x7fabd0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 -- 172.21.15.66:0/1453806144 shutdown_connections 2024-03-30T13:09:20.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.813+0000 7fabd7ec8640 1 -- 172.21.15.66:0/1453806144 wait complete. 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.814+0000 7fabd7ec8640 1 Processor -- start 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.814+0000 7fabd7ec8640 1 -- start start 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd7ec8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 0x7fabd00ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd7ec8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabd00d48c0 0x7fabd01614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd7ec8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd7ec8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fabd015ed30 con 0x7fabd00d30d0 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd7ec8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabd00d4d00 con 0x7fabd00d48c0 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd7ec8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fabd015eeb0 con 0x7fabd015e950 2024-03-30T13:09:20.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 0x7fabd00ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 0x7fabd00ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59116/0 (socket says 172.21.15.66:59116) 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 -- 172.21.15.66:0/453717473 learned_addr learned my addr 172.21.15.66:0/453717473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd6465640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5463640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabd00d48c0 0x7fabd01614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 -- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 msgr2=0x7fabd00c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 -- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabd00d48c0 msgr2=0x7fabd01614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabd00d48c0 0x7fabd01614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.815+0000 7fabd5c64640 1 -- 172.21.15.66:0/453717473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd00cab90 con 0x7fabd00d30d0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabd5c64640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 0x7fabd00ca600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0097d0 tx=0x7fabcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc034040 con 0x7fabd00d30d0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabcc03aa90 con 0x7fabd00d30d0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd01f7a70 con 0x7fabd00d30d0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc004030 con 0x7fabd00d30d0 2024-03-30T13:09:20.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01f8030 con 0x7fabd00d30d0 2024-03-30T13:09:20.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fabcc04b060 con 0x7fabd00d30d0 2024-03-30T13:09:20.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.816+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fabcc0041d0 con 0x7fabd00d30d0 2024-03-30T13:09:20.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.817+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd0075a10 con 0x7fabd00d30d0 2024-03-30T13:09:20.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.819+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fabd00cab90 con 0x7fabd00d30d0 2024-03-30T13:09:20.848 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.845+0000 7f9f89a93040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T13:09:20.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.882+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabd00aadf0 con 0x7fabd00d30d0 2024-03-30T13:09:20.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.882+0000 7fabc67fc640 1 -- 172.21.15.66:0/453717473 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fabcc03f070 con 0x7fabd00d30d0 2024-03-30T13:09:20.886 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:20.886 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:20.620223+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 msgr2=0x7fabd00ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 0x7fabd00ca600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0097d0 tx=0x7fabcc0046c0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 shutdown_connections 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabd015e950 0x7fabd00c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabd00d48c0 0x7fabd01614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 --2- 172.21.15.66:0/453717473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabd00d30d0 0x7fabd00ca600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:20.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 >> 172.21.15.66:0/453717473 conn(0x7fabd00676e0 msgr2=0x7fabd00d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:20.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 shutdown_connections 2024-03-30T13:09:20.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:20.883+0000 7fabd7ec8640 1 -- 172.21.15.66:0/453717473 wait complete. 2024-03-30T13:09:20.897 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-30T13:09:20.897 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-30T13:09:20.898 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:20.895+0000 7fd4a2c10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T13:09:20.968 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:20.965+0000 7f9f89a93040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:09:21.030 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.027+0000 7fd4a2c10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T13:09:21.068 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:21.065+0000 7f9f89a93040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T13:09:21.140 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.136+0000 7fd4a2c10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T13:09:21.184 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.181+0000 7fd4a2c10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T13:09:21.231 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:21.227+0000 7f9f89a93040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:09:21.268 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T13:09:21.264+0000 7fd480848640 -1 Falling back to public interface 2024-03-30T13:09:21.278 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T13:09:21.273+0000 7f11eca19640 -1 Falling back to public interface 2024-03-30T13:09:21.278 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:21.273+0000 7f9f89a93040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T13:09:21.280 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T13:09:21.277+0000 7f4454d8a640 -1 Falling back to public interface 2024-03-30T13:09:21.287 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-30T13:09:21.283+0000 7f56a2eae640 -1 Falling back to public interface 2024-03-30T13:09:21.287 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-30T13:09:21.283+0000 7fcda6613640 -1 Falling back to public interface 2024-03-30T13:09:21.287 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-30T13:09:21.283+0000 7fcda501d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-30T13:09:21.288 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-30T13:09:21.283+0000 7fcda501d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-03-30T13:09:21.292 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T13:09:21.288+0000 7fdfe7ad9640 -1 Falling back to public interface 2024-03-30T13:09:21.298 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.294+0000 7fd4a2c10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:09:21.299 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-30T13:09:21.295+0000 7f1a69015640 -1 Falling back to public interface 2024-03-30T13:09:21.323 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-30T13:09:21.320+0000 7f9c6419a640 -1 Falling back to public interface 2024-03-30T13:09:21.398 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.394+0000 7fd4a2c10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T13:09:21.444 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:21.440+0000 7f9f89a93040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T13:09:21.555 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.551+0000 7fd4a2c10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:09:21.597 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.594+0000 7fd4a2c10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T13:09:21.703 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:21.750 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:21.747+0000 7fd4a2c10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T13:09:21.795 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.791+0000 7fb1ea7aa640 1 Processor -- start 2024-03-30T13:09:21.795 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1ea7aa640 1 -- start start 2024-03-30T13:09:21.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1ea7aa640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1e40d15d0 0x7fb1e4169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1ea7aa640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e4162330 con 0x7fb1e415e770 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1ea7aa640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e4162610 con 0x7fb1e40d3130 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1ea7aa640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e40b3b80 con 0x7fb1e40d15d0 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1e8d47640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1e40d15d0 0x7fb1e4169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1e3fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb1e415e770 0x7fb1e40d2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46058/0 (socket says 172.21.15.66:46058) 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.792+0000 7fb1e8d47640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1e40d15d0 0x7fb1e4169520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42962/0 (socket says 172.21.15.66:42962) 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.793+0000 7fb1e3fff640 1 -- 172.21.15.66:0/3467579199 learned_addr learned my addr 172.21.15.66:0/3467579199 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.793+0000 7fb1e8d47640 1 -- 172.21.15.66:0/3467579199 >> v1:172.21.15.117:6789/0 conn(0x7fb1e40d3130 legacy=0x7fb1e40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.793+0000 7fb1e8d47640 1 -- 172.21.15.66:0/3467579199 >> v1:172.21.15.66:6789/0 conn(0x7fb1e415e770 legacy=0x7fb1e40d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.793+0000 7fb1e2ffd640 1 -- 172.21.15.66:0/3467579199 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2537186003 0 0) 0x7fb1e4162330 con 0x7fb1e415e770 2024-03-30T13:09:21.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.793+0000 7fb1e8d47640 1 -- 172.21.15.66:0/3467579199 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e41628f0 con 0x7fb1e40d15d0 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1e8d47640 1 --2- 172.21.15.66:0/3467579199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d15d0 0x7fb1e4169520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009d90 tx=0x7fb1dc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a55016e7c85228 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1e2ffd640 1 -- 172.21.15.66:0/3467579199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc039070 con 0x7fb1e40d15d0 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1e2ffd640 1 -- 172.21.15.66:0/3467579199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1dc02cc60 con 0x7fb1e40d15d0 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1e2ffd640 1 -- 172.21.15.66:0/3467579199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc034660 con 0x7fb1e40d15d0 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/3467579199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d15d0 msgr2=0x7fb1e4169520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 --2- 172.21.15.66:0/3467579199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d15d0 0x7fb1e4169520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009d90 tx=0x7fb1dc02beb0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/3467579199 shutdown_connections 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 --2- 172.21.15.66:0/3467579199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d15d0 0x7fb1e4169520 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.799 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/3467579199 >> 172.21.15.66:0/3467579199 conn(0x7fb1e408bf90 msgr2=0x7fb1e40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:21.800 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/3467579199 shutdown_connections 2024-03-30T13:09:21.800 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.794+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/3467579199 wait complete. 2024-03-30T13:09:21.800 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.796+0000 7fb1ea7aa640 1 Processor -- start 2024-03-30T13:09:21.800 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.796+0000 7fb1ea7aa640 1 -- start start 2024-03-30T13:09:21.802 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.796+0000 7fb1ea7aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1e40d15d0 0x7fb1e41519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:21.802 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 0x7fb1e40b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:21.802 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1e415e770 0x7fb1e414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:21.802 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e40d02a0 con 0x7fb1e40d3130 2024-03-30T13:09:21.802 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4162eb0 con 0x7fb1e40d15d0 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e40d0420 con 0x7fb1e415e770 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 0x7fb1e40b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 0x7fb1e40b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42972/0 (socket says 172.21.15.66:42972) 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 -- 172.21.15.66:0/1782253325 learned_addr learned my addr 172.21.15.66:0/1782253325 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e3fff640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1e40d15d0 0x7fb1e41519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e8d47640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1e415e770 0x7fb1e414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 -- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1e415e770 msgr2=0x7fb1e414d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1e415e770 0x7fb1e414d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 -- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1e40d15d0 msgr2=0x7fb1e41519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1e40d15d0 0x7fb1e41519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 -- 172.21.15.66:0/1782253325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e40b7270 con 0x7fb1e40d3130 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e37fe640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 0x7fb1e40b2890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c480 tx=0x7fb1d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40052d0 con 0x7fb1e40d3130 2024-03-30T13:09:21.803 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d4005470 con 0x7fb1e40d3130 2024-03-30T13:09:21.804 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e41528a0 con 0x7fb1e40d3130 2024-03-30T13:09:21.804 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4012870 con 0x7fb1e40d3130 2024-03-30T13:09:21.804 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.797+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e40b2dd0 con 0x7fb1e40d3130 2024-03-30T13:09:21.804 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.798+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb1d4012a10 con 0x7fb1e40d3130 2024-03-30T13:09:21.804 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.798+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb1d401bb20 con 0x7fb1e40d3130 2024-03-30T13:09:21.804 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.798+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e40758d0 con 0x7fb1e40d3130 2024-03-30T13:09:21.805 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.800+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb1d401b450 con 0x7fb1e40d3130 2024-03-30T13:09:21.867 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.863+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1e40c7980 con 0x7fb1e40d3130 2024-03-30T13:09:21.867 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.864+0000 7fb1e0ff9640 1 -- 172.21.15.66:0/1782253325 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb1d401b630 con 0x7fb1e40d3130 2024-03-30T13:09:21.868 INFO:teuthology.misc.health.smithi066.stdout: 2024-03-30T13:09:21.868 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":10,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:20.620223+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 msgr2=0x7fb1e40b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 0x7fb1e40b2890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c480 tx=0x7fb1d400c940 comp rx=0 tx=0).stop 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 shutdown_connections 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1e415e770 0x7fb1e414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1e40d3130 0x7fb1e40b2890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 --2- 172.21.15.66:0/1782253325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1e40d15d0 0x7fb1e41519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 >> 172.21.15.66:0/1782253325 conn(0x7fb1e408bf90 msgr2=0x7fb1e415c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 shutdown_connections 2024-03-30T13:09:21.869 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:21.865+0000 7fb1ea7aa640 1 -- 172.21.15.66:0/1782253325 wait complete. 2024-03-30T13:09:21.877 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-30T13:09:22.319 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:22.315+0000 7f9f89a93040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T13:09:22.590 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:22.586+0000 7fd4a2c10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T13:09:22.611 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:22.608+0000 7f9f89a93040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T13:09:22.666 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:22.663+0000 7f9f89a93040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T13:09:22.787 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:22.784+0000 7f9f89a93040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T13:09:22.872 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:22.868+0000 7fd4a2c10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T13:09:22.924 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:22.920+0000 7fd4a2c10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T13:09:23.042 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-03-30T13:09:23.038+0000 7fd4a2c10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T13:09:23.800 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:23.797+0000 7f9f2aec2640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-30T13:09:23.800 INFO:tasks.ceph.mgr.y.smithi066.stderr:2024-03-30T13:09:23.797+0000 7f9f2aec2640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-30T13:09:24.718 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T13:09:24.715+0000 7fd480848640 -1 osd.2 0 log_to_monitors true 2024-03-30T13:09:24.763 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T13:09:24.759+0000 7f11eca19640 -1 osd.1 0 log_to_monitors true 2024-03-30T13:09:24.765 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-30T13:09:24.762+0000 7fcda6613640 -1 osd.5 0 log_to_monitors true 2024-03-30T13:09:24.792 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-30T13:09:24.788+0000 7f56a2eae640 -1 osd.6 0 log_to_monitors true 2024-03-30T13:09:24.801 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T13:09:24.797+0000 7fdfe7ad9640 -1 osd.3 0 log_to_monitors true 2024-03-30T13:09:24.819 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T13:09:24.815+0000 7f4454d8a640 -1 osd.0 0 log_to_monitors true 2024-03-30T13:09:24.822 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-30T13:09:24.818+0000 7f1a69015640 -1 osd.7 0 log_to_monitors true 2024-03-30T13:09:24.834 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-30T13:09:24.831+0000 7f9c6419a640 -1 osd.4 0 log_to_monitors true 2024-03-30T13:09:26.584 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-30T13:09:26.581+0000 7fcda2004640 -1 osd.5 0 waiting for initial osdmap 2024-03-30T13:09:26.597 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-03-30T13:09:26.594+0000 7fcd99e24640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:27.215 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T13:09:27.212+0000 7fd47c7de640 -1 osd.2 0 waiting for initial osdmap 2024-03-30T13:09:27.242 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T13:09:27.238+0000 7f11e840c640 -1 osd.1 0 waiting for initial osdmap 2024-03-30T13:09:27.254 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-03-30T13:09:27.251+0000 7fd4745fe640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:27.313 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-03-30T13:09:27.310+0000 7f11e022c640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:28.066 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T13:09:28.062+0000 7fdfe3a6f640 -1 osd.3 0 waiting for initial osdmap 2024-03-30T13:09:28.081 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-03-30T13:09:28.078+0000 7fdfdb88f640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:28.097 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T13:09:28.094+0000 7f4450d20640 -1 osd.0 0 waiting for initial osdmap 2024-03-30T13:09:28.111 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-03-30T13:09:28.108+0000 7f4448b40640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:28.476 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-30T13:09:28.472+0000 7f569ee44640 -1 osd.6 0 waiting for initial osdmap 2024-03-30T13:09:28.479 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-30T13:09:28.475+0000 7f9c60130640 -1 osd.4 0 waiting for initial osdmap 2024-03-30T13:09:28.490 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-03-30T13:09:28.487+0000 7f5696c64640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:28.495 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-03-30T13:09:28.492+0000 7f9c57f50640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:28.497 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-30T13:09:28.493+0000 7f1a64a0c640 -1 osd.7 0 waiting for initial osdmap 2024-03-30T13:09:28.520 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-03-30T13:09:28.516+0000 7f1a5c82c640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:09:28.684 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:28.789 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.785+0000 7f7159433640 1 Processor -- start 2024-03-30T13:09:28.789 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.785+0000 7f7159433640 1 -- start start 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7159433640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540d1560 0x7f715406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7159433640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f715406d910 0x7f715409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7159433640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715406e600 con 0x7f71540ce560 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7159433640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f715406f8e0 con 0x7f715406d910 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7159433640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f715406d350 con 0x7f71540d1560 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f71537fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f715406d910 0x7f715409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7152ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f71540ce560 0x7f715406c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46068/0 (socket says 172.21.15.66:46068) 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f71527fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540d1560 0x7f715406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:28.790 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.786+0000 7f7152ffd640 1 -- 172.21.15.66:0/2225914830 learned_addr learned my addr 172.21.15.66:0/2225914830 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.787+0000 7f71537fe640 1 -- 172.21.15.66:0/2225914830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540d1560 msgr2=0x7f715406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.787+0000 7f71537fe640 1 --2- 172.21.15.66:0/2225914830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540d1560 0x7f715406cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.787+0000 7f71537fe640 1 -- 172.21.15.66:0/2225914830 >> v1:172.21.15.66:6789/0 conn(0x7f71540ce560 legacy=0x7f715406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.787+0000 7f71537fe640 1 -- 172.21.15.66:0/2225914830 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715406d630 con 0x7f715406d910 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.787+0000 7f7151ffb640 1 -- 172.21.15.66:0/2225914830 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551312237 0 0) 0x7f714c001110 con 0x7f71540ce560 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.787+0000 7f71527fc640 1 --2- 172.21.15.66:0/2225914830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540d1560 0x7f715406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:28.792 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f71537fe640 1 --2- 172.21.15.66:0/2225914830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f715406d910 0x7f715409aa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7144009d90 tx=0x7f714402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d0e6af166ff99a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7151ffb640 1 -- 172.21.15.66:0/2225914830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144039070 con 0x7f715406d910 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7151ffb640 1 -- 172.21.15.66:0/2225914830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714402cbc0 con 0x7f715406d910 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7151ffb640 1 -- 172.21.15.66:0/2225914830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144034440 con 0x7f715406d910 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 -- 172.21.15.66:0/2225914830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f715406d910 msgr2=0x7f715409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 --2- 172.21.15.66:0/2225914830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f715406d910 0x7f715409aa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7144009d90 tx=0x7f714402be10 comp rx=0 tx=0).stop 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 -- 172.21.15.66:0/2225914830 shutdown_connections 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 --2- 172.21.15.66:0/2225914830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f715406d910 0x7f715409aa00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 --2- 172.21.15.66:0/2225914830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540d1560 0x7f715406cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 -- 172.21.15.66:0/2225914830 >> 172.21.15.66:0/2225914830 conn(0x7f71540b3e20 msgr2=0x7f71540b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 -- 172.21.15.66:0/2225914830 shutdown_connections 2024-03-30T13:09:28.793 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.788+0000 7f7159433640 1 -- 172.21.15.66:0/2225914830 wait complete. 2024-03-30T13:09:28.795 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.790+0000 7f7159433640 1 Processor -- start 2024-03-30T13:09:28.795 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.790+0000 7f7159433640 1 -- start start 2024-03-30T13:09:28.795 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.790+0000 7f7159433640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f715406d910 0x7f7154158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7159433640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540ce560 0x7f71540b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7159433640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 0x7f71540b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7159433640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f715409b0c0 con 0x7f71540d1560 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7159433640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f715409af40 con 0x7f715406d910 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7159433640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f715409b240 con 0x7f71540ce560 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 0x7f71540b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 0x7f71540b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43086/0 (socket says 172.21.15.66:43086) 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 -- 172.21.15.66:0/4223612393 learned_addr learned my addr 172.21.15.66:0/4223612393 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71527fc640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540ce560 0x7f71540b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7152ffd640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f715406d910 0x7f7154158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 -- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540ce560 msgr2=0x7f71540b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540ce560 0x7f71540b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.796 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 -- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f715406d910 msgr2=0x7f7154158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f715406d910 0x7f7154158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 -- 172.21.15.66:0/4223612393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71540c9400 con 0x7f71540d1560 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f71537fe640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 0x7f71540b0920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7144009eb0 tx=0x7f7144004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144039050 con 0x7f71540d1560 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144045070 con 0x7f71540d1560 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.791+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71540c96e0 con 0x7f71540d1560 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.792+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144009450 con 0x7f71540d1560 2024-03-30T13:09:28.797 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.792+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71540c9ca0 con 0x7f71540d1560 2024-03-30T13:09:28.798 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.793+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71540758d0 con 0x7f71540d1560 2024-03-30T13:09:28.798 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.793+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) ==== 90800+0+0 (secure 0 0 0) 0x7f714403e850 con 0x7f71540d1560 2024-03-30T13:09:28.798 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.793+0000 7f714b7fe640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f713406c4a0 0x7f713406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:28.798 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.794+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3071+0+0 (secure 0 0 0) 0x7f71440ad810 con 0x7f71540d1560 2024-03-30T13:09:28.798 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.794+0000 7f7152ffd640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f713406c4a0 0x7f713406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:28.798 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.794+0000 7f7152ffd640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f713406c4a0 0x7f713406e960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f714c000f60 tx=0x7f714c007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:28.802 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.798+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f714407c4c0 con 0x7f71540d1560 2024-03-30T13:09:28.991 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.988+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71540cc6e0 con 0x7f71540d1560 2024-03-30T13:09:28.992 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.988+0000 7f714b7fe640 1 -- 172.21.15.66:0/4223612393 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8526 (secure 0 0 0) 0x7f71440807c0 con 0x7f71540d1560 2024-03-30T13:09:28.992 INFO:teuthology.misc.health.smithi066.stdout: 2024-03-30T13:09:28.992 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":14,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:27.979608+0000","last_up_change":"2024-03-30T13:09:27.979608+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f713406c4a0 msgr2=0x7f713406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f713406c4a0 0x7f713406e960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f714c000f60 tx=0x7f714c007b50 comp rx=0 tx=0).stop 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 msgr2=0x7f71540b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 0x7f71540b0920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7144009eb0 tx=0x7f7144004a40 comp rx=0 tx=0).stop 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 shutdown_connections 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f713406c4a0 0x7f713406e960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.994 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71540d1560 0x7f71540b0920 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.995 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71540ce560 0x7f71540b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.995 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 --2- 172.21.15.66:0/4223612393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f715406d910 0x7f7154158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:28.995 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 >> 172.21.15.66:0/4223612393 conn(0x7f71540b3e20 msgr2=0x7f71540ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:28.995 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 shutdown_connections 2024-03-30T13:09:28.995 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:28.990+0000 7f7159433640 1 -- 172.21.15.66:0/4223612393 wait complete. 2024-03-30T13:09:29.003 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-03-30T13:09:35.810 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:35.926 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.911+0000 7f6dc2f24640 1 Processor -- start 2024-03-30T13:09:35.927 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.911+0000 7f6dc2f24640 1 -- start start 2024-03-30T13:09:35.927 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.911+0000 7f6dc2f24640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 0x7f6dbc0d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.927 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.911+0000 7f6dc2f24640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dbc15ec90 con 0x7f6dbc15e8b0 2024-03-30T13:09:35.927 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.911+0000 7f6dc2f24640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dbc0d3690 con 0x7f6dbc0b5f90 2024-03-30T13:09:35.927 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.911+0000 7f6dc2f24640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc0ced00 con 0x7f6dbc0b3870 2024-03-30T13:09:35.928 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dc0cc0640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6dbc15e8b0 0x7f6dbc0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45538/0 (socket says 172.21.15.66:45538) 2024-03-30T13:09:35.928 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dc0cc0640 1 -- 172.21.15.66:0/2243497911 learned_addr learned my addr 172.21.15.66:0/2243497911 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:35.928 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 0x7f6dbc0d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.928 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997317608 0 0) 0x7f6dbc15ec90 con 0x7f6dbc15e8b0 2024-03-30T13:09:35.928 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dac003520 con 0x7f6dbc15e8b0 2024-03-30T13:09:35.929 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503505506 0 0) 0x7f6dbc0d3690 con 0x7f6dbc0b5f90 2024-03-30T13:09:35.929 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dbc15ec90 con 0x7f6dbc0b5f90 2024-03-30T13:09:35.929 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 926742434 0 0) 0x7f6dac003520 con 0x7f6dbc15e8b0 2024-03-30T13:09:35.929 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 msgr2=0x7f6dbc0d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:35.929 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 0x7f6dbc0d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:35.929 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 >> v1:172.21.15.117:6789/0 conn(0x7f6dbc0b5f90 legacy=0x7f6dbc06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:35.930 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc0cefe0 con 0x7f6dbc15e8b0 2024-03-30T13:09:35.930 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f6db0001670 con 0x7f6dbc15e8b0 2024-03-30T13:09:35.930 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 >> v1:172.21.15.66:6789/0 conn(0x7f6dbc15e8b0 legacy=0x7f6dbc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:35.930 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dac003b70 0x7f6dac003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.930 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.930 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 0x7f6dac005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.931 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc0ced00 con 0x7f6dac004800 2024-03-30T13:09:35.931 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac00bb50 con 0x7f6dac003b70 2024-03-30T13:09:35.931 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.912+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6dac00bd20 con 0x7f6dbc0b3870 2024-03-30T13:09:35.931 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.931 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dc0cc0640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dac003b70 0x7f6dac003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.931 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dc14c1640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 0x7f6dac005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.932 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dbbfff640 1 -- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 msgr2=0x7f6dac005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:35.932 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b3870 0x7f6dac005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:35.932 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dbbfff640 1 -- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dac003b70 msgr2=0x7f6dac003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:35.932 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dac003b70 0x7f6dac003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.913+0000 7f6dbbfff640 1 -- 172.21.15.66:0/2243497911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac003520 con 0x7f6dac004800 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dac004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009af0 tx=0x7f6dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bd60b3793d9048 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8039070 con 0x7f6dac004800 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da802c6b0 con 0x7f6dac004800 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dbb7fe640 1 -- 172.21.15.66:0/2243497911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8009690 con 0x7f6dac004800 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc2f24640 1 -- 172.21.15.66:0/2243497911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 msgr2=0x7f6dac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:35.933 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/2243497911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dac004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009af0 tx=0x7f6dac005830 comp rx=0 tx=0).stop 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc14c1640 1 -- 172.21.15.66:0/2243497911 reap_dead start 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc2f24640 1 -- 172.21.15.66:0/2243497911 shutdown_connections 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc2f24640 1 -- 172.21.15.66:0/2243497911 >> 172.21.15.66:0/2243497911 conn(0x7f6dbc08bf90 msgr2=0x7f6dbc0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc2f24640 1 -- 172.21.15.66:0/2243497911 shutdown_connections 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.914+0000 7f6dc2f24640 1 -- 172.21.15.66:0/2243497911 wait complete. 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.916+0000 7f6dc2f24640 1 Processor -- start 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.916+0000 7f6dc2f24640 1 -- start start 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dbc1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dbc1f3ae0 0x7f6dbc0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b8590 0x7f6dbc0a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc0d4e50 con 0x7f6dac004800 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc0cfef0 con 0x7f6dbc1f3ae0 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc06ca10 con 0x7f6dbc0b8590 2024-03-30T13:09:35.934 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dbc1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dbc1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46490/0 (socket says 172.21.15.66:46490) 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 -- 172.21.15.66:0/1804308045 learned_addr learned my addr 172.21.15.66:0/1804308045 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc14c1640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b8590 0x7f6dbc0a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dbc1f3ae0 0x7f6dbc0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 -- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b8590 msgr2=0x7f6dbc0a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b8590 0x7f6dbc0a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 -- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dbc1f3ae0 msgr2=0x7f6dbc0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dbc1f3ae0 0x7f6dbc0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 -- 172.21.15.66:0/1804308045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc0b8a60 con 0x7f6dac004800 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dbc1f3ae0 0x7f6dbc0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:35.935 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc0cc0640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dbc1533e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6db0004f20 tx=0x7f6db00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db001a070 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db000f960 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db000fd50 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc09e9f0 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.917+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc09efb0 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.918+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc0758d0 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.919+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6db001e430 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.919+0000 7f6db97fa640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6da006c560 0x7f6da006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.919+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6db008a9f0 con 0x7f6dac004800 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.919+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6da006c560 0x7f6da006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:35.936 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.920+0000 7f6dbbfff640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6da006c560 0x7f6da006ea20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6da8036000 tx=0x7f6da8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:35.937 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:35.924+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6db0058d10 con 0x7f6dac004800 2024-03-30T13:09:36.112 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.108+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6dbc0b4700 con 0x7f6dac004800 2024-03-30T13:09:36.112 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.109+0000 7f6db97fa640 1 -- 172.21.15.66:0/1804308045 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12872 (secure 0 0 0) 0x7f6db005d010 con 0x7f6dac004800 2024-03-30T13:09:36.113 INFO:teuthology.misc.health.smithi066.stdout: 2024-03-30T13:09:36.113 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":17,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:31.068610+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6da006c560 msgr2=0x7f6da006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6da006c560 0x7f6da006ea20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6da8036000 tx=0x7f6da8030000 comp rx=0 tx=0).stop 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 msgr2=0x7f6dbc1533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dbc1533e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6db0004f20 tx=0x7f6db00074f0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 shutdown_connections 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6da006c560 0x7f6da006ea20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dbc0b8590 0x7f6dbc0a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dbc1f3ae0 0x7f6dbc0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 --2- 172.21.15.66:0/1804308045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dac004800 0x7f6dbc1533e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 >> 172.21.15.66:0/1804308045 conn(0x7f6dbc08bf90 msgr2=0x7f6dbc074a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:36.115 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 shutdown_connections 2024-03-30T13:09:36.116 INFO:teuthology.misc.health.smithi066.stderr:2024-03-30T13:09:36.111+0000 7f6dc2f24640 1 -- 172.21.15.66:0/1804308045 wait complete. 2024-03-30T13:09:36.124 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-30T13:09:36.124 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-30T13:09:36.125 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T13:09:36.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T13:09:36.127 INFO:tasks.ceph.mds.a:Started 2024-03-30T13:09:36.127 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T13:09:36.127 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T13:09:36.129 INFO:tasks.ceph.mds.c:Started 2024-03-30T13:09:36.129 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T13:09:36.129 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T13:09:36.131 INFO:tasks.ceph.mds.b:Started 2024-03-30T13:09:36.132 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T13:09:36.132 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T13:09:36.134 INFO:tasks.ceph.mds.d:Started 2024-03-30T13:09:36.134 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-30T13:09:36.134 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:36.210 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T13:09:36.211 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T13:09:36.216 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-30T13:09:36.217 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-30T13:09:36.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.243+0000 7f6de8b8f640 1 Processor -- start 2024-03-30T13:09:36.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.243+0000 7f6de8b8f640 1 -- start start 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de8b8f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6de40b3570 0x7f6de40d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de8b8f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de8b8f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de40ba3d0 con 0x7f6de40d1390 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de8b8f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f6de40cf6e0 con 0x7f6de40b3570 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de8b8f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6de40b9050 con 0x7f6de40b39d0 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6ddbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6de40b3570 0x7f6de40d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6ddbfff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6de40b3570 0x7f6de40d2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:42242/0 (socket says 172.21.15.66:42242) 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:45682/0 (socket says 172.21.15.66:45682) 2024-03-30T13:09:36.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6ddbfff640 1 -- 172.21.15.66:0/3894496199 learned_addr learned my addr 172.21.15.66:0/3894496199 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.244+0000 7f6de259c640 1 --1- 172.21.15.66:0/3894496199 >> v1:172.21.15.117:6789/0 conn(0x7f6de40d1390 0x7f6de40d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:48104/0 (socket says 172.21.15.66:48104) 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6ddbfff640 1 -- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 msgr2=0x7f6de40d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6ddbfff640 1 --2- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6ddbfff640 1 -- 172.21.15.66:0/3894496199 >> v1:172.21.15.117:6789/0 conn(0x7f6de40d1390 legacy=0x7f6de40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6de1d9b640 1 -- 172.21.15.66:0/3894496199 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649909374 0 0) 0x7f6de40ba3d0 con 0x7f6de40d1390 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6ddbfff640 1 -- 172.21.15.66:0/3894496199 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40b9330 con 0x7f6de40b3570 2024-03-30T13:09:36.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6de2d9d640 1 --2- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.245+0000 7f6ddbfff640 1 --2- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40d2480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009870 tx=0x7f6dcc02ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ad22c05ad705b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de1d9b640 1 -- 172.21.15.66:0/3894496199 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc02f850 con 0x7f6de40b3570 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de1d9b640 1 -- 172.21.15.66:0/3894496199 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc02f9f0 con 0x7f6de40b3570 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de1d9b640 1 -- 172.21.15.66:0/3894496199 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc02fdc0 con 0x7f6de40b3570 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 -- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 msgr2=0x7f6de40d2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40d2480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009870 tx=0x7f6dcc02ba30 comp rx=0 tx=0).stop 2024-03-30T13:09:36.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 -- 172.21.15.66:0/3894496199 shutdown_connections 2024-03-30T13:09:36.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/3894496199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40d2480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 -- 172.21.15.66:0/3894496199 >> 172.21.15.66:0/3894496199 conn(0x7f6de40676e0 msgr2=0x7f6de4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:36.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 -- 172.21.15.66:0/3894496199 shutdown_connections 2024-03-30T13:09:36.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.246+0000 7f6de8b8f640 1 -- 172.21.15.66:0/3894496199 wait complete. 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.247+0000 7f6de8b8f640 1 Processor -- start 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.247+0000 7f6de8b8f640 1 -- start start 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de8b8f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de8b8f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de8b8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 0x7f6de415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de8b8f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40b9bd0 con 0x7f6de40d1390 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de8b8f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40b98f0 con 0x7f6de40b3570 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de8b8f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6de40b9eb0 con 0x7f6de40b39d0 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 0x7f6de415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 0x7f6de415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46534/0 (socket says 172.21.15.66:46534) 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de2d9d640 1 -- 172.21.15.66:0/4173612967 learned_addr learned my addr 172.21.15.66:0/4173612967 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6de259c640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.248+0000 7f6ddbfff640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de2d9d640 1 -- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 msgr2=0x7f6de40cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de2d9d640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de2d9d640 1 -- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 msgr2=0x7f6de40c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de2d9d640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de2d9d640 1 -- 172.21.15.66:0/4173612967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de41f4060 con 0x7f6de40d1390 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de2d9d640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 0x7f6de415a460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00efc0 tx=0x7f6ddc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc018030 con 0x7f6de40d1390 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00cde0 con 0x7f6de40d1390 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc003bb0 con 0x7f6de40d1390 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de41f4340 con 0x7f6de40d1390 2024-03-30T13:09:36.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.249+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de41f4900 con 0x7f6de40d1390 2024-03-30T13:09:36.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.250+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de4075a10 con 0x7f6de40d1390 2024-03-30T13:09:36.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.251+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6ddc005d10 con 0x7f6de40d1390 2024-03-30T13:09:36.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.251+0000 7f6ddaffd640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6dc406c530 0x7f6dc406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.251+0000 7f6de259c640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6dc406c530 0x7f6dc406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.251+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6ddc08bc90 con 0x7f6de40d1390 2024-03-30T13:09:36.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.252+0000 7f6de259c640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6dc406c530 0x7f6dc406e9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4000f60 tx=0x7f6dd4007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.255+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6ddc022070 con 0x7f6de40d1390 2024-03-30T13:09:36.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.440+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6de40d1ab0 con 0x7f6de40d1390 2024-03-30T13:09:36.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.441+0000 7f6ddaffd640 1 -- 172.21.15.66:0/4173612967 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12872 (secure 0 0 0) 0x7f6ddc010040 con 0x7f6de40d1390 2024-03-30T13:09:36.445 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:36.445 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:31.068610+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6dc406c530 msgr2=0x7f6dc406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6dc406c530 0x7f6dc406e9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4000f60 tx=0x7f6dd4007e40 comp rx=0 tx=0).stop 2024-03-30T13:09:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 msgr2=0x7f6de415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 0x7f6de415a460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00efc0 tx=0x7f6ddc009f90 comp rx=0 tx=0).stop 2024-03-30T13:09:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 shutdown_connections 2024-03-30T13:09:36.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6dc406c530 0x7f6dc406e9f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6de40d1390 0x7f6de415a460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6de40b39d0 0x7f6de40cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 --2- 172.21.15.66:0/4173612967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6de40b3570 0x7f6de40c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.443+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 >> 172.21.15.66:0/4173612967 conn(0x7f6de40676e0 msgr2=0x7f6de40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.444+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 shutdown_connections 2024-03-30T13:09:36.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.444+0000 7f6de8b8f640 1 -- 172.21.15.66:0/4173612967 wait complete. 2024-03-30T13:09:36.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:09:36.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:09:36.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.564+0000 7ff70d488640 1 Processor -- start 2024-03-30T13:09:36.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.564+0000 7ff70d488640 1 -- start start 2024-03-30T13:09:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff70d488640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff708155b10 0x7ff7080b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff70d488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7080d03c0 0x7ff7080d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff70d488640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7080b5ff0 con 0x7ff70815fae0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff70d488640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff708155ef0 con 0x7ff708155b10 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff70d488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7080b41b0 con 0x7ff7080d03c0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff7067fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff70815fae0 0x7ff7080b57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45540/0 (socket says 172.21.15.66:45540) 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.565+0000 7ff7067fc640 1 -- 172.21.15.66:0/3203747377 learned_addr learned my addr 172.21.15.66:0/3203747377 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff7077fe640 1 --2- 172.21.15.66:0/3203747377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7080d03c0 0x7ff7080d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff706ffd640 1 --2- 172.21.15.66:0/3203747377 >> v2:172.21.15.117:3300/0 conn(0x7ff708155b10 0x7ff7080b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff705ffb640 1 -- 172.21.15.66:0/3203747377 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984628867 0 0) 0x7ff7080b5ff0 con 0x7ff70815fae0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff705ffb640 1 -- 172.21.15.66:0/3203747377 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f4003520 con 0x7ff70815fae0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff7077fe640 1 -- 172.21.15.66:0/3203747377 >> v2:172.21.15.117:3300/0 conn(0x7ff708155b10 msgr2=0x7ff7080b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff705ffb640 1 -- 172.21.15.66:0/3203747377 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302701546 0 0) 0x7ff6f4003520 con 0x7ff70815fae0 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff7077fe640 1 --2- 172.21.15.66:0/3203747377 >> v2:172.21.15.117:3300/0 conn(0x7ff708155b10 0x7ff7080b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff7077fe640 1 -- 172.21.15.66:0/3203747377 >> v1:172.21.15.66:6789/0 conn(0x7ff70815fae0 legacy=0x7ff7080b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff7077fe640 1 -- 172.21.15.66:0/3203747377 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7080b4490 con 0x7ff7080d03c0 2024-03-30T13:09:36.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.566+0000 7ff706ffd640 1 --2- 172.21.15.66:0/3203747377 >> v2:172.21.15.117:3300/0 conn(0x7ff708155b10 0x7ff7080b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff7077fe640 1 --2- 172.21.15.66:0/3203747377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7080d03c0 0x7ff7080d0850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009d90 tx=0x7ff6f802bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b06fa62c264adc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff705ffb640 1 -- 172.21.15.66:0/3203747377 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8039070 con 0x7ff7080d03c0 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff705ffb640 1 -- 172.21.15.66:0/3203747377 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f802c8a0 con 0x7ff7080d03c0 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff705ffb640 1 -- 172.21.15.66:0/3203747377 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0001670 con 0x7ff7080d03c0 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff70d488640 1 -- 172.21.15.66:0/3203747377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7080d03c0 msgr2=0x7ff7080d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff70d488640 1 --2- 172.21.15.66:0/3203747377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7080d03c0 0x7ff7080d0850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009d90 tx=0x7ff6f802bd70 comp rx=0 tx=0).stop 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff70d488640 1 -- 172.21.15.66:0/3203747377 shutdown_connections 2024-03-30T13:09:36.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff70d488640 1 --2- 172.21.15.66:0/3203747377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7080d03c0 0x7ff7080d0850 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff70d488640 1 --2- 172.21.15.66:0/3203747377 >> v2:172.21.15.117:3300/0 conn(0x7ff708155b10 0x7ff7080b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.567+0000 7ff70d488640 1 -- 172.21.15.66:0/3203747377 >> 172.21.15.66:0/3203747377 conn(0x7ff7080d1160 msgr2=0x7ff7080d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.568+0000 7ff70d488640 1 -- 172.21.15.66:0/3203747377 shutdown_connections 2024-03-30T13:09:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.568+0000 7ff70d488640 1 -- 172.21.15.66:0/3203747377 wait complete. 2024-03-30T13:09:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.569+0000 7ff70d488640 1 Processor -- start 2024-03-30T13:09:36.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.569+0000 7ff70d488640 1 -- start start 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 0x7ff70814a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff708155b10 0x7ff7081f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff70815fae0 0x7ff708151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff70815b9d0 con 0x7ff7080d03c0 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080b62d0 con 0x7ff708155b10 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff70815bb50 con 0x7ff70815fae0 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 0x7ff70814a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 0x7ff70814a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46542/0 (socket says 172.21.15.66:46542) 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 -- 172.21.15.66:0/99681343 learned_addr learned my addr 172.21.15.66:0/99681343 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff7077fe640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff70815fae0 0x7ff708151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 -- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff70815fae0 msgr2=0x7ff708151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff70815fae0 0x7ff708151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 -- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff708155b10 msgr2=0x7ff7081f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff708155b10 0x7ff7081f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 -- 172.21.15.66:0/99681343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70814f9b0 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff706ffd640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 0x7ff70814a5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff70000ebe0 tx=0x7ff70000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700017070 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70000cc30 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7081f2ab0 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700013470 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.570+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff708151010 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.571+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff708075890 con 0x7ff7080d03c0 2024-03-30T13:09:36.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.572+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff70001a070 con 0x7ff7080d03c0 2024-03-30T13:09:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.572+0000 7ff6ff7fe640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff6e806c530 0x7ff6e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.572+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff70008b2e0 con 0x7ff7080d03c0 2024-03-30T13:09:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.572+0000 7ff7067fc640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff6e806c530 0x7ff6e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.573+0000 7ff7067fc640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff6e806c530 0x7ff6e806e9f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00011e0 tx=0x7ff6f00019a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.577+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff700059ce0 con 0x7ff7080d03c0 2024-03-30T13:09:36.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.760+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff70814b420 con 0x7ff7080d03c0 2024-03-30T13:09:36.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.760+0000 7ff6ff7fe640 1 -- 172.21.15.66:0/99681343 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7ff7000595e0 con 0x7ff7080d03c0 2024-03-30T13:09:36.764 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:09:36.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff6e806c530 msgr2=0x7ff6e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff6e806c530 0x7ff6e806e9f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00011e0 tx=0x7ff6f00019a0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 msgr2=0x7ff70814a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 0x7ff70814a5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff70000ebe0 tx=0x7ff70000c520 comp rx=0 tx=0).stop 2024-03-30T13:09:36.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 shutdown_connections 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff6e806c530 0x7ff6e806e9f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff70815fae0 0x7ff708151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff708155b10 0x7ff7081f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 --2- 172.21.15.66:0/99681343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7080d03c0 0x7ff70814a5a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 >> 172.21.15.66:0/99681343 conn(0x7ff7080d1160 msgr2=0x7ff70806ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 shutdown_connections 2024-03-30T13:09:36.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.762+0000 7ff70d488640 1 -- 172.21.15.66:0/99681343 wait complete. 2024-03-30T13:09:36.776 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:09:36.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.880+0000 7f66b1140640 1 Processor -- start 2024-03-30T13:09:36.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.880+0000 7f66b1140640 1 -- start start 2024-03-30T13:09:36.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66b1140640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f66ac160540 0x7f66ac0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66b1140640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66b1140640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ac0b5f80 con 0x7f66ac145090 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66b1140640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f66ac0b6260 con 0x7f66ac160540 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66b1140640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f66ac0b6540 con 0x7f66ac0b5aa0 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66aad9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f66ac145090 0x7f66ac15fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45548/0 (socket says 172.21.15.66:45548) 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66aad9d640 1 -- 172.21.15.66:0/1954305561 learned_addr learned my addr 172.21.15.66:0/1954305561 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66ab59e640 1 --2- 172.21.15.66:0/1954305561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.881+0000 7f66aa59c640 1 --2- 172.21.15.66:0/1954305561 >> v2:172.21.15.117:3300/0 conn(0x7f66ac160540 0x7f66ac0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.882+0000 7f66a9d9b640 1 -- 172.21.15.66:0/1954305561 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3096594473 0 0) 0x7f66ac0b5f80 con 0x7f66ac145090 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.882+0000 7f66a9d9b640 1 -- 172.21.15.66:0/1954305561 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6694003520 con 0x7f66ac145090 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.882+0000 7f66ab59e640 1 -- 172.21.15.66:0/1954305561 >> v2:172.21.15.117:3300/0 conn(0x7f66ac160540 msgr2=0x7f66ac0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.882+0000 7f66ab59e640 1 --2- 172.21.15.66:0/1954305561 >> v2:172.21.15.117:3300/0 conn(0x7f66ac160540 0x7f66ac0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.882+0000 7f66ab59e640 1 -- 172.21.15.66:0/1954305561 >> v1:172.21.15.66:6789/0 conn(0x7f66ac145090 legacy=0x7f66ac15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.882+0000 7f66ab59e640 1 -- 172.21.15.66:0/1954305561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac0b6820 con 0x7f66ac0b5aa0 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66ab59e640 1 --2- 172.21.15.66:0/1954305561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0d11c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009d90 tx=0x7f66a402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52684728ae8fa655 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66a9d9b640 1 -- 172.21.15.66:0/1954305561 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4039070 con 0x7f66ac0b5aa0 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66a9d9b640 1 -- 172.21.15.66:0/1954305561 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a402cbc0 con 0x7f66ac0b5aa0 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66a9d9b640 1 -- 172.21.15.66:0/1954305561 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4034440 con 0x7f66ac0b5aa0 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 -- 172.21.15.66:0/1954305561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 msgr2=0x7f66ac0d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 --2- 172.21.15.66:0/1954305561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0d11c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009d90 tx=0x7f66a402be10 comp rx=0 tx=0).stop 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 -- 172.21.15.66:0/1954305561 shutdown_connections 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 --2- 172.21.15.66:0/1954305561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0d11c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 --2- 172.21.15.66:0/1954305561 >> v2:172.21.15.117:3300/0 conn(0x7f66ac160540 0x7f66ac0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 -- 172.21.15.66:0/1954305561 >> 172.21.15.66:0/1954305561 conn(0x7f66ac06f8b0 msgr2=0x7f66ac06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 -- 172.21.15.66:0/1954305561 shutdown_connections 2024-03-30T13:09:36.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.883+0000 7f66b1140640 1 -- 172.21.15.66:0/1954305561 wait complete. 2024-03-30T13:09:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.885+0000 7f66b1140640 1 Processor -- start 2024-03-30T13:09:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.885+0000 7f66b1140640 1 -- start start 2024-03-30T13:09:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.885+0000 7f66b1140640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 0x7f66ac1413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66ac160540 0x7f66ac1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac145470 con 0x7f66ac145090 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac146000 con 0x7f66ac160540 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f66ac1455f0 con 0x7f66ac0b5aa0 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 0x7f66ac1413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 0x7f66ac1413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46558/0 (socket says 172.21.15.66:46558) 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 -- 172.21.15.66:0/3239426796 learned_addr learned my addr 172.21.15.66:0/3239426796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aad9d640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66ab59e640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66ac160540 0x7f66ac1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 -- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 msgr2=0x7f66ac0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 -- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66ac160540 msgr2=0x7f66ac1f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66ac160540 0x7f66ac1f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 -- 172.21.15.66:0/3239426796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac0a9b90 con 0x7f66ac145090 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66ab59e640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66ac160540 0x7f66ac1f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66aa59c640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 0x7f66ac1413f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9b0 tx=0x7f66a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0017070 con 0x7f66ac145090 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a000c9a0 con 0x7f66ac145090 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000cd90 con 0x7f66ac145090 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac15e4c0 con 0x7f66ac145090 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.886+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac15ea80 con 0x7f66ac145090 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.887+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ac075a10 con 0x7f66ac145090 2024-03-30T13:09:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.888+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f66a001a070 con 0x7f66ac145090 2024-03-30T13:09:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.888+0000 7f66937fe640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f669806c530 0x7f669806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.888+0000 7f66aad9d640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f669806c530 0x7f669806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.888+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f66a008b3f0 con 0x7f66ac145090 2024-03-30T13:09:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.889+0000 7f66aad9d640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f669806c530 0x7f669806e9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f669c005880 tx=0x7f669c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:36.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:36.893+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f66ac0a9b90 con 0x7f66ac145090 2024-03-30T13:09:37.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.118+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66ac0d0e40 con 0x7f66ac145090 2024-03-30T13:09:37.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.118+0000 7f66937fe640 1 -- 172.21.15.66:0/3239426796 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f66a0059d70 con 0x7f66ac145090 2024-03-30T13:09:37.122 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:37.122 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-30T13:09:37.122 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2024-03-30T13:09:37.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f669806c530 msgr2=0x7f669806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:37.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f669806c530 0x7f669806e9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f669c005880 tx=0x7f669c009040 comp rx=0 tx=0).stop 2024-03-30T13:09:37.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 msgr2=0x7f66ac1413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:37.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 0x7f66ac1413f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9b0 tx=0x7f66a000ee70 comp rx=0 tx=0).stop 2024-03-30T13:09:37.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 shutdown_connections 2024-03-30T13:09:37.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f669806c530 0x7f669806e9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:37.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66ac160540 0x7f66ac1f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:37.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66ac145090 0x7f66ac1413f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:37.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 --2- 172.21.15.66:0/3239426796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66ac0b5aa0 0x7f66ac0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:37.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 >> 172.21.15.66:0/3239426796 conn(0x7f66ac06f8b0 msgr2=0x7f66ac15a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:37.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 shutdown_connections 2024-03-30T13:09:37.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:37.120+0000 7f66b1140640 1 -- 172.21.15.66:0/3239426796 wait complete. 2024-03-30T13:09:39.135 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:09:39.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.252+0000 7f61cdfcd640 1 Processor -- start 2024-03-30T13:09:39.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.252+0000 7f61cdfcd640 1 -- start start 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61cdfcd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 0x7f61c80b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61cdfcd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c80cf970 con 0x7f61c80b6af0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61cdfcd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c80cfc50 con 0x7f61c80cedb0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61cdfcd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f61c80b3c80 con 0x7f61c815e9c0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61c7fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f61c80b6af0 0x7f61c80d2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45558/0 (socket says 172.21.15.66:45558) 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61c7fff640 1 -- 172.21.15.66:0/443505860 learned_addr learned my addr 172.21.15.66:0/443505860 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61c77fe640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 0x7f61c80b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620878313 0 0) 0x7f61c80cf970 con 0x7f61c80b6af0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.253+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61b0003520 con 0x7f61c80b6af0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 884767703 0 0) 0x7f61b0003520 con 0x7f61c80b6af0 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 msgr2=0x7f61c80b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:39.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 0x7f61c80b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 >> v1:172.21.15.117:6789/0 conn(0x7f61c80cedb0 legacy=0x7f61c80b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80d27a0 con 0x7f61c80b6af0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f61c00019f0 con 0x7f61c80b6af0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 >> v1:172.21.15.66:6789/0 conn(0x7f61c80b6af0 legacy=0x7f61c80d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00039e0 0x7f61b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 0x7f61b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0004d40 0x7f61b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61b000ba70 con 0x7f61b00039e0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61b000bbf0 con 0x7f61b0004d40 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f61b000bdf0 con 0x7f61c815e9c0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c77fe640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00039e0 0x7f61b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c6ffd640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 0x7f61b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c7fff640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0004d40 0x7f61b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c77fe640 1 -- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 msgr2=0x7f61b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c77fe640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c815e9c0 0x7f61b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c77fe640 1 -- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0004d40 msgr2=0x7f61b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c77fe640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b0004d40 0x7f61b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.254+0000 7f61c77fe640 1 -- 172.21.15.66:0/443505860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0003520 con 0x7f61b00039e0 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.255+0000 7f61c77fe640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00039e0 0x7f61b0003e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009a90 tx=0x7f61b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22bb997209f52b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.255+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8039070 con 0x7f61b00039e0 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.255+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b802c680 con 0x7f61b00039e0 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.255+0000 7f61c67fc640 1 -- 172.21.15.66:0/443505860 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8009690 con 0x7f61b00039e0 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61cdfcd640 1 -- 172.21.15.66:0/443505860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00039e0 msgr2=0x7f61b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/443505860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b00039e0 0x7f61b0003e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009a90 tx=0x7f61b0005830 comp rx=0 tx=0).stop 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61c7fff640 1 -- 172.21.15.66:0/443505860 reap_dead start 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61cdfcd640 1 -- 172.21.15.66:0/443505860 shutdown_connections 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61cdfcd640 1 -- 172.21.15.66:0/443505860 >> 172.21.15.66:0/443505860 conn(0x7f61c80676e0 msgr2=0x7f61c8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:39.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61cdfcd640 1 -- 172.21.15.66:0/443505860 shutdown_connections 2024-03-30T13:09:39.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.256+0000 7f61cdfcd640 1 -- 172.21.15.66:0/443505860 wait complete. 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.257+0000 7f61cdfcd640 1 Processor -- start 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.257+0000 7f61cdfcd640 1 -- start start 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61cdfcd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b00039e0 0x7f61c815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61cdfcd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 0x7f61c80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61cdfcd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c81590e0 0x7f61c8159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61cdfcd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80d36b0 con 0x7f61c80cc550 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61cdfcd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80d2560 con 0x7f61b00039e0 2024-03-30T13:09:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61cdfcd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f61c80d3830 con 0x7f61c81590e0 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 0x7f61c80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 0x7f61c80cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46566/0 (socket says 172.21.15.66:46566) 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 -- 172.21.15.66:0/2898172986 learned_addr learned my addr 172.21.15.66:0/2898172986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c77fe640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b00039e0 0x7f61c815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c7fff640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c81590e0 0x7f61c8159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 -- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c81590e0 msgr2=0x7f61c8159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c81590e0 0x7f61c8159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 -- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b00039e0 msgr2=0x7f61c815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b00039e0 0x7f61c815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 -- 172.21.15.66:0/2898172986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80ccff0 con 0x7f61c80cc550 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.258+0000 7f61c6ffd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 0x7f61c80cc9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00f880 tx=0x7f61bc00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.259+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc001d20 con 0x7f61c80cc550 2024-03-30T13:09:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.259+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61bc018070 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.259+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00d730 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.259+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c80cd2d0 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.259+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c80cd890 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.260+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c8075a10 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.260+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f61bc00d8d0 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.260+0000 7f61affff640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f61b406c560 0x7f61b406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.261+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f61bc08afe0 con 0x7f61c80cc550 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.261+0000 7f61c77fe640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f61b406c560 0x7f61b406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.261+0000 7f61c77fe640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f61b406c560 0x7f61b406ea20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61b8036000 tx=0x7f61b8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.265+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f61bc059390 con 0x7f61c80cc550 2024-03-30T13:09:39.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.488+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61c80d0360 con 0x7f61c80cc550 2024-03-30T13:09:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.488+0000 7f61affff640 1 -- 172.21.15.66:0/2898172986 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f61bc05d690 con 0x7f61c80cc550 2024-03-30T13:09:39.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:39.492 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-30T13:09:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f61b406c560 msgr2=0x7f61b406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f61b406c560 0x7f61b406ea20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61b8036000 tx=0x7f61b8030000 comp rx=0 tx=0).stop 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 msgr2=0x7f61c80cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 0x7f61c80cc9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00f880 tx=0x7f61bc00fd40 comp rx=0 tx=0).stop 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 shutdown_connections 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f61b406c560 0x7f61b406ea20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61c81590e0 0x7f61c8159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61c80cc550 0x7f61c80cc9d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 --2- 172.21.15.66:0/2898172986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b00039e0 0x7f61c815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 >> 172.21.15.66:0/2898172986 conn(0x7f61c80676e0 msgr2=0x7f61c8160db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 shutdown_connections 2024-03-30T13:09:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.490+0000 7f61cdfcd640 1 -- 172.21.15.66:0/2898172986 wait complete. 2024-03-30T13:09:39.505 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.607+0000 7fbec7c21640 1 Processor -- start 2024-03-30T13:09:39.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.608+0000 7fbec7c21640 1 -- start start 2024-03-30T13:09:39.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.608+0000 7fbec7c21640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec0162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.608+0000 7fbec7c21640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbec00b0780 0x7fbec00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.608+0000 7fbec7c21640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec00b5000 con 0x7fbec00b41b0 2024-03-30T13:09:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.608+0000 7fbec7c21640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fbec0162850 con 0x7fbec00b0780 2024-03-30T13:09:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.608+0000 7fbec7c21640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00b4d20 con 0x7fbec015e450 2024-03-30T13:09:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec61be640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbec00b0780 0x7fbec00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec59bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec0162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec61be640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbec00b0780 0x7fbec00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:42324/0 (socket says 172.21.15.66:42324) 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec59bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec0162ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:45748/0 (socket says 172.21.15.66:45748) 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec61be640 1 -- 172.21.15.66:0/1736828378 learned_addr learned my addr 172.21.15.66:0/1736828378 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec59bd640 1 -- 172.21.15.66:0/1736828378 >> v2:172.21.15.117:3300/0 conn(0x7fbec00b0780 msgr2=0x7fbec00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec59bd640 1 --2- 172.21.15.66:0/1736828378 >> v2:172.21.15.117:3300/0 conn(0x7fbec00b0780 0x7fbec00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec59bd640 1 -- 172.21.15.66:0/1736828378 >> v1:172.21.15.117:6789/0 conn(0x7fbec00b41b0 legacy=0x7fbec00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec49bb640 1 -- 172.21.15.66:0/1736828378 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360512566 0 0) 0x7fbec00b5000 con 0x7fbec00b41b0 2024-03-30T13:09:39.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.609+0000 7fbec59bd640 1 -- 172.21.15.66:0/1736828378 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec0162200 con 0x7fbec015e450 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.610+0000 7fbec59bd640 1 --2- 172.21.15.66:0/1736828378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec0162ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbec01629d0 tx=0x7fbeb402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d67e5af2a9299a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.610+0000 7fbec49bb640 1 -- 172.21.15.66:0/1736828378 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4031770 con 0x7fbec015e450 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.610+0000 7fbec49bb640 1 -- 172.21.15.66:0/1736828378 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb4037070 con 0x7fbec015e450 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.610+0000 7fbec49bb640 1 -- 172.21.15.66:0/1736828378 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4031b30 con 0x7fbec015e450 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 -- 172.21.15.66:0/1736828378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 msgr2=0x7fbec0162ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 --2- 172.21.15.66:0/1736828378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec0162ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbec01629d0 tx=0x7fbeb402b810 comp rx=0 tx=0).stop 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 -- 172.21.15.66:0/1736828378 shutdown_connections 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 --2- 172.21.15.66:0/1736828378 >> v2:172.21.15.117:3300/0 conn(0x7fbec00b0780 0x7fbec00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 --2- 172.21.15.66:0/1736828378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec0162ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 -- 172.21.15.66:0/1736828378 >> 172.21.15.66:0/1736828378 conn(0x7fbec006c690 msgr2=0x7fbec006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 -- 172.21.15.66:0/1736828378 shutdown_connections 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.611+0000 7fbec7c21640 1 -- 172.21.15.66:0/1736828378 wait complete. 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.612+0000 7fbec7c21640 1 Processor -- start 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.612+0000 7fbec7c21640 1 -- start start 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec00b0780 0x7fbec0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 0x7fbec015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbec015e830 con 0x7fbec00b41b0 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbec006ed20 con 0x7fbec00b0780 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbec015e9b0 con 0x7fbec015e450 2024-03-30T13:09:39.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 0x7fbec015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 0x7fbec015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46582/0 (socket says 172.21.15.66:46582) 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 -- 172.21.15.66:0/745987419 learned_addr learned my addr 172.21.15.66:0/745987419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec61be640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec59bd640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec00b0780 0x7fbec0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 -- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 msgr2=0x7fbec015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 -- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec00b0780 msgr2=0x7fbec0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec00b0780 0x7fbec0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 -- 172.21.15.66:0/745987419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec015aea0 con 0x7fbec00b41b0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec51bc640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 0x7fbec015b970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c4b0 tx=0x7fbebc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc017070 con 0x7fbec00b41b0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc005480 con 0x7fbec00b41b0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec015b180 con 0x7fbec00b41b0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.613+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc003830 con 0x7fbec00b41b0 2024-03-30T13:09:39.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.614+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec0156180 con 0x7fbec00b41b0 2024-03-30T13:09:39.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.614+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec0075a10 con 0x7fbec00b41b0 2024-03-30T13:09:39.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.615+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbebc01a070 con 0x7fbec00b41b0 2024-03-30T13:09:39.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.615+0000 7fbeb27fc640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbea406c530 0x7fbea406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.615+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbebc08a710 con 0x7fbec00b41b0 2024-03-30T13:09:39.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.615+0000 7fbec59bd640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbea406c530 0x7fbea406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.616+0000 7fbec59bd640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbea406c530 0x7fbea406e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036000 tx=0x7fbeb403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.619+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbebc058a30 con 0x7fbec00b41b0 2024-03-30T13:09:39.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.804+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbec015b6a0 con 0x7fbec00b41b0 2024-03-30T13:09:39.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.804+0000 7fbeb27fc640 1 -- 172.21.15.66:0/745987419 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12872 (secure 0 0 0) 0x7fbebc05cd30 con 0x7fbec00b41b0 2024-03-30T13:09:39.808 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:39.808 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:31.068610+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbea406c530 msgr2=0x7fbea406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbea406c530 0x7fbea406e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4036000 tx=0x7fbeb403d040 comp rx=0 tx=0).stop 2024-03-30T13:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 msgr2=0x7fbec015b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 0x7fbec015b970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c4b0 tx=0x7fbebc00c970 comp rx=0 tx=0).stop 2024-03-30T13:09:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 shutdown_connections 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbea406c530 0x7fbea406e9f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbec015e450 0x7fbec015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec00b41b0 0x7fbec015b970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 --2- 172.21.15.66:0/745987419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec00b0780 0x7fbec0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.806+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 >> 172.21.15.66:0/745987419 conn(0x7fbec006c690 msgr2=0x7fbec0159320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.807+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 shutdown_connections 2024-03-30T13:09:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.807+0000 7fbec7c21640 1 -- 172.21.15.66:0/745987419 wait complete. 2024-03-30T13:09:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:09:39.821 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:09:39.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.928+0000 7f60cd235640 1 Processor -- start 2024-03-30T13:09:39.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.928+0000 7f60cd235640 1 -- start start 2024-03-30T13:09:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60cd235640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60c815a710 0x7f60c80b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60cd235640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f60c80718c0 0x7f60c806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60cd235640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c80b4d70 con 0x7f60c815aaf0 2024-03-30T13:09:39.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60cd235640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f60c80b41d0 con 0x7f60c80718c0 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60cd235640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f60c80b35c0 con 0x7f60c815a710 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c759e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f60c80718c0 0x7f60c806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60be59c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f60c815aaf0 0x7f60c80b4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45570/0 (socket says 172.21.15.66:45570) 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60be59c640 1 -- 172.21.15.66:0/89237357 learned_addr learned my addr 172.21.15.66:0/89237357 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c6d9d640 1 --2- 172.21.15.66:0/89237357 >> v2:172.21.15.117:3300/0 conn(0x7f60c815a710 0x7f60c80b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c759e640 1 -- 172.21.15.66:0/89237357 >> v2:172.21.15.117:3300/0 conn(0x7f60c815a710 msgr2=0x7f60c80b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c759e640 1 --2- 172.21.15.66:0/89237357 >> v2:172.21.15.117:3300/0 conn(0x7f60c815a710 0x7f60c80b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c759e640 1 -- 172.21.15.66:0/89237357 >> v1:172.21.15.66:6789/0 conn(0x7f60c815aaf0 legacy=0x7f60c80b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c759e640 1 -- 172.21.15.66:0/89237357 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c80b38a0 con 0x7f60c80718c0 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.929+0000 7f60c659c640 1 -- 172.21.15.66:0/89237357 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2322266949 0 0) 0x7f60c80b4d70 con 0x7f60c815aaf0 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60c759e640 1 --2- 172.21.15.66:0/89237357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c806d1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60c806bf40 tx=0x7f60c002cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921366ba2730897 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60c659c640 1 -- 172.21.15.66:0/89237357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c002db70 con 0x7f60c80718c0 2024-03-30T13:09:39.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60c659c640 1 -- 172.21.15.66:0/89237357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c002dd10 con 0x7f60c80718c0 2024-03-30T13:09:39.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60c659c640 1 -- 172.21.15.66:0/89237357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0008660 con 0x7f60c80718c0 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60cd235640 1 -- 172.21.15.66:0/89237357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 msgr2=0x7f60c806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60cd235640 1 --2- 172.21.15.66:0/89237357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c806d1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60c806bf40 tx=0x7f60c002cfd0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60cd235640 1 -- 172.21.15.66:0/89237357 shutdown_connections 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60cd235640 1 --2- 172.21.15.66:0/89237357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c806d1e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60cd235640 1 --2- 172.21.15.66:0/89237357 >> v2:172.21.15.117:3300/0 conn(0x7f60c815a710 0x7f60c80b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.930+0000 7f60cd235640 1 -- 172.21.15.66:0/89237357 >> 172.21.15.66:0/89237357 conn(0x7f60c80b5b00 msgr2=0x7f60c806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.931+0000 7f60cd235640 1 -- 172.21.15.66:0/89237357 shutdown_connections 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.931+0000 7f60cd235640 1 -- 172.21.15.66:0/89237357 wait complete. 2024-03-30T13:09:39.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.932+0000 7f60cd235640 1 Processor -- start 2024-03-30T13:09:39.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.932+0000 7f60cd235640 1 -- start start 2024-03-30T13:09:39.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60cd235640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c81f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60cd235640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c815a710 0x7f60c81f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60cd235640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c815aaf0 0x7f60c8158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60cd235640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8072c50 con 0x7f60c80718c0 2024-03-30T13:09:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60cd235640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60c80b3e60 con 0x7f60c815a710 2024-03-30T13:09:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60cd235640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f60c806d720 con 0x7f60c815aaf0 2024-03-30T13:09:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c81f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c81f1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46586/0 (socket says 172.21.15.66:46586) 2024-03-30T13:09:39.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 -- 172.21.15.66:0/2656178319 learned_addr learned my addr 172.21.15.66:0/2656178319 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60be59c640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c815a710 0x7f60c81f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c759e640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c815aaf0 0x7f60c8158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 -- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c815aaf0 msgr2=0x7f60c8158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c815aaf0 0x7f60c8158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 -- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c815a710 msgr2=0x7f60c81f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c815a710 0x7f60c81f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 -- 172.21.15.66:0/2656178319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81f6150 con 0x7f60c80718c0 2024-03-30T13:09:39.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60c6d9d640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c81f1980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9b0 tx=0x7f60b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8017070 con 0x7f60c80718c0 2024-03-30T13:09:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.933+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b800c9a0 con 0x7f60c80718c0 2024-03-30T13:09:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.934+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81f6430 con 0x7f60c80718c0 2024-03-30T13:09:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.934+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b800cd90 con 0x7f60c80718c0 2024-03-30T13:09:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.934+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c81e51e0 con 0x7f60c80718c0 2024-03-30T13:09:39.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.934+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c8075890 con 0x7f60c80718c0 2024-03-30T13:09:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.935+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f60b801a070 con 0x7f60c80718c0 2024-03-30T13:09:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.935+0000 7f60bffff640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f60a806c560 0x7f60a806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.936+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f60b808b680 con 0x7f60c80718c0 2024-03-30T13:09:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.936+0000 7f60be59c640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f60a806c560 0x7f60a806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:39.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.936+0000 7f60be59c640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f60a806c560 0x7f60a806ea20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60c81569d0 tx=0x7f60b0007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:39.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:39.940+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f60b805a080 con 0x7f60c80718c0 2024-03-30T13:09:40.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.123+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60c815d880 con 0x7f60c80718c0 2024-03-30T13:09:40.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.123+0000 7f60bffff640 1 -- 172.21.15.66:0/2656178319 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f60b8059980 con 0x7f60c80718c0 2024-03-30T13:09:40.127 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f60a806c560 msgr2=0x7f60a806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f60a806c560 0x7f60a806ea20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60c81569d0 tx=0x7f60b0007c00 comp rx=0 tx=0).stop 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 msgr2=0x7f60c81f1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c81f1980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9b0 tx=0x7f60b800ee70 comp rx=0 tx=0).stop 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 shutdown_connections 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f60a806c560 0x7f60a806ea20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c815aaf0 0x7f60c8158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c815a710 0x7f60c81f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 --2- 172.21.15.66:0/2656178319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60c80718c0 0x7f60c81f1980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 >> 172.21.15.66:0/2656178319 conn(0x7f60c80b5b00 msgr2=0x7f60c815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:40.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 shutdown_connections 2024-03-30T13:09:40.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.125+0000 7f60cd235640 1 -- 172.21.15.66:0/2656178319 wait complete. 2024-03-30T13:09:40.140 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T13:09:40.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.243+0000 7f484e752640 1 Processor -- start 2024-03-30T13:09:40.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.243+0000 7f484e752640 1 -- start start 2024-03-30T13:09:40.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f484e752640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f484815e950 0x7f48480d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f484e752640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48480d36f0 con 0x7f48480d30d0 2024-03-30T13:09:40.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f484e752640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48480d39d0 con 0x7f48480d48c0 2024-03-30T13:09:40.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f484e752640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f48480d3cb0 con 0x7f484815e950 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f484ccef640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f48480d30d0 0x7f48480b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45574/0 (socket says 172.21.15.66:45574) 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f484ccef640 1 -- 172.21.15.66:0/322956426 learned_addr learned my addr 172.21.15.66:0/322956426 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762537471 0 0) 0x7f48480d36f0 con 0x7f48480d30d0 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4830003520 con 0x7f48480d30d0 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.244+0000 7f4847fff640 1 --2- 172.21.15.66:0/322956426 >> v2:172.21.15.117:3300/0 conn(0x7f484815e950 0x7f48480d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2247462548 0 0) 0x7f4830003520 con 0x7f48480d30d0 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 >> v2:172.21.15.117:3300/0 conn(0x7f484815e950 msgr2=0x7f48480d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 --2- 172.21.15.66:0/322956426 >> v2:172.21.15.117:3300/0 conn(0x7f484815e950 0x7f48480d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 >> v1:172.21.15.117:6789/0 conn(0x7f48480d48c0 legacy=0x7f48480d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48480b7ac0 con 0x7f48480d30d0 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f48400019f0 con 0x7f48480d30d0 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 >> v1:172.21.15.66:6789/0 conn(0x7f48480d30d0 legacy=0x7f48480b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4830004800 0x7f4830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48300051c0 0x7f483000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48480d3cb0 con 0x7f48300039b0 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f483000c070 con 0x7f48300051c0 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f483000c270 con 0x7f4830004800 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4847fff640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f484ccef640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48300051c0 0x7f483000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f48477fe640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4830004800 0x7f4830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4847fff640 1 -- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4830004800 msgr2=0x7f4830004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4847fff640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4830004800 0x7f4830004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4847fff640 1 -- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48300051c0 msgr2=0x7f483000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4847fff640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48300051c0 0x7f483000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f4847fff640 1 -- 172.21.15.66:0/322956426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4830003520 con 0x7f48300039b0 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f484ccef640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48300051c0 0x7f483000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.245+0000 7f484ccef640 1 -- 172.21.15.66:0/322956426 reap_dead start 2024-03-30T13:09:40.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.246+0000 7f4847fff640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4838009ac0 tx=0x7f4830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894fbd6ffeff6ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:40.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.246+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838039070 con 0x7f48300039b0 2024-03-30T13:09:40.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.246+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483802c6b0 con 0x7f48300039b0 2024-03-30T13:09:40.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.246+0000 7f4846ffd640 1 -- 172.21.15.66:0/322956426 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838009690 con 0x7f48300039b0 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 -- 172.21.15.66:0/322956426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48300039b0 msgr2=0x7f4830003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4838009ac0 tx=0x7f4830005830 comp rx=0 tx=0).stop 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 -- 172.21.15.66:0/322956426 shutdown_connections 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 --2- 172.21.15.66:0/322956426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 -- 172.21.15.66:0/322956426 >> 172.21.15.66:0/322956426 conn(0x7f48480676e0 msgr2=0x7f4848067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 -- 172.21.15.66:0/322956426 shutdown_connections 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.247+0000 7f484e752640 1 -- 172.21.15.66:0/322956426 wait complete. 2024-03-30T13:09:40.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.248+0000 7f484e752640 1 Processor -- start 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.248+0000 7f484e752640 1 -- start start 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48300039b0 0x7f484814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 0x7f4848154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48480c8ca0 0x7f48480c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f484815ed30 con 0x7f4848153940 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48480d4d00 con 0x7f48480c8ca0 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f484815eeb0 con 0x7f48300039b0 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 0x7f4848154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 0x7f4848154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46602/0 (socket says 172.21.15.66:46602) 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 -- 172.21.15.66:0/3317131658 learned_addr learned my addr 172.21.15.66:0/3317131658 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f4847fff640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48300039b0 0x7f484814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 -- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48300039b0 msgr2=0x7f484814a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48300039b0 0x7f484814a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484ccef640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48480c8ca0 0x7f48480c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 -- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48480c8ca0 msgr2=0x7f48480c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48480c8ca0 0x7f48480c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 -- 172.21.15.66:0/3317131658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48480c8970 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f4847fff640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48300039b0 0x7f484814a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484ccef640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48480c8ca0 0x7f48480c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f48477fe640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 0x7f4848154730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f483c00f850 tx=0x7f483c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00fe40 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483c018070 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48480c83b0 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00d6d0 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.249+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4848157440 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.250+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848075a10 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.251+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f483c00d870 con 0x7f4848153940 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.251+0000 7f4844ff9640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f483406c530 0x7f483406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.251+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f483c08b1f0 con 0x7f4848153940 2024-03-30T13:09:40.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.251+0000 7f4847fff640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f483406c530 0x7f483406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.252+0000 7f4847fff640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f483406c530 0x7f483406e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4838036000 tx=0x7f4838030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:40.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.256+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f483c059ad0 con 0x7f4848153940 2024-03-30T13:09:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.442+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f484815e070 con 0x7f4848153940 2024-03-30T13:09:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.443+0000 7f4844ff9640 1 -- 172.21.15.66:0/3317131658 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f483c0593d0 con 0x7f4848153940 2024-03-30T13:09:40.446 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:40.446 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T13:09:40.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:09:40.446 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T13:09:40.447 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T13:09:40.447 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:09:40.447 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T13:09:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f483406c530 msgr2=0x7f483406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f483406c530 0x7f483406e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4838036000 tx=0x7f4838030000 comp rx=0 tx=0).stop 2024-03-30T13:09:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 msgr2=0x7f4848154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 0x7f4848154730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f483c00f850 tx=0x7f483c00fd10 comp rx=0 tx=0).stop 2024-03-30T13:09:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 shutdown_connections 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f483406c530 0x7f483406e9f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48480c8ca0 0x7f48480c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4848153940 0x7f4848154730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 --2- 172.21.15.66:0/3317131658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48300039b0 0x7f484814a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.444+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 >> 172.21.15.66:0/3317131658 conn(0x7f48480676e0 msgr2=0x7f48480cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.445+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 shutdown_connections 2024-03-30T13:09:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.445+0000 7f484e752640 1 -- 172.21.15.66:0/3317131658 wait complete. 2024-03-30T13:09:40.459 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T13:09:40.459 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T13:09:40.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.566+0000 7f5a2fe16640 1 Processor -- start 2024-03-30T13:09:40.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.566+0000 7f5a2fe16640 1 -- start start 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2fe16640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5a280b0900 0x7f5a28072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2fe16640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a2806f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2fe16640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2815eac0 con 0x7f5a2815e6e0 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2fe16640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5a28073430 con 0x7f5a280b0900 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2fe16640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b4110 con 0x7f5a2806bc90 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2d3b1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5a280b0900 0x7f5a28072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.567+0000 7f5a2d3b1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5a280b0900 0x7f5a28072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46608/0 (socket says 172.21.15.66:46608) 2024-03-30T13:09:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.568+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/2240470280 learned_addr learned my addr 172.21.15.66:0/2240470280 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.568+0000 7f5a2e3b3640 1 --2- 172.21.15.66:0/2240470280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a2806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.568+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/2240470280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 msgr2=0x7f5a2806f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.568+0000 7f5a2d3b1640 1 --2- 172.21.15.66:0/2240470280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a2806f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.568+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/2240470280 >> v1:172.21.15.117:6789/0 conn(0x7f5a2815e6e0 legacy=0x7f5a2806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.568+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/2240470280 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280b43f0 con 0x7f5a280b0900 2024-03-30T13:09:40.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2d3b1640 1 --2- 172.21.15.66:0/2240470280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a28072ef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009870 tx=0x7f5a1802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd21c9e036713d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:40.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2cbb0640 1 -- 172.21.15.66:0/2240470280 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18034070 con 0x7f5a280b0900 2024-03-30T13:09:40.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2cbb0640 1 -- 172.21.15.66:0/2240470280 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a18030840 con 0x7f5a280b0900 2024-03-30T13:09:40.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2cbb0640 1 -- 172.21.15.66:0/2240470280 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18030c10 con 0x7f5a280b0900 2024-03-30T13:09:40.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 -- 172.21.15.66:0/2240470280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 msgr2=0x7f5a28072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/2240470280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a28072ef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009870 tx=0x7f5a1802b980 comp rx=0 tx=0).stop 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 -- 172.21.15.66:0/2240470280 shutdown_connections 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/2240470280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a2806f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/2240470280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a28072ef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 -- 172.21.15.66:0/2240470280 >> 172.21.15.66:0/2240470280 conn(0x7f5a280d1180 msgr2=0x7f5a280d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 -- 172.21.15.66:0/2240470280 shutdown_connections 2024-03-30T13:09:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.569+0000 7f5a2fe16640 1 -- 172.21.15.66:0/2240470280 wait complete. 2024-03-30T13:09:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 Processor -- start 2024-03-30T13:09:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 -- start start 2024-03-30T13:09:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a281e8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a2815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a2815e6e0 0x7f5a2815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b4c90 con 0x7f5a280b0900 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b49b0 con 0x7f5a2815e6e0 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2fe16640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b4f70 con 0x7f5a2806bc90 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2d3b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a2815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2d3b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a2815ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46610/0 (socket says 172.21.15.66:46610) 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.571+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/1070647416 learned_addr learned my addr 172.21.15.66:0/1070647416 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2dbb2640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a281e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2e3b3640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a2815e6e0 0x7f5a2815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 msgr2=0x7f5a281e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2d3b1640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a281e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a2815e6e0 msgr2=0x7f5a2815fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2d3b1640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a2815e6e0 0x7f5a2815fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2d3b1640 1 -- 172.21.15.66:0/1070647416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2815b060 con 0x7f5a280b0900 2024-03-30T13:09:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2e3b3640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a2815e6e0 0x7f5a2815fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2d3b1640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a2815ab20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a18000c00 tx=0x7f5a18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18048070 con 0x7f5a280b0900 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a18030640 con 0x7f5a280b0900 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a28160120 con 0x7f5a280b0900 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18034040 con 0x7f5a280b0900 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.572+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281606d0 con 0x7f5a280b0900 2024-03-30T13:09:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.573+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28075900 con 0x7f5a280b0900 2024-03-30T13:09:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.574+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5a1804b070 con 0x7f5a280b0900 2024-03-30T13:09:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.574+0000 7f5a1e7fc640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5a0c06c530 0x7f5a0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.574+0000 7f5a2dbb2640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5a0c06c530 0x7f5a0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.574+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5a180ad290 con 0x7f5a280b0900 2024-03-30T13:09:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.575+0000 7f5a2dbb2640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5a0c06c530 0x7f5a0c06e9f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a20007fd0 tx=0x7f5a20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:40.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.579+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5a1807bb70 con 0x7f5a280b0900 2024-03-30T13:09:40.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:40.763+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5a2815ba90 con 0x7f5a280b0900 2024-03-30T13:09:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.091+0000 7f5a1e7fc640 1 -- 172.21.15.66:0/1070647416 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f5a18030810 con 0x7f5a280b0900 2024-03-30T13:09:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T13:09:41.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5a0c06c530 msgr2=0x7f5a0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5a0c06c530 0x7f5a0c06e9f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a20007fd0 tx=0x7f5a20009040 comp rx=0 tx=0).stop 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 msgr2=0x7f5a2815ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a2815ab20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a18000c00 tx=0x7f5a18004730 comp rx=0 tx=0).stop 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 shutdown_connections 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5a0c06c530 0x7f5a0c06e9f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a2815e6e0 0x7f5a2815fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a280b0900 0x7f5a2815ab20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 --2- 172.21.15.66:0/1070647416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a2806bc90 0x7f5a281e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 >> 172.21.15.66:0/1070647416 conn(0x7f5a280d1180 msgr2=0x7f5a281573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 shutdown_connections 2024-03-30T13:09:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.094+0000 7f5a2fe16640 1 -- 172.21.15.66:0/1070647416 wait complete. 2024-03-30T13:09:41.113 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-30T13:09:41.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.225+0000 7f7ebd782640 1 Processor -- start 2024-03-30T13:09:41.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.225+0000 7f7ebd782640 1 -- start start 2024-03-30T13:09:41.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7ebd782640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7eb80b4110 0x7f7eb80b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:41.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7ebd782640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb80b5010 con 0x7f7eb80b4c00 2024-03-30T13:09:41.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7ebd782640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb8070670 con 0x7f7eb80d07b0 2024-03-30T13:09:41.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7ebd782640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb80b54d0 con 0x7f7eb80b4110 2024-03-30T13:09:41.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7eb6ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7eb80b4110 0x7f7eb80b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:41.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7eb77fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7eb80b4c00 0x7f7eb8072590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:45576/0 (socket says 172.21.15.66:45576) 2024-03-30T13:09:41.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.226+0000 7f7eb77fe640 1 -- 172.21.15.66:0/946221160 learned_addr learned my addr 172.21.15.66:0/946221160 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:41.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.227+0000 7f7eb6ffd640 1 -- 172.21.15.66:0/946221160 >> v1:172.21.15.117:6789/0 conn(0x7f7eb80d07b0 legacy=0x7f7eb80b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.227+0000 7f7eb6ffd640 1 -- 172.21.15.66:0/946221160 >> v1:172.21.15.66:6789/0 conn(0x7f7eb80b4c00 legacy=0x7f7eb8072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.227+0000 7f7eb6ffd640 1 -- 172.21.15.66:0/946221160 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb80b57b0 con 0x7f7eb80b4110 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.227+0000 7f7eb6ffd640 1 --2- 172.21.15.66:0/946221160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4110 0x7f7eb80b5fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00097a0 tx=0x7f7eb002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325ec2508cd413bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.227+0000 7f7eb5ffb640 1 -- 172.21.15.66:0/946221160 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0034070 con 0x7f7eb80b4110 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7eb5ffb640 1 -- 172.21.15.66:0/946221160 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb0030610 con 0x7f7eb80b4110 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7eb5ffb640 1 -- 172.21.15.66:0/946221160 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00309e0 con 0x7f7eb80b4110 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 -- 172.21.15.66:0/946221160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4110 msgr2=0x7f7eb80b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 --2- 172.21.15.66:0/946221160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4110 0x7f7eb80b5fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00097a0 tx=0x7f7eb002b960 comp rx=0 tx=0).stop 2024-03-30T13:09:41.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 -- 172.21.15.66:0/946221160 shutdown_connections 2024-03-30T13:09:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 --2- 172.21.15.66:0/946221160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4110 0x7f7eb80b5fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 -- 172.21.15.66:0/946221160 >> 172.21.15.66:0/946221160 conn(0x7f7eb80d1180 msgr2=0x7f7eb80d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 -- 172.21.15.66:0/946221160 shutdown_connections 2024-03-30T13:09:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.228+0000 7f7ebd782640 1 -- 172.21.15.66:0/946221160 wait complete. 2024-03-30T13:09:41.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.229+0000 7f7ebd782640 1 Processor -- start 2024-03-30T13:09:41.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.229+0000 7f7ebd782640 1 -- start start 2024-03-30T13:09:41.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7ebd782640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80b4110 0x7f7eb8151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:41.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7ebd782640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 0x7f7eb81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7ebd782640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7eb80d07b0 0x7f7eb8145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7ebd782640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb815b540 con 0x7f7eb80b4c00 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7ebd782640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8070950 con 0x7f7eb80b4110 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7ebd782640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb815b6c0 con 0x7f7eb80d07b0 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 0x7f7eb81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 0x7f7eb81575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46618/0 (socket says 172.21.15.66:46618) 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7eb67fc640 1 -- 172.21.15.66:0/4157369768 learned_addr learned my addr 172.21.15.66:0/4157369768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.230+0000 7f7eb6ffd640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80b4110 0x7f7eb8151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb77fe640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7eb80d07b0 0x7f7eb8145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb67fc640 1 -- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7eb80d07b0 msgr2=0x7f7eb8145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb67fc640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7eb80d07b0 0x7f7eb8145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb67fc640 1 -- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80b4110 msgr2=0x7f7eb8151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb67fc640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80b4110 0x7f7eb8151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb67fc640 1 -- 172.21.15.66:0/4157369768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb8157af0 con 0x7f7eb80b4c00 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb6ffd640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80b4110 0x7f7eb8151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb77fe640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7eb80d07b0 0x7f7eb8145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eb67fc640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 0x7f7eb81575b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c400 tx=0x7f7ea000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00052d0 con 0x7f7eb80b4c00 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea0005490 con 0x7f7eb80b4c00 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb815a850 con 0x7f7eb80b4c00 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0007260 con 0x7f7eb80b4c00 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.231+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb8158070 con 0x7f7eb80b4c00 2024-03-30T13:09:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.232+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb8075890 con 0x7f7eb80b4c00 2024-03-30T13:09:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.232+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ea0007400 con 0x7f7eb80b4c00 2024-03-30T13:09:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.233+0000 7f7eaf7fe640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7e9806c530 0x7f7e9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:41.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.233+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f7ea008aa70 con 0x7f7eb80b4c00 2024-03-30T13:09:41.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.233+0000 7f7eb6ffd640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7e9806c530 0x7f7e9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:41.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.233+0000 7f7eb6ffd640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7e9806c530 0x7f7e9806e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0036000 tx=0x7f7eb0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.238+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ea0059280 con 0x7f7eb80b4c00 2024-03-30T13:09:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:41.422+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7eb806fef0 con 0x7f7eb80b4c00 2024-03-30T13:09:41.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:41.830 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:42.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.101+0000 7f7eaf7fe640 1 -- 172.21.15.66:0/4157369768 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f7ea0058b80 con 0x7f7eb80b4c00 2024-03-30T13:09:42.105 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7e9806c530 msgr2=0x7f7e9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7e9806c530 0x7f7e9806e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0036000 tx=0x7f7eb0008030 comp rx=0 tx=0).stop 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 msgr2=0x7f7eb81575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 0x7f7eb81575b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c400 tx=0x7f7ea000c8c0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 shutdown_connections 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7e9806c530 0x7f7e9806e9f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7eb80d07b0 0x7f7eb8145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eb80b4c00 0x7f7eb81575b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 --2- 172.21.15.66:0/4157369768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80b4110 0x7f7eb8151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 >> 172.21.15.66:0/4157369768 conn(0x7f7eb80d1180 msgr2=0x7f7eb815da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.103+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 shutdown_connections 2024-03-30T13:09:42.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.104+0000 7f7ebd782640 1 -- 172.21.15.66:0/4157369768 wait complete. 2024-03-30T13:09:42.134 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T13:09:42.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.243+0000 7f63f4b46640 1 Processor -- start 2024-03-30T13:09:42.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.243+0000 7f63f4b46640 1 -- start start 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.244+0000 7f63f4b46640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f63f0167120 0x7f63f0169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.244+0000 7f63f4b46640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f00d2ff0 con 0x7f63f00ba090 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.244+0000 7f63f4b46640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f00d32d0 con 0x7f63f00b5dc0 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.244+0000 7f63f4b46640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f63f00d2d90 con 0x7f63f0167120 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.245+0000 7f63eed9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f63f0167120 0x7f63f0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.245+0000 7f63edd9b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f63f00ba090 0x7f63f00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34122/0 (socket says 172.21.15.66:34122) 2024-03-30T13:09:42.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.245+0000 7f63edd9b640 1 -- 172.21.15.66:0/549809291 learned_addr learned my addr 172.21.15.66:0/549809291 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:42.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.245+0000 7f63eed9d640 1 -- 172.21.15.66:0/549809291 >> v1:172.21.15.117:6789/0 conn(0x7f63f00b5dc0 legacy=0x7f63f00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:42.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.245+0000 7f63eed9d640 1 -- 172.21.15.66:0/549809291 >> v1:172.21.15.66:6789/0 conn(0x7f63f00ba090 legacy=0x7f63f00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:42.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.245+0000 7f63eed9d640 1 -- 172.21.15.66:0/549809291 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f00d35b0 con 0x7f63f0167120 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63eed9d640 1 --2- 172.21.15.66:0/549809291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f0167120 0x7f63f0169510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009dc0 tx=0x7f63e002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51de243856c0bc68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63ed59a640 1 -- 172.21.15.66:0/549809291 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0039070 con 0x7f63f0167120 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63ed59a640 1 -- 172.21.15.66:0/549809291 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e002cc40 con 0x7f63f0167120 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63ed59a640 1 -- 172.21.15.66:0/549809291 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0034440 con 0x7f63f0167120 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 -- 172.21.15.66:0/549809291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f0167120 msgr2=0x7f63f0169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 --2- 172.21.15.66:0/549809291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f0167120 0x7f63f0169510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009dc0 tx=0x7f63e002be90 comp rx=0 tx=0).stop 2024-03-30T13:09:42.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 -- 172.21.15.66:0/549809291 shutdown_connections 2024-03-30T13:09:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 --2- 172.21.15.66:0/549809291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f0167120 0x7f63f0169510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 -- 172.21.15.66:0/549809291 >> 172.21.15.66:0/549809291 conn(0x7f63f00678f0 msgr2=0x7f63f00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 -- 172.21.15.66:0/549809291 shutdown_connections 2024-03-30T13:09:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.246+0000 7f63f4b46640 1 -- 172.21.15.66:0/549809291 wait complete. 2024-03-30T13:09:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.248+0000 7f63f4b46640 1 Processor -- start 2024-03-30T13:09:42.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.248+0000 7f63f4b46640 1 -- start start 2024-03-30T13:09:42.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 0x7f63f0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:42.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f00ba090 0x7f63f015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:42.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f0167120 0x7f63f015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63f00d4090 con 0x7f63f00b5dc0 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63f00d3b70 con 0x7f63f0167120 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f63f00b83e0 con 0x7f63f00ba090 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 0x7f63f0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 0x7f63f0158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41338/0 (socket says 172.21.15.66:41338) 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 -- 172.21.15.66:0/3541218997 learned_addr learned my addr 172.21.15.66:0/3541218997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63edd9b640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f00ba090 0x7f63f015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63eed9d640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f0167120 0x7f63f015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 -- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f00ba090 msgr2=0x7f63f015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f00ba090 0x7f63f015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 -- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f0167120 msgr2=0x7f63f015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f0167120 0x7f63f015cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 -- 172.21.15.66:0/3541218997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f015b1d0 con 0x7f63f00b5dc0 2024-03-30T13:09:42.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63ee59c640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 0x7f63f0158360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b3f0 tx=0x7f63e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:42.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8015020 con 0x7f63f00b5dc0 2024-03-30T13:09:42.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e8011830 con 0x7f63f00b5dc0 2024-03-30T13:09:42.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f015b4b0 con 0x7f63f00b5dc0 2024-03-30T13:09:42.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8011c20 con 0x7f63f00b5dc0 2024-03-30T13:09:42.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.249+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01588a0 con 0x7f63f00b5dc0 2024-03-30T13:09:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.250+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f0075890 con 0x7f63f00b5dc0 2024-03-30T13:09:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.251+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f63e8018020 con 0x7f63f00b5dc0 2024-03-30T13:09:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.251+0000 7f63e6ffd640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f63d006c530 0x7f63d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.252+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f63e808b280 con 0x7f63f00b5dc0 2024-03-30T13:09:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.252+0000 7f63edd9b640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f63d006c530 0x7f63d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:42.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.252+0000 7f63edd9b640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f63d006c530 0x7f63d006e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63f015ddc0 tx=0x7f63d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:42.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.256+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f63e801f030 con 0x7f63f00b5dc0 2024-03-30T13:09:42.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:42.478+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f63f015fd30 con 0x7f63f00b5dc0 2024-03-30T13:09:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.108+0000 7f63e6ffd640 1 -- 172.21.15.66:0/3541218997 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f63e80172c0 con 0x7f63f00b5dc0 2024-03-30T13:09:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2024-03-30T13:09:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.117+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f63d006c530 msgr2=0x7f63d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.117+0000 7f63f4b46640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f63d006c530 0x7f63d006e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63f015ddc0 tx=0x7f63d8009040 comp rx=0 tx=0).stop 2024-03-30T13:09:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.117+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 msgr2=0x7f63f0158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.117+0000 7f63f4b46640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 0x7f63f0158360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b3f0 tx=0x7f63e800b8b0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.119+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 shutdown_connections 2024-03-30T13:09:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.119+0000 7f63f4b46640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f63d006c530 0x7f63d006e9f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.119+0000 7f63f4b46640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f0167120 0x7f63f015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.119+0000 7f63f4b46640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f00ba090 0x7f63f015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.119+0000 7f63f4b46640 1 --2- 172.21.15.66:0/3541218997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63f00b5dc0 0x7f63f0158360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.119+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 >> 172.21.15.66:0/3541218997 conn(0x7f63f00678f0 msgr2=0x7f63f00b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.120+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 shutdown_connections 2024-03-30T13:09:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.121+0000 7f63f4b46640 1 -- 172.21.15.66:0/3541218997 wait complete. 2024-03-30T13:09:43.147 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-30T13:09:43.147 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-30T13:09:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.259+0000 7f404c0c8640 1 Processor -- start 2024-03-30T13:09:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.259+0000 7f404c0c8640 1 -- start start 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f404c0c8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f404415e4d0 0x7f40440ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f404c0c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40440cc7f0 0x7f40440ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f404c0c8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40440cf030 con 0x7f404406c690 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f404c0c8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4044162100 con 0x7f404415e4d0 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f404c0c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4044162f80 con 0x7f40440cc7f0 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f4049e64640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f404415e4d0 0x7f40440ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f4049663640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f404406c690 0x7f4044162870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34134/0 (socket says 172.21.15.66:34134) 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.260+0000 7f4049663640 1 -- 172.21.15.66:0/4209045073 learned_addr learned my addr 172.21.15.66:0/4209045073 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f404a665640 1 --2- 172.21.15.66:0/4209045073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40440cc7f0 0x7f40440ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f4049e64640 1 -- 172.21.15.66:0/4209045073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40440cc7f0 msgr2=0x7f40440ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f4049e64640 1 --2- 172.21.15.66:0/4209045073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40440cc7f0 0x7f40440ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f4049e64640 1 -- 172.21.15.66:0/4209045073 >> v1:172.21.15.66:6789/0 conn(0x7f404406c690 legacy=0x7f4044162870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f4049e64640 1 -- 172.21.15.66:0/4209045073 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440cf2d0 con 0x7f404415e4d0 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f4048e62640 1 -- 172.21.15.66:0/4209045073 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4056195450 0 0) 0x7f40440cf030 con 0x7f404406c690 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.261+0000 7f4049e64640 1 --2- 172.21.15.66:0/4209045073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f404415e4d0 0x7f40440ceaf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3a6cf4aca5a1fa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f4048e62640 1 -- 172.21.15.66:0/4209045073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0315a0 con 0x7f404415e4d0 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f4048e62640 1 -- 172.21.15.66:0/4209045073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403c037070 con 0x7f404415e4d0 2024-03-30T13:09:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f4048e62640 1 -- 172.21.15.66:0/4209045073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c031960 con 0x7f404415e4d0 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 -- 172.21.15.66:0/4209045073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f404415e4d0 msgr2=0x7f40440ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 --2- 172.21.15.66:0/4209045073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f404415e4d0 0x7f40440ceaf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c02b840 comp rx=0 tx=0).stop 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 -- 172.21.15.66:0/4209045073 shutdown_connections 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 --2- 172.21.15.66:0/4209045073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40440cc7f0 0x7f40440ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 --2- 172.21.15.66:0/4209045073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f404415e4d0 0x7f40440ceaf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 -- 172.21.15.66:0/4209045073 >> 172.21.15.66:0/4209045073 conn(0x7f4044071c90 msgr2=0x7f40440672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 -- 172.21.15.66:0/4209045073 shutdown_connections 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.262+0000 7f404c0c8640 1 -- 172.21.15.66:0/4209045073 wait complete. 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.263+0000 7f404c0c8640 1 Processor -- start 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 -- start start 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f404406c690 0x7f4044161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 0x7f40441498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:43.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404415e4d0 0x7f404415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4044162350 con 0x7f40440cc7f0 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40440cf890 con 0x7f404415e4d0 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404c0c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f404415e8b0 con 0x7f404406c690 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f4049663640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 0x7f40441498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f4049663640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 0x7f40441498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41354/0 (socket says 172.21.15.66:41354) 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f4049663640 1 -- 172.21.15.66:0/2589701226 learned_addr learned my addr 172.21.15.66:0/2589701226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f4049e64640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f404406c690 0x7f4044161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.264+0000 7f404a665640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404415e4d0 0x7f404415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f4049663640 1 -- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f404406c690 msgr2=0x7f4044161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f4049663640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f404406c690 0x7f4044161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f4049663640 1 -- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404415e4d0 msgr2=0x7f404415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f4049663640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404415e4d0 0x7f404415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f4049663640 1 -- 172.21.15.66:0/2589701226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4044155200 con 0x7f40440cc7f0 2024-03-30T13:09:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f4049663640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 0x7f40441498a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f403800c430 tx=0x7f403800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380074e0 con 0x7f40440cc7f0 2024-03-30T13:09:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40380047e0 con 0x7f40440cc7f0 2024-03-30T13:09:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038004bd0 con 0x7f40440cc7f0 2024-03-30T13:09:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044149de0 con 0x7f40440cc7f0 2024-03-30T13:09:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.265+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4044155a30 con 0x7f40440cc7f0 2024-03-30T13:09:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.266+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4044075900 con 0x7f40440cc7f0 2024-03-30T13:09:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.266+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4038011070 con 0x7f40440cc7f0 2024-03-30T13:09:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.267+0000 7f40427fc640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f403006c530 0x7f403006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.267+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f403808b190 con 0x7f40440cc7f0 2024-03-30T13:09:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.267+0000 7f4049e64640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f403006c530 0x7f403006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.267+0000 7f4049e64640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f403006c530 0x7f403006e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f403c036000 tx=0x7f403c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.271+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4038059160 con 0x7f40440cc7f0 2024-03-30T13:09:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:43.466+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f4044070150 con 0x7f40440cc7f0 2024-03-30T13:09:44.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.104+0000 7f40427fc640 1 -- 172.21.15.66:0/2589701226 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f403805d460 con 0x7f40440cc7f0 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f403006c530 msgr2=0x7f403006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f403006c530 0x7f403006e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f403c036000 tx=0x7f403c03d040 comp rx=0 tx=0).stop 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 msgr2=0x7f40441498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 0x7f40441498a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f403800c430 tx=0x7f403800c8f0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 shutdown_connections 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f403006c530 0x7f403006e9f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f404415e4d0 0x7f404415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40440cc7f0 0x7f40441498a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 --2- 172.21.15.66:0/2589701226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f404406c690 0x7f4044161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 >> 172.21.15.66:0/2589701226 conn(0x7f4044071c90 msgr2=0x7f40440ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 shutdown_connections 2024-03-30T13:09:44.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.107+0000 7f404c0c8640 1 -- 172.21.15.66:0/2589701226 wait complete. 2024-03-30T13:09:44.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T13:09:44.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.243+0000 7fc045d52640 1 Processor -- start 2024-03-30T13:09:44.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.243+0000 7fc045d52640 1 -- start start 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc045d52640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc0400cf2d0 0x7fc0400ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc045d52640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc0400ceb10 0x7fc0400cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc045d52640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0400cc5f0 con 0x7fc04015ef20 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc045d52640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc0400affa0 con 0x7fc0400cf2d0 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc045d52640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc0400672c0 con 0x7fc0400ceb10 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc03f7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc0400cf2d0 0x7fc0400ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc03f7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc0400cf2d0 0x7fc0400ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41362/0 (socket says 172.21.15.66:41362) 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.244+0000 7fc03f7fe640 1 -- 172.21.15.66:0/4148414006 learned_addr learned my addr 172.21.15.66:0/4148414006 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:44.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.245+0000 7fc03effd640 1 --2- 172.21.15.66:0/4148414006 >> v2:172.21.15.117:3300/0 conn(0x7fc0400ceb10 0x7fc0400cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.245+0000 7fc03f7fe640 1 -- 172.21.15.66:0/4148414006 >> v2:172.21.15.117:3300/0 conn(0x7fc0400ceb10 msgr2=0x7fc0400cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.245+0000 7fc03f7fe640 1 --2- 172.21.15.66:0/4148414006 >> v2:172.21.15.117:3300/0 conn(0x7fc0400ceb10 0x7fc0400cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.245+0000 7fc03f7fe640 1 -- 172.21.15.66:0/4148414006 >> v1:172.21.15.117:6789/0 conn(0x7fc04015ef20 legacy=0x7fc04015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.245+0000 7fc03f7fe640 1 -- 172.21.15.66:0/4148414006 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0400cfbf0 con 0x7fc0400cf2d0 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.245+0000 7fc03e7fc640 1 -- 172.21.15.66:0/4148414006 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239865280 0 0) 0x7fc0400cc5f0 con 0x7fc04015ef20 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc03f7fe640 1 --2- 172.21.15.66:0/4148414006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400ce3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc03000bb10 tx=0x7fc03002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf69965edc7ae5fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc03e7fc640 1 -- 172.21.15.66:0/4148414006 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03003b070 con 0x7fc0400cf2d0 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc03e7fc640 1 -- 172.21.15.66:0/4148414006 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030036440 con 0x7fc0400cf2d0 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc03e7fc640 1 -- 172.21.15.66:0/4148414006 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030036810 con 0x7fc0400cf2d0 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 -- 172.21.15.66:0/4148414006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 msgr2=0x7fc0400ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 --2- 172.21.15.66:0/4148414006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400ce3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc03000bb10 tx=0x7fc03002e170 comp rx=0 tx=0).stop 2024-03-30T13:09:44.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 -- 172.21.15.66:0/4148414006 shutdown_connections 2024-03-30T13:09:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 --2- 172.21.15.66:0/4148414006 >> v2:172.21.15.117:3300/0 conn(0x7fc0400ceb10 0x7fc0400cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 --2- 172.21.15.66:0/4148414006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400ce3a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 -- 172.21.15.66:0/4148414006 >> 172.21.15.66:0/4148414006 conn(0x7fc0400cd250 msgr2=0x7fc0400cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 -- 172.21.15.66:0/4148414006 shutdown_connections 2024-03-30T13:09:44.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.246+0000 7fc045d52640 1 -- 172.21.15.66:0/4148414006 wait complete. 2024-03-30T13:09:44.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.247+0000 7fc045d52640 1 Processor -- start 2024-03-30T13:09:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 -- start start 2024-03-30T13:09:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0400ceb10 0x7fc04015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:44.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04015ef20 0x7fc04015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400cc890 con 0x7fc0400cf2d0 2024-03-30T13:09:44.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400677f0 con 0x7fc0400ceb10 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc045d52640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc0400ccb70 con 0x7fc04015ef20 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc03effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc03effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41374/0 (socket says 172.21.15.66:41374) 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc03effd640 1 -- 172.21.15.66:0/1100227489 learned_addr learned my addr 172.21.15.66:0/1100227489 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.248+0000 7fc03ffff640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04015ef20 0x7fc04015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03f7fe640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0400ceb10 0x7fc04015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03effd640 1 -- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04015ef20 msgr2=0x7fc04015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03effd640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04015ef20 0x7fc04015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03effd640 1 -- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0400ceb10 msgr2=0x7fc04015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03effd640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0400ceb10 0x7fc04015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03effd640 1 -- 172.21.15.66:0/1100227489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0400c86a0 con 0x7fc0400cf2d0 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03ffff640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04015ef20 0x7fc04015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc03effd640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400aeb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc03400e9b0 tx=0x7fc03400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:44.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034017070 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03400c9a0 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03400cd70 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0400c8980 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.249+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0400c8f40 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.250+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc040075900 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.251+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc03401a070 con 0x7fc0400cf2d0 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.251+0000 7fc027fff640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc02c06c530 0x7fc02c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:44.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.251+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fc03408bbc0 con 0x7fc0400cf2d0 2024-03-30T13:09:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.251+0000 7fc03f7fe640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc02c06c530 0x7fc02c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.251+0000 7fc03f7fe640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc02c06c530 0x7fc02c06e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc030038000 tx=0x7fc030008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:44.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.255+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc034059950 con 0x7fc0400cf2d0 2024-03-30T13:09:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:44.439+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc04006ce10 con 0x7fc0400cf2d0 2024-03-30T13:09:46.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.120+0000 7fc027fff640 1 -- 172.21.15.66:0/1100227489 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fc03405dc50 con 0x7fc0400cf2d0 2024-03-30T13:09:46.155 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' created 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.124+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc02c06c530 msgr2=0x7fc02c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.124+0000 7fc045d52640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc02c06c530 0x7fc02c06e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc030038000 tx=0x7fc030008040 comp rx=0 tx=0).stop 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.124+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 msgr2=0x7fc0400aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.124+0000 7fc045d52640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400aeb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc03400e9b0 tx=0x7fc03400ee70 comp rx=0 tx=0).stop 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.125+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 shutdown_connections 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.125+0000 7fc045d52640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc02c06c530 0x7fc02c06e9f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.125+0000 7fc045d52640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04015ef20 0x7fc04015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.125+0000 7fc045d52640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0400cf2d0 0x7fc0400aeb50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.125+0000 7fc045d52640 1 --2- 172.21.15.66:0/1100227489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0400ceb10 0x7fc04015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.125+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 >> 172.21.15.66:0/1100227489 conn(0x7fc0400cd250 msgr2=0x7fc040159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:46.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.127+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 shutdown_connections 2024-03-30T13:09:46.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.127+0000 7fc045d52640 1 -- 172.21.15.66:0/1100227489 wait complete. 2024-03-30T13:09:46.158 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-30T13:09:46.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.272+0000 7fd54c8d3640 1 Processor -- start 2024-03-30T13:09:46.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.272+0000 7fd54c8d3640 1 -- start start 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54c8d3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5480b9c60 0x7fd5480b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54c8d3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5480ba0c0 0x7fd5480b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54c8d3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5480d2a80 con 0x7fd54815e430 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54c8d3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd5480d2d60 con 0x7fd5480b9c60 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54c8d3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd5480d3040 con 0x7fd5480ba0c0 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54659c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd54815e430 0x7fd5480d3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34136/0 (socket says 172.21.15.66:34136) 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd54659c640 1 -- 172.21.15.66:0/2885293565 learned_addr learned my addr 172.21.15.66:0/2885293565 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd53ffff640 1 --2- 172.21.15.66:0/2885293565 >> v2:172.21.15.117:3300/0 conn(0x7fd5480b9c60 0x7fd5480b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd546d9d640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5480ba0c0 0x7fd5480b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd545d9b640 1 -- 172.21.15.66:0/2885293565 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399850347 0 0) 0x7fd5480d2a80 con 0x7fd54815e430 2024-03-30T13:09:46.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd545d9b640 1 -- 172.21.15.66:0/2885293565 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd534003520 con 0x7fd54815e430 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd53ffff640 1 -- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5480ba0c0 msgr2=0x7fd5480b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd53ffff640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5480ba0c0 0x7fd5480b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd53ffff640 1 -- 172.21.15.66:0/2885293565 >> v1:172.21.15.66:6789/0 conn(0x7fd54815e430 legacy=0x7fd5480d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd53ffff640 1 -- 172.21.15.66:0/2885293565 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5480d3320 con 0x7fd5480b9c60 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.273+0000 7fd546d9d640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5480ba0c0 0x7fd5480b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.274+0000 7fd53ffff640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480b9c60 0x7fd5480b5dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5300097a0 tx=0x7fd53002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb6174dba937be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.274+0000 7fd545d9b640 1 -- 172.21.15.66:0/2885293565 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5300315a0 con 0x7fd5480b9c60 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.274+0000 7fd545d9b640 1 -- 172.21.15.66:0/2885293565 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd530037070 con 0x7fd5480b9c60 2024-03-30T13:09:46.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.274+0000 7fd545d9b640 1 -- 172.21.15.66:0/2885293565 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530031960 con 0x7fd5480b9c60 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 -- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480b9c60 msgr2=0x7fd5480b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480b9c60 0x7fd5480b5dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5300097a0 tx=0x7fd53002b840 comp rx=0 tx=0).stop 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 -- 172.21.15.66:0/2885293565 shutdown_connections 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5480ba0c0 0x7fd5480b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/2885293565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480b9c60 0x7fd5480b5dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 -- 172.21.15.66:0/2885293565 >> 172.21.15.66:0/2885293565 conn(0x7fd5480678f0 msgr2=0x7fd5480d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 -- 172.21.15.66:0/2885293565 shutdown_connections 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.275+0000 7fd54c8d3640 1 -- 172.21.15.66:0/2885293565 wait complete. 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.276+0000 7fd54c8d3640 1 Processor -- start 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.276+0000 7fd54c8d3640 1 -- start start 2024-03-30T13:09:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54c8d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 0x7fd548153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54c8d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480ba0c0 0x7fd548157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54c8d3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd54815e430 0x7fd548157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54c8d3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd54815e990 con 0x7fd5480b9c60 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54c8d3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd54815e810 con 0x7fd5480ba0c0 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54c8d3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd5480d26a0 con 0x7fd54815e430 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 0x7fd548153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 0x7fd548153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41378/0 (socket says 172.21.15.66:41378) 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 -- 172.21.15.66:0/3167540989 learned_addr learned my addr 172.21.15.66:0/3167540989 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd53ffff640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480ba0c0 0x7fd548157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd546d9d640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd54815e430 0x7fd548157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 -- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd54815e430 msgr2=0x7fd548157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd54815e430 0x7fd548157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 -- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480ba0c0 msgr2=0x7fd548157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480ba0c0 0x7fd548157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 -- 172.21.15.66:0/3167540989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5480c9500 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.277+0000 7fd54659c640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 0x7fd548153860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd53800c390 tx=0x7fd53800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.278+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538017070 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.278+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd538005ce0 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.278+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5380072f0 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.278+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5480c97e0 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.278+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54815f8c0 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.279+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd548075900 con 0x7fd5480b9c60 2024-03-30T13:09:46.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.279+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd53801a070 con 0x7fd5480b9c60 2024-03-30T13:09:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.280+0000 7fd53effd640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd52806c530 0x7fd52806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.280+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fd53808afc0 con 0x7fd5480b9c60 2024-03-30T13:09:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.280+0000 7fd53ffff640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd52806c530 0x7fd52806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:46.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.280+0000 7fd53ffff640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd52806c530 0x7fd52806e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd530036000 tx=0x7fd53003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:46.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.284+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd538021020 con 0x7fd5480b9c60 2024-03-30T13:09:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:46.467+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd5480b7430 con 0x7fd5480b9c60 2024-03-30T13:09:47.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.126+0000 7fd53effd640 1 -- 172.21.15.66:0/3167540989 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7fd538058de0 con 0x7fd5480b9c60 2024-03-30T13:09:47.130 INFO:teuthology.orchestra.run.smithi066.stderr:set pool 4 allow_ec_overwrites to true 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd52806c530 msgr2=0x7fd52806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd52806c530 0x7fd52806e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd530036000 tx=0x7fd53003d040 comp rx=0 tx=0).stop 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 msgr2=0x7fd548153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 0x7fd548153860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd53800c390 tx=0x7fd53800c790 comp rx=0 tx=0).stop 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 shutdown_connections 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd52806c530 0x7fd52806e9f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd54815e430 0x7fd548157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5480ba0c0 0x7fd548157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 --2- 172.21.15.66:0/3167540989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5480b9c60 0x7fd548153860 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.128+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 >> 172.21.15.66:0/3167540989 conn(0x7fd5480678f0 msgr2=0x7fd54806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:47.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.129+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 shutdown_connections 2024-03-30T13:09:47.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.129+0000 7fd54c8d3640 1 -- 172.21.15.66:0/3167540989 wait complete. 2024-03-30T13:09:47.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-30T13:09:47.225 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-03-30T13:09:47.222+0000 7f2d77caa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T13:09:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.264+0000 7faa12f1c640 1 Processor -- start 2024-03-30T13:09:47.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.264+0000 7faa12f1c640 1 -- start start 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa12f1c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa0c15e7c0 0x7faa0c06c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa12f1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa12f1c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faa0c0b4800 0x7faa0c06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa12f1c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0cc840 con 0x7faa0c0b4800 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa12f1c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c0d1170 con 0x7faa0c15e7c0 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa12f1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c0d1450 con 0x7faa0c15eba0 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa114b9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faa0c0b4800 0x7faa0c06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa114b9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faa0c0b4800 0x7faa0c06d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41394/0 (socket says 172.21.15.66:41394) 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa114b9640 1 -- 172.21.15.66:0/2564806210 learned_addr learned my addr 172.21.15.66:0/2564806210 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:47.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa10cb8640 1 --2- 172.21.15.66:0/2564806210 >> v2:172.21.15.117:3300/0 conn(0x7faa0c15e7c0 0x7faa0c06c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.265+0000 7faa0bfff640 1 --2- 172.21.15.66:0/2564806210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa114b9640 1 -- 172.21.15.66:0/2564806210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 msgr2=0x7faa0c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa114b9640 1 --2- 172.21.15.66:0/2564806210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa114b9640 1 -- 172.21.15.66:0/2564806210 >> v2:172.21.15.117:3300/0 conn(0x7faa0c15e7c0 msgr2=0x7faa0c06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa114b9640 1 --2- 172.21.15.66:0/2564806210 >> v2:172.21.15.117:3300/0 conn(0x7faa0c15e7c0 0x7faa0c06c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa114b9640 1 -- 172.21.15.66:0/2564806210 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0d1730 con 0x7faa0c0b4800 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa114b9640 1 --2- 172.21.15.66:0/2564806210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c0b4800 0x7faa0c06d2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa0c09bc30 tx=0x7fa9f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cb4ea974073da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa0b7fe640 1 -- 172.21.15.66:0/2564806210 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4035020 con 0x7faa0c0b4800 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa0b7fe640 1 -- 172.21.15.66:0/2564806210 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4031870 con 0x7faa0c0b4800 2024-03-30T13:09:47.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.266+0000 7faa0b7fe640 1 -- 172.21.15.66:0/2564806210 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4031c40 con 0x7faa0c0b4800 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 -- 172.21.15.66:0/2564806210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c0b4800 msgr2=0x7faa0c06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 --2- 172.21.15.66:0/2564806210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c0b4800 0x7faa0c06d2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa0c09bc30 tx=0x7fa9f402b910 comp rx=0 tx=0).stop 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 -- 172.21.15.66:0/2564806210 shutdown_connections 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 --2- 172.21.15.66:0/2564806210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c0b4800 0x7faa0c06d2e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 --2- 172.21.15.66:0/2564806210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 --2- 172.21.15.66:0/2564806210 >> v2:172.21.15.117:3300/0 conn(0x7faa0c15e7c0 0x7faa0c06c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 -- 172.21.15.66:0/2564806210 >> 172.21.15.66:0/2564806210 conn(0x7faa0c0b3ce0 msgr2=0x7faa0c0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 -- 172.21.15.66:0/2564806210 shutdown_connections 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.267+0000 7faa12f1c640 1 -- 172.21.15.66:0/2564806210 wait complete. 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.268+0000 7faa12f1c640 1 Processor -- start 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.268+0000 7faa12f1c640 1 -- start start 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa12f1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa0c0b4800 0x7faa0c1f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa12f1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 0x7faa0c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa12f1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa12f1c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c09abb0 con 0x7faa0c15e7c0 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa12f1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c09aa30 con 0x7faa0c0b4800 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa12f1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c09ad30 con 0x7faa0c15eba0 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 0x7faa0c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 0x7faa0c0c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41404/0 (socket says 172.21.15.66:41404) 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa10cb8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa0c0b4800 0x7faa0c1f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa0bfff640 1 -- 172.21.15.66:0/1723726121 learned_addr learned my addr 172.21.15.66:0/1723726121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa114b9640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa10cb8640 1 -- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 msgr2=0x7faa0c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa10cb8640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa10cb8640 1 -- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 msgr2=0x7faa0c0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.269+0000 7faa10cb8640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 0x7faa0c0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa10cb8640 1 -- 172.21.15.66:0/1723726121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c1533a0 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa0bfff640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 0x7faa0c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa10cb8640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa0c0b4800 0x7faa0c1f1380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa0400e870 tx=0x7faa0400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04017070 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0400c940 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0400ccf0 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c0c85a0 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.270+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c0c8b60 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.271+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c075890 con 0x7faa0c0b4800 2024-03-30T13:09:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.271+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faa0401a070 con 0x7faa0c0b4800 2024-03-30T13:09:47.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.272+0000 7faa097fa640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa9ec06c560 0x7fa9ec06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:47.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.272+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7faa0408bc20 con 0x7faa0c0b4800 2024-03-30T13:09:47.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.272+0000 7faa0bfff640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa9ec06c560 0x7fa9ec06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:47.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.272+0000 7faa0bfff640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa9ec06c560 0x7fa9ec06ea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0c9810 tx=0x7fa9fc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:47.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.277+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faa04059840 con 0x7faa0c0b4800 2024-03-30T13:09:47.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:47.503+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7faa0c0b4310 con 0x7faa0c0b4800 2024-03-30T13:09:48.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.138+0000 7faa097fa640 1 -- 172.21.15.66:0/1723726121 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7faa0408b0b0 con 0x7faa0c0b4800 2024-03-30T13:09:48.142 INFO:teuthology.orchestra.run.smithi066.stderr:added data pool 4 to fsmap 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa9ec06c560 msgr2=0x7fa9ec06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa9ec06c560 0x7fa9ec06ea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0c9810 tx=0x7fa9fc00b040 comp rx=0 tx=0).stop 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa0c0b4800 msgr2=0x7faa0c1f1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa0c0b4800 0x7faa0c1f1380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa0400e870 tx=0x7faa0400ed30 comp rx=0 tx=0).stop 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 shutdown_connections 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa9ec06c560 0x7fa9ec06ea20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa0c15eba0 0x7faa0c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa0c15e7c0 0x7faa0c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 --2- 172.21.15.66:0/1723726121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa0c0b4800 0x7faa0c1f1380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 >> 172.21.15.66:0/1723726121 conn(0x7faa0c0b3ce0 msgr2=0x7faa0c06ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:48.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 shutdown_connections 2024-03-30T13:09:48.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.141+0000 7faa12f1c640 1 -- 172.21.15.66:0/1723726121 wait complete. 2024-03-30T13:09:48.157 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:09:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.270+0000 7f5128670640 1 Processor -- start 2024-03-30T13:09:48.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.270+0000 7f5128670640 1 -- start start 2024-03-30T13:09:48.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.270+0000 7f5128670640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51200d1390 0x7f51200d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f5128670640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51200b39d0 0x7f51200b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f5128670640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51200ba3d0 con 0x7f51200b3570 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f5128670640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f51200d1cb0 con 0x7f51200d1390 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f5128670640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f51200cf6e0 con 0x7f51200b39d0 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51200d1390 0x7f51200d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51200d1390 0x7f51200d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41412/0 (socket says 172.21.15.66:41412) 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 -- 172.21.15.66:0/2482402284 learned_addr learned my addr 172.21.15.66:0/2482402284 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f5126c0d640 1 --2- 172.21.15.66:0/2482402284 >> v2:172.21.15.117:3300/0 conn(0x7f51200b39d0 0x7f51200b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 -- 172.21.15.66:0/2482402284 >> v2:172.21.15.117:3300/0 conn(0x7f51200b39d0 msgr2=0x7f51200b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 --2- 172.21.15.66:0/2482402284 >> v2:172.21.15.117:3300/0 conn(0x7f51200b39d0 0x7f51200b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 -- 172.21.15.66:0/2482402284 >> v1:172.21.15.117:6789/0 conn(0x7f51200b3570 legacy=0x7f51200b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512540a640 1 -- 172.21.15.66:0/2482402284 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360808015 0 0) 0x7f51200ba3d0 con 0x7f51200b3570 2024-03-30T13:09:48.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.271+0000 7f512640c640 1 -- 172.21.15.66:0/2482402284 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51200b9050 con 0x7f51200d1390 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.272+0000 7f512640c640 1 --2- 172.21.15.66:0/2482402284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200d1390 0x7f51200d1770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51200b5e80 tx=0x7f511802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61aba1ec0f7a3ea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.272+0000 7f512540a640 1 -- 172.21.15.66:0/2482402284 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511802f850 con 0x7f51200d1390 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.272+0000 7f512540a640 1 -- 172.21.15.66:0/2482402284 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511802f9f0 con 0x7f51200d1390 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.272+0000 7f512540a640 1 -- 172.21.15.66:0/2482402284 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511802fdc0 con 0x7f51200d1390 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.272+0000 7f5128670640 1 -- 172.21.15.66:0/2482402284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200d1390 msgr2=0x7f51200d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.272+0000 7f5128670640 1 --2- 172.21.15.66:0/2482402284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200d1390 0x7f51200d1770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51200b5e80 tx=0x7f511802ba30 comp rx=0 tx=0).stop 2024-03-30T13:09:48.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.273+0000 7f5128670640 1 -- 172.21.15.66:0/2482402284 shutdown_connections 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.273+0000 7f5128670640 1 --2- 172.21.15.66:0/2482402284 >> v2:172.21.15.117:3300/0 conn(0x7f51200b39d0 0x7f51200b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.273+0000 7f5128670640 1 --2- 172.21.15.66:0/2482402284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200d1390 0x7f51200d1770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.273+0000 7f5128670640 1 -- 172.21.15.66:0/2482402284 >> 172.21.15.66:0/2482402284 conn(0x7f51200676e0 msgr2=0x7f5120067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.273+0000 7f5128670640 1 -- 172.21.15.66:0/2482402284 shutdown_connections 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.273+0000 7f5128670640 1 -- 172.21.15.66:0/2482402284 wait complete. 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 Processor -- start 2024-03-30T13:09:48.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 -- start start 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 0x7f51200cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51200b39d0 0x7f5120149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51200d1390 0x7f512014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51200b98f0 con 0x7f51200b3570 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51200b9610 con 0x7f51200d1390 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.274+0000 7f5128670640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f51200b9bd0 con 0x7f51200b39d0 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 0x7f51200cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 0x7f51200cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41426/0 (socket says 172.21.15.66:41426) 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 -- 172.21.15.66:0/2283748274 learned_addr learned my addr 172.21.15.66:0/2283748274 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f5126c0d640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51200d1390 0x7f512014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f5125c0b640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51200b39d0 0x7f5120149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 -- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51200b39d0 msgr2=0x7f5120149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51200b39d0 0x7f5120149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 -- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51200d1390 msgr2=0x7f512014a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51200d1390 0x7f512014a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 -- 172.21.15.66:0/2283748274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51200cd600 con 0x7f51200b3570 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f512640c640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 0x7f51200cbbc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f5118004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118047070 con 0x7f51200b3570 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5118037d30 con 0x7f51200b3570 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51200cc560 con 0x7f51200b3570 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180426c0 con 0x7f51200b3570 2024-03-30T13:09:48.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.275+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51200ccb20 con 0x7f51200b3570 2024-03-30T13:09:48.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.276+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5120075a10 con 0x7f51200b3570 2024-03-30T13:09:48.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.277+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f511804a070 con 0x7f51200b3570 2024-03-30T13:09:48.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.277+0000 7f511effd640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f510c06c530 0x7f510c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.277+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f51180adc20 con 0x7f51200b3570 2024-03-30T13:09:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.277+0000 7f5125c0b640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f510c06c530 0x7f510c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.278+0000 7f5125c0b640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f510c06c530 0x7f510c06e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51200c9a50 tx=0x7f5114009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.282+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5118033030 con 0x7f51200b3570 2024-03-30T13:09:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.506+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f512015dfc0 con 0x7f51200b3570 2024-03-30T13:09:48.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.507+0000 7f511effd640 1 -- 172.21.15.66:0/2283748274 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5183 (secure 0 0 0) 0x7f511807b9d0 con 0x7f51200b3570 2024-03-30T13:09:48.511 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:48.511 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:09:47.505254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:09:48.511 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 6 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f510c06c530 msgr2=0x7f510c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f510c06c530 0x7f510c06e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51200c9a50 tx=0x7f5114009040 comp rx=0 tx=0).stop 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 msgr2=0x7f51200cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 0x7f51200cbbc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f5118004730 comp rx=0 tx=0).stop 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 shutdown_connections 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f510c06c530 0x7f510c06e9f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51200d1390 0x7f512014a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51200b39d0 0x7f5120149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 --2- 172.21.15.66:0/2283748274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51200b3570 0x7f51200cbbc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 >> 172.21.15.66:0/2283748274 conn(0x7f51200676e0 msgr2=0x7f51200d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 shutdown_connections 2024-03-30T13:09:48.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.509+0000 7f5128670640 1 -- 172.21.15.66:0/2283748274 wait complete. 2024-03-30T13:09:48.524 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:48.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 Processor -- start 2024-03-30T13:09:48.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 -- start start 2024-03-30T13:09:48.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f89340b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89340cedb0 0x7f89340cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89340cf970 con 0x7f89340b63e0 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f89340b3c80 con 0x7f89340cedb0 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.627+0000 7f8938f96640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f89340d2560 con 0x7f893415e9c0 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f8932d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f89340b63e0 0x7f89340d2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34152/0 (socket says 172.21.15.66:34152) 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f8932d9d640 1 -- 172.21.15.66:0/2890476290 learned_addr learned my addr 172.21.15.66:0/2890476290 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f892bfff640 1 --2- 172.21.15.66:0/2890476290 >> v2:172.21.15.117:3300/0 conn(0x7f89340cedb0 0x7f89340cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f893259c640 1 --2- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f89340b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f892bfff640 1 -- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 msgr2=0x7f89340b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f892bfff640 1 --2- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f89340b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f8931d9b640 1 -- 172.21.15.66:0/2890476290 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757617668 0 0) 0x7f89340cf970 con 0x7f89340b63e0 2024-03-30T13:09:48.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f892bfff640 1 -- 172.21.15.66:0/2890476290 >> v1:172.21.15.66:6789/0 conn(0x7f89340b63e0 legacy=0x7f89340d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.628+0000 7f892bfff640 1 -- 172.21.15.66:0/2890476290 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89340cfc50 con 0x7f89340cedb0 2024-03-30T13:09:48.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f892bfff640 1 --2- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340cf1f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f891c0097a0 tx=0x7f891c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0e9988b2cb472d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8931d9b640 1 -- 172.21.15.66:0/2890476290 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c035020 con 0x7f89340cedb0 2024-03-30T13:09:48.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8931d9b640 1 -- 172.21.15.66:0/2890476290 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c031830 con 0x7f89340cedb0 2024-03-30T13:09:48.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8931d9b640 1 -- 172.21.15.66:0/2890476290 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c031c00 con 0x7f89340cedb0 2024-03-30T13:09:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 -- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 msgr2=0x7f89340cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 --2- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340cf1f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f891c0097a0 tx=0x7f891c02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 -- 172.21.15.66:0/2890476290 shutdown_connections 2024-03-30T13:09:48.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 --2- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340cf1f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 --2- 172.21.15.66:0/2890476290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f89340b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 -- 172.21.15.66:0/2890476290 >> 172.21.15.66:0/2890476290 conn(0x7f89340676e0 msgr2=0x7f8934067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 -- 172.21.15.66:0/2890476290 shutdown_connections 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.629+0000 7f8938f96640 1 -- 172.21.15.66:0/2890476290 wait complete. 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 Processor -- start 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 -- start start 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 0x7f8934160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f8934159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89340d2a80 con 0x7f89340b63e0 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89340cf730 con 0x7f89340cedb0 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f8938f96640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f89340b7d80 con 0x7f893415e9c0 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f893259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 0x7f8934160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f893259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 0x7f8934160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41430/0 (socket says 172.21.15.66:41430) 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.631+0000 7f893259c640 1 -- 172.21.15.66:0/3917181531 learned_addr learned my addr 172.21.15.66:0/3917181531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f892bfff640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f8932d9d640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f8934159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f893259c640 1 -- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 msgr2=0x7f8934159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f893259c640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f8934159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f893259c640 1 -- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 msgr2=0x7f89340ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f893259c640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f893259c640 1 -- 172.21.15.66:0/3917181531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8934153bd0 con 0x7f89340b63e0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f893259c640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 0x7f8934160860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f892400e870 tx=0x7f892400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924017070 con 0x7f89340b63e0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892400c940 con 0x7f89340b63e0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400cd30 con 0x7f89340b63e0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8934160da0 con 0x7f89340b63e0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.632+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89340c9fe0 con 0x7f89340b63e0 2024-03-30T13:09:48.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.633+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8934075a10 con 0x7f89340b63e0 2024-03-30T13:09:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.634+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f892401a070 con 0x7f89340b63e0 2024-03-30T13:09:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.634+0000 7f892affd640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f891406c530 0x7f891406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.634+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f892408b830 con 0x7f89340b63e0 2024-03-30T13:09:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.634+0000 7f892bfff640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f891406c530 0x7f891406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.634+0000 7f892bfff640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f891406c530 0x7f891406e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f891c036000 tx=0x7f891c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.638+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8934160da0 con 0x7f89340b63e0 2024-03-30T13:09:48.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.825+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f893415e140 con 0x7f89340b63e0 2024-03-30T13:09:48.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.826+0000 7f892affd640 1 -- 172.21.15.66:0/3917181531 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18515 (secure 0 0 0) 0x7f8934153bd0 con 0x7f89340b63e0 2024-03-30T13:09:48.830 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:48.830 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:47.505046+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f891406c530 msgr2=0x7f891406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f891406c530 0x7f891406e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f891c036000 tx=0x7f891c03d040 comp rx=0 tx=0).stop 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 msgr2=0x7f8934160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 0x7f8934160860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f892400e870 tx=0x7f892400ed30 comp rx=0 tx=0).stop 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 shutdown_connections 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f891406c530 0x7f891406e9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f893415e9c0 0x7f8934159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89340cedb0 0x7f89340ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 --2- 172.21.15.66:0/3917181531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89340b63e0 0x7f8934160860 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 >> 172.21.15.66:0/3917181531 conn(0x7f89340676e0 msgr2=0x7f89340b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 shutdown_connections 2024-03-30T13:09:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.828+0000 7f8938f96640 1 -- 172.21.15.66:0/3917181531 wait complete. 2024-03-30T13:09:48.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.947+0000 7ff13a24a640 1 Processor -- start 2024-03-30T13:09:48.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.947+0000 7ff13a24a640 1 -- start start 2024-03-30T13:09:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff13a24a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff1340d1390 0x7ff1340d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff13a24a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340b39d0 0x7ff1340b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff13a24a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1340ba3d0 con 0x7ff1340b3570 2024-03-30T13:09:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff13a24a640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff1340d1cb0 con 0x7ff1340d1390 2024-03-30T13:09:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff132ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff1340b3570 0x7ff1340b7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34154/0 (socket says 172.21.15.66:34154) 2024-03-30T13:09:48.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff13a24a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340cf6e0 con 0x7ff1340b39d0 2024-03-30T13:09:48.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff132ffd640 1 -- 172.21.15.66:0/1825738012 learned_addr learned my addr 172.21.15.66:0/1825738012 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:48.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1337fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff1340d1390 0x7ff1340d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff133fff640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340b39d0 0x7ff1340b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1327fc640 1 -- 172.21.15.66:0/1825738012 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924077498 0 0) 0x7ff1340ba3d0 con 0x7ff1340b3570 2024-03-30T13:09:48.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1327fc640 1 -- 172.21.15.66:0/1825738012 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff11c003520 con 0x7ff1340b3570 2024-03-30T13:09:48.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1337fe640 1 -- 172.21.15.66:0/1825738012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340b39d0 msgr2=0x7ff1340b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:48.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1337fe640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340b39d0 0x7ff1340b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1337fe640 1 -- 172.21.15.66:0/1825738012 >> v1:172.21.15.66:6789/0 conn(0x7ff1340b3570 legacy=0x7ff1340b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff1337fe640 1 -- 172.21.15.66:0/1825738012 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340b9050 con 0x7ff1340d1390 2024-03-30T13:09:48.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.948+0000 7ff133fff640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340b39d0 0x7ff1340b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:48.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.949+0000 7ff1337fe640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340d1390 0x7ff1340d1770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1240097a0 tx=0x7ff12402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5943d1d381fb43cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.949+0000 7ff1327fc640 1 -- 172.21.15.66:0/1825738012 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124034070 con 0x7ff1340d1390 2024-03-30T13:09:48.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.949+0000 7ff1327fc640 1 -- 172.21.15.66:0/1825738012 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff124030800 con 0x7ff1340d1390 2024-03-30T13:09:48.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.949+0000 7ff1327fc640 1 -- 172.21.15.66:0/1825738012 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124030bd0 con 0x7ff1340d1390 2024-03-30T13:09:48.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 -- 172.21.15.66:0/1825738012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340d1390 msgr2=0x7ff1340d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340d1390 0x7ff1340d1770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1240097a0 tx=0x7ff12402b940 comp rx=0 tx=0).stop 2024-03-30T13:09:48.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 -- 172.21.15.66:0/1825738012 shutdown_connections 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340b39d0 0x7ff1340b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 --2- 172.21.15.66:0/1825738012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340d1390 0x7ff1340d1770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 -- 172.21.15.66:0/1825738012 >> 172.21.15.66:0/1825738012 conn(0x7ff1340676e0 msgr2=0x7ff134067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 -- 172.21.15.66:0/1825738012 shutdown_connections 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.950+0000 7ff13a24a640 1 -- 172.21.15.66:0/1825738012 wait complete. 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.951+0000 7ff13a24a640 1 Processor -- start 2024-03-30T13:09:48.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.951+0000 7ff13a24a640 1 -- start start 2024-03-30T13:09:48.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff13a24a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1340b3570 0x7ff1341523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff13a24a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 0x7ff1340cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff13a24a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340d1390 0x7ff1340cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff13a24a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b98f0 con 0x7ff1340b39d0 2024-03-30T13:09:48.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff13a24a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b9610 con 0x7ff1340b3570 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff13a24a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340b9bd0 con 0x7ff1340d1390 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 0x7ff1340cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 0x7ff1340cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41450/0 (socket says 172.21.15.66:41450) 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 -- 172.21.15.66:0/525975051 learned_addr learned my addr 172.21.15.66:0/525975051 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff133fff640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340d1390 0x7ff1340cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff1337fe640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1340b3570 0x7ff1341523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 -- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340d1390 msgr2=0x7ff1340cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340d1390 0x7ff1340cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 -- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1340b3570 msgr2=0x7ff1341523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:48.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1340b3570 0x7ff1341523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 -- 172.21.15.66:0/525975051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff134152eb0 con 0x7ff1340b39d0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.952+0000 7ff132ffd640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 0x7ff1340cc950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c460 tx=0x7ff12800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.953+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128017070 con 0x7ff1340b39d0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.953+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff128005ce0 con 0x7ff1340b39d0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.953+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280072f0 con 0x7ff1340b39d0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.953+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1340cb8b0 con 0x7ff1340b39d0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.953+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1340cbe70 con 0x7ff1340b39d0 2024-03-30T13:09:48.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.954+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff134075a10 con 0x7ff1340b39d0 2024-03-30T13:09:48.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.954+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff12801a070 con 0x7ff1340b39d0 2024-03-30T13:09:48.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.955+0000 7ff11bfff640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff12006c560 0x7ff12006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:48.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.955+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff12808adf0 con 0x7ff1340b39d0 2024-03-30T13:09:48.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.955+0000 7ff1337fe640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff12006c560 0x7ff12006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:48.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.955+0000 7ff1337fe640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff12006c560 0x7ff12006ea20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff124036000 tx=0x7ff12403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:48.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:48.959+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff128058970 con 0x7ff1340b39d0 2024-03-30T13:09:49.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.146+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1340b4650 con 0x7ff1340b39d0 2024-03-30T13:09:49.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.147+0000 7ff11bfff640 1 -- 172.21.15.66:0/525975051 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7ff12805cc70 con 0x7ff1340b39d0 2024-03-30T13:09:49.151 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:49.151 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:49.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff12006c560 msgr2=0x7ff12006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff12006c560 0x7ff12006ea20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff124036000 tx=0x7ff12403b040 comp rx=0 tx=0).stop 2024-03-30T13:09:49.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 msgr2=0x7ff1340cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 0x7ff1340cc950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c460 tx=0x7ff12800c860 comp rx=0 tx=0).stop 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 shutdown_connections 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff12006c560 0x7ff12006ea20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1340d1390 0x7ff1340cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1340b39d0 0x7ff1340cc950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 --2- 172.21.15.66:0/525975051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1340b3570 0x7ff1341523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 >> 172.21.15.66:0/525975051 conn(0x7ff1340676e0 msgr2=0x7ff1340d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 shutdown_connections 2024-03-30T13:09:49.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.150+0000 7ff13a24a640 1 -- 172.21.15.66:0/525975051 wait complete. 2024-03-30T13:09:49.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:09:49.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.288+0000 7fd124fb5640 1 Processor -- start 2024-03-30T13:09:49.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.288+0000 7fd124fb5640 1 -- start start 2024-03-30T13:09:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd124fb5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1200d1390 0x7fd1200d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd124fb5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1200b3570 0x7fd1200b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd124fb5640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1200ba3d0 con 0x7fd1200b39d0 2024-03-30T13:09:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd124fb5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd1200d1cb0 con 0x7fd1200d1390 2024-03-30T13:09:49.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd124fb5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd1200cf6e0 con 0x7fd1200b3570 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11e59c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1200d1390 0x7fd1200d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11ed9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd1200b39d0 0x7fd1200b7a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34170/0 (socket says 172.21.15.66:34170) 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11ed9d640 1 -- 172.21.15.66:0/3133816091 learned_addr learned my addr 172.21.15.66:0/3133816091 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd117fff640 1 --2- 172.21.15.66:0/3133816091 >> v2:172.21.15.117:3300/0 conn(0x7fd1200b3570 0x7fd1200b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11e59c640 1 -- 172.21.15.66:0/3133816091 >> v2:172.21.15.117:3300/0 conn(0x7fd1200b3570 msgr2=0x7fd1200b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11e59c640 1 --2- 172.21.15.66:0/3133816091 >> v2:172.21.15.117:3300/0 conn(0x7fd1200b3570 0x7fd1200b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11e59c640 1 -- 172.21.15.66:0/3133816091 >> v1:172.21.15.66:6789/0 conn(0x7fd1200b39d0 legacy=0x7fd1200b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11e59c640 1 -- 172.21.15.66:0/3133816091 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200b9050 con 0x7fd1200d1390 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.289+0000 7fd11dd9b640 1 -- 172.21.15.66:0/3133816091 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158451785 0 0) 0x7fd1200ba3d0 con 0x7fd1200b39d0 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.290+0000 7fd11e59c640 1 --2- 172.21.15.66:0/3133816091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200d1770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1200b63b0 tx=0x7fd11002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf22b81a8441da0e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.290+0000 7fd11dd9b640 1 -- 172.21.15.66:0/3133816091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110034070 con 0x7fd1200d1390 2024-03-30T13:09:49.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.290+0000 7fd11dd9b640 1 -- 172.21.15.66:0/3133816091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd110030610 con 0x7fd1200d1390 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.290+0000 7fd11dd9b640 1 -- 172.21.15.66:0/3133816091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1100309e0 con 0x7fd1200d1390 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 -- 172.21.15.66:0/3133816091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 msgr2=0x7fd1200d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 --2- 172.21.15.66:0/3133816091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200d1770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1200b63b0 tx=0x7fd11002b960 comp rx=0 tx=0).stop 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 -- 172.21.15.66:0/3133816091 shutdown_connections 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 --2- 172.21.15.66:0/3133816091 >> v2:172.21.15.117:3300/0 conn(0x7fd1200b3570 0x7fd1200b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 --2- 172.21.15.66:0/3133816091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200d1770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 -- 172.21.15.66:0/3133816091 >> 172.21.15.66:0/3133816091 conn(0x7fd1200676e0 msgr2=0x7fd120067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 -- 172.21.15.66:0/3133816091 shutdown_connections 2024-03-30T13:09:49.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.291+0000 7fd124fb5640 1 -- 172.21.15.66:0/3133816091 wait complete. 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.292+0000 7fd124fb5640 1 Processor -- start 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.292+0000 7fd124fb5640 1 -- start start 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd124fb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200b3570 0x7fd1200cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd124fb5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b39d0 0x7fd12015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd124fb5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd124fb5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200b98f0 con 0x7fd1200d1390 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd124fb5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200b9610 con 0x7fd1200b3570 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd124fb5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd1200b9bd0 con 0x7fd1200b39d0 2024-03-30T13:09:49.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200b3570 0x7fd1200cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200b3570 0x7fd1200cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:38636/0 (socket says 172.21.15.66:38636) 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41474/0 (socket says 172.21.15.66:41474) 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11e59c640 1 -- 172.21.15.66:0/1730774624 learned_addr learned my addr 172.21.15.66:0/1730774624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd117fff640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b39d0 0x7fd12015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 -- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b39d0 msgr2=0x7fd12015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b39d0 0x7fd12015aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 -- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200b3570 msgr2=0x7fd1200cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200b3570 0x7fd1200cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 -- 172.21.15.66:0/1730774624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200cce90 con 0x7fd1200d1390 2024-03-30T13:09:49.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.293+0000 7fd11ed9d640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200cc640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd11800c700 tx=0x7fd11800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.294+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118018070 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.294+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11800ce10 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.294+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11800a880 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.294+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1200cd170 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.294+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1200cd730 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.295+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd120075a10 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.295+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd11801b070 con 0x7fd1200d1390 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.295+0000 7fd116ffd640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10006c530 0x7fd10006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.296+0000 7fd11e59c640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10006c530 0x7fd10006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.296+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd11808ae10 con 0x7fd1200d1390 2024-03-30T13:09:49.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.296+0000 7fd11e59c640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10006c530 0x7fd10006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd110009560 tx=0x7fd11003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:49.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.300+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd118058b30 con 0x7fd1200d1390 2024-03-30T13:09:49.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.523+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1200d1810 con 0x7fd1200d1390 2024-03-30T13:09:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.523+0000 7fd116ffd640 1 -- 172.21.15.66:0/1730774624 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5183 (secure 0 0 0) 0x7fd11805ce30 con 0x7fd1200d1390 2024-03-30T13:09:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:49.527 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:09:47.505254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:09:49.527 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 6 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10006c530 msgr2=0x7fd10006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10006c530 0x7fd10006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd110009560 tx=0x7fd11003b040 comp rx=0 tx=0).stop 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 msgr2=0x7fd1200cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200cc640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd11800c700 tx=0x7fd11800cbc0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 shutdown_connections 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10006c530 0x7fd10006e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200d1390 0x7fd1200cc640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b39d0 0x7fd12015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 --2- 172.21.15.66:0/1730774624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200b3570 0x7fd1200cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 >> 172.21.15.66:0/1730774624 conn(0x7fd1200676e0 msgr2=0x7fd1200d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:49.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 shutdown_connections 2024-03-30T13:09:49.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.525+0000 7fd124fb5640 1 -- 172.21.15.66:0/1730774624 wait complete. 2024-03-30T13:09:49.540 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:49.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.644+0000 7fb344d1c640 1 Processor -- start 2024-03-30T13:09:49.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.644+0000 7fb344d1c640 1 -- start start 2024-03-30T13:09:49.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb344d1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb344d1c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3400b7530 0x7fb3400d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb344d1c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3400b9d20 con 0x7fb3400b7990 2024-03-30T13:09:49.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb344d1c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb3400ba000 con 0x7fb3400b7530 2024-03-30T13:09:49.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb344d1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb3400ba2e0 con 0x7fb34015e4f0 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3400b7530 0x7fb3400d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3400b7530 0x7fb3400d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41486/0 (socket says 172.21.15.66:41486) 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 -- 172.21.15.66:0/1043657892 learned_addr learned my addr 172.21.15.66:0/1043657892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb33e59c640 1 --2- 172.21.15.66:0/1043657892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb33dd9b640 1 -- 172.21.15.66:0/1043657892 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3057540879 0 0) 0x7fb3400b9d20 con 0x7fb3400b7990 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 -- 172.21.15.66:0/1043657892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 msgr2=0x7fb3400cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 --2- 172.21.15.66:0/1043657892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 -- 172.21.15.66:0/1043657892 >> v1:172.21.15.117:6789/0 conn(0x7fb3400b7990 legacy=0x7fb3400b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.645+0000 7fb335d9b640 1 -- 172.21.15.66:0/1043657892 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3400ceaf0 con 0x7fb3400b7530 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.646+0000 7fb335d9b640 1 --2- 172.21.15.66:0/1043657892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb3400d29b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb328009870 tx=0x7fb32802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fa544f6984dae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.646+0000 7fb33dd9b640 1 -- 172.21.15.66:0/1043657892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328035020 con 0x7fb3400b7530 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.646+0000 7fb33dd9b640 1 -- 172.21.15.66:0/1043657892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb328031640 con 0x7fb3400b7530 2024-03-30T13:09:49.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.646+0000 7fb33dd9b640 1 -- 172.21.15.66:0/1043657892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32802d040 con 0x7fb3400b7530 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.646+0000 7fb344d1c640 1 -- 172.21.15.66:0/1043657892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 msgr2=0x7fb3400d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.646+0000 7fb344d1c640 1 --2- 172.21.15.66:0/1043657892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb3400d29b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb328009870 tx=0x7fb32802b8f0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.647+0000 7fb344d1c640 1 -- 172.21.15.66:0/1043657892 shutdown_connections 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.647+0000 7fb344d1c640 1 --2- 172.21.15.66:0/1043657892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb3400d29b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.647+0000 7fb344d1c640 1 --2- 172.21.15.66:0/1043657892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.647+0000 7fb344d1c640 1 -- 172.21.15.66:0/1043657892 >> 172.21.15.66:0/1043657892 conn(0x7fb3400676e0 msgr2=0x7fb340067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.647+0000 7fb344d1c640 1 -- 172.21.15.66:0/1043657892 shutdown_connections 2024-03-30T13:09:49.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.647+0000 7fb344d1c640 1 -- 172.21.15.66:0/1043657892 wait complete. 2024-03-30T13:09:49.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.648+0000 7fb344d1c640 1 Processor -- start 2024-03-30T13:09:49.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.648+0000 7fb344d1c640 1 -- start start 2024-03-30T13:09:49.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb344d1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb340154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb344d1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3400b7990 0x7fb3400c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb344d1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb344d1c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb34015ea50 con 0x7fb3400b7530 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb344d1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb34015e8d0 con 0x7fb3400b7990 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb344d1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb3400d2ef0 con 0x7fb34015e4f0 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb340154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb340154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41498/0 (socket says 172.21.15.66:41498) 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 -- 172.21.15.66:0/359863896 learned_addr learned my addr 172.21.15.66:0/359863896 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33ed9d640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb335d9b640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3400b7990 0x7fb3400c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 -- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 msgr2=0x7fb3400c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 -- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3400b7990 msgr2=0x7fb3400c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3400b7990 0x7fb3400c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 -- 172.21.15.66:0/359863896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3400c9a30 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.649+0000 7fb33e59c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb340154c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e870 tx=0x7fb33000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.650+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330017070 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.650+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33000c940 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.650+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb340155170 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.650+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000cd30 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.650+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3400c8c70 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.651+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3400c9660 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.651+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb33001a070 con 0x7fb3400b7530 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.652+0000 7fb3377fe640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb32006c560 0x7fb32006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:49.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.652+0000 7fb335d9b640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb32006c560 0x7fb32006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.652+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb33008b820 con 0x7fb3400b7530 2024-03-30T13:09:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.652+0000 7fb335d9b640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb32006c560 0x7fb32006ea20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb328036000 tx=0x7fb32803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:49.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.656+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb3300594b0 con 0x7fb3400b7530 2024-03-30T13:09:49.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.845+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3400ce810 con 0x7fb3400b7530 2024-03-30T13:09:49.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.846+0000 7fb3377fe640 1 -- 172.21.15.66:0/359863896 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7fb33005d7b0 con 0x7fb3400b7530 2024-03-30T13:09:49.850 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:49.850 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb32006c560 msgr2=0x7fb32006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb32006c560 0x7fb32006ea20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb328036000 tx=0x7fb32803c040 comp rx=0 tx=0).stop 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 msgr2=0x7fb340154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb340154c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e870 tx=0x7fb33000ed30 comp rx=0 tx=0).stop 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 shutdown_connections 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb32006c560 0x7fb32006ea20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34015e4f0 0x7fb3400c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3400b7990 0x7fb3400c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 --2- 172.21.15.66:0/359863896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3400b7530 0x7fb340154c30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 >> 172.21.15.66:0/359863896 conn(0x7fb3400676e0 msgr2=0x7fb3400cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 shutdown_connections 2024-03-30T13:09:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:49.848+0000 7fb344d1c640 1 -- 172.21.15.66:0/359863896 wait complete. 2024-03-30T13:09:50.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T13:09:42.480275+0000', 'modified': '2024-03-30T13:09:47.505254+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/4158016355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6833', 'nonce': 4158016355}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 4158016355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4290, 'qdb_cluster': [4290]} 2024-03-30T13:09:50.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T13:09:50.266 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-30T13:09:50.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.374+0000 7fb807e1b640 1 Processor -- start 2024-03-30T13:09:50.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.374+0000 7fb807e1b640 1 -- start start 2024-03-30T13:09:50.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb807e1b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb80015e450 0x7fb8000cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb807e1b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb8000d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb807e1b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8000b9d20 con 0x7fb8000d38d0 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb807e1b640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb8000cea50 con 0x7fb80015e450 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb807e1b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000b9fc0 con 0x7fb800162a80 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb80015e450 0x7fb8000cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb80015e450 0x7fb8000cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41500/0 (socket says 172.21.15.66:41500) 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 -- 172.21.15.66:0/27841656 learned_addr learned my addr 172.21.15.66:0/27841656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb8053b6640 1 --2- 172.21.15.66:0/27841656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb8000d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 -- 172.21.15.66:0/27841656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 msgr2=0x7fb8000d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 --2- 172.21.15.66:0/27841656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb8000d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 -- 172.21.15.66:0/27841656 >> v1:172.21.15.117:6789/0 conn(0x7fb8000d38d0 legacy=0x7fb8000b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb805bb7640 1 -- 172.21.15.66:0/27841656 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800162ee0 con 0x7fb80015e450 2024-03-30T13:09:50.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.375+0000 7fb804bb5640 1 -- 172.21.15.66:0/27841656 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609800994 0 0) 0x7fb8000b9d20 con 0x7fb8000d38d0 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.376+0000 7fb805bb7640 1 --2- 172.21.15.66:0/27841656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80015e450 0x7fb8000cf3c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009870 tx=0x7fb7fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f41816bae2a1f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.376+0000 7fb804bb5640 1 -- 172.21.15.66:0/27841656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc035020 con 0x7fb80015e450 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.376+0000 7fb804bb5640 1 -- 172.21.15.66:0/27841656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc031640 con 0x7fb80015e450 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.376+0000 7fb804bb5640 1 -- 172.21.15.66:0/27841656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02d040 con 0x7fb80015e450 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.376+0000 7fb807e1b640 1 -- 172.21.15.66:0/27841656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80015e450 msgr2=0x7fb8000cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 --2- 172.21.15.66:0/27841656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80015e450 0x7fb8000cf3c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009870 tx=0x7fb7fc02b8f0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 -- 172.21.15.66:0/27841656 shutdown_connections 2024-03-30T13:09:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 --2- 172.21.15.66:0/27841656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb8000d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 --2- 172.21.15.66:0/27841656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb80015e450 0x7fb8000cf3c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 -- 172.21.15.66:0/27841656 >> 172.21.15.66:0/27841656 conn(0x7fb8000678f0 msgr2=0x7fb8000d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 -- 172.21.15.66:0/27841656 shutdown_connections 2024-03-30T13:09:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.377+0000 7fb807e1b640 1 -- 172.21.15.66:0/27841656 wait complete. 2024-03-30T13:09:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.378+0000 7fb807e1b640 1 Processor -- start 2024-03-30T13:09:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.378+0000 7fb807e1b640 1 -- start start 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.378+0000 7fb807e1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 0x7fb8001f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80015e450 0x7fb8001f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb800158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb80015e830 con 0x7fb8000d38d0 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000cef30 con 0x7fb80015e450 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb80015e9b0 con 0x7fb800162a80 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 0x7fb8001f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 0x7fb8001f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41512/0 (socket says 172.21.15.66:41512) 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 -- 172.21.15.66:0/3499226025 learned_addr learned my addr 172.21.15.66:0/3499226025 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb8063b8640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb800158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb8053b6640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80015e450 0x7fb8001f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 -- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 msgr2=0x7fb800158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb800158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 -- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80015e450 msgr2=0x7fb8001f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80015e450 0x7fb8001f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 -- 172.21.15.66:0/3499226025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8001f87a0 con 0x7fb8000d38d0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb805bb7640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 0x7fb8001f8260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc000c00 tx=0x7fb7fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb7f67fc640 1 -- 172.21.15.66:0/3499226025 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc040070 con 0x7fb8000d38d0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb7f67fc640 1 -- 172.21.15.66:0/3499226025 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0399f0 con 0x7fb8000d38d0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8000b4880 con 0x7fb8000d38d0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb7f67fc640 1 -- 172.21.15.66:0/3499226025 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc035040 con 0x7fb8000d38d0 2024-03-30T13:09:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.379+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8000b50c0 con 0x7fb8000d38d0 2024-03-30T13:09:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.381+0000 7fb7f67fc640 1 -- 172.21.15.66:0/3499226025 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb7fc038020 con 0x7fb8000d38d0 2024-03-30T13:09:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.381+0000 7fb7f67fc640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7e406c530 0x7fb7e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.381+0000 7fb7f67fc640 1 -- 172.21.15.66:0/3499226025 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb7fc0ad230 con 0x7fb8000d38d0 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.381+0000 7fb8053b6640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7e406c530 0x7fb7e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.382+0000 7fb8053b6640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7e406c530 0x7fb7e406e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b5970 tx=0x7fb7f0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.382+0000 7fb80aa07740 1 Processor -- start 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.382+0000 7fb80aa07740 1 -- start start 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea790800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea790d40 0x55f2ea7c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb80aa07740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea6fa020 con 0x55f2ea7bd860 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea6a25f0 con 0x55f2ea790d40 2024-03-30T13:09:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55f2ea6f5fd0 con 0x55f2ea7bdc40 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea790800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea790800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41528/0 (socket says 172.21.15.66:41528) 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 -- 172.21.15.66:0/3921042973 learned_addr learned my addr 172.21.15.66:0/3921042973 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb8063b8640 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea790d40 0x55f2ea7c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb8053b6640 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 -- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 msgr2=0x55f2ea7c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 -- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea790d40 msgr2=0x55f2ea7c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea790d40 0x55f2ea7c0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.383+0000 7fb805bb7640 1 -- 172.21.15.66:0/3921042973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ea40e2d0 con 0x55f2ea7bd860 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb805bb7640 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea790800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0090b0 tx=0x7fb7fc07a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ff15a83bd1d052 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb7e37fe640 1 -- 172.21.15.66:0/3921042973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0ad070 con 0x55f2ea7bd860 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb7e37fe640 1 -- 172.21.15.66:0/3921042973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc07f1c0 con 0x55f2ea7bd860 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb7e37fe640 1 -- 172.21.15.66:0/3921042973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0839f0 con 0x55f2ea7bd860 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 -- 172.21.15.66:0/3921042973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 msgr2=0x55f2ea790800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea790800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0090b0 tx=0x7fb7fc07a820 comp rx=0 tx=0).stop 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 -- 172.21.15.66:0/3921042973 shutdown_connections 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea790d40 0x55f2ea7c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 --2- 172.21.15.66:0/3921042973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea790800 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 -- 172.21.15.66:0/3921042973 >> 172.21.15.66:0/3921042973 conn(0x55f2ea792a70 msgr2=0x55f2ea7db5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 -- 172.21.15.66:0/3921042973 shutdown_connections 2024-03-30T13:09:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.384+0000 7fb80aa07740 1 -- 172.21.15.66:0/3921042973 wait complete. 2024-03-30T13:09:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.385+0000 7fb80aa07740 1 -- start start 2024-03-30T13:09:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.385+0000 7fb80aa07740 1 Processor -- start 2024-03-30T13:09:50.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7eed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea7ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea6fa020 con 0x55f2ea7bd860 2024-03-30T13:09:50.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea6b59c0 con 0x55f2ea7b9250 2024-03-30T13:09:50.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55f2ea795110 con 0x55f2ea7bdc40 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb80aa07740 10 client.0 _ll_register_callbacks cb 0x55f2ea699fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb8063b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea7ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41534/0 (socket says 172.21.15.66:41534) 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 -- 172.21.15.66:0/4241995391 learned_addr learned my addr 172.21.15.66:0/4241995391 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb8053b6640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 -- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 msgr2=0x55f2ea7eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 -- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 msgr2=0x55f2ea7ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea7ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb805bb7640 1 -- 172.21.15.66:0/4241995391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ea4d86b0 con 0x55f2ea7bd860 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.386+0000 7fb8063b8640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea7ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb805bb7640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc07ae50 tx=0x7fb7fc031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a19979fa411d64 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb7e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:09:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4241995391 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0839f0 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4241995391 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb7fc078f60 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7fc07f1c0 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4241995391 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc0791c0 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 10 client.4428 fetch_fsmap learned FSMap version 6 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55f2ea7c4770 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4241995391 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc07f1c0 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4241995391 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fb7fc077530 con 0x55f2ea7bd860 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 10 client.4428 fetch_fsmap finished waiting for FSMap version 6 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 20 client.4428 populate_metadata read hostname 'smithi066' 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7a42b0 0x55f2ea7a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 4 client.4428 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 --> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2ea6ce4e0 con 0x55f2ea7a42b0 2024-03-30T13:09:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.387+0000 7fb8053b6640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7a42b0 0x55f2ea7a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb8053b6640 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7a42b0 0x55f2ea7a4690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76ef7ba6edb6878 server_cookie=874cbba66036864f in_seq=0 out_seq=0 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb7e0ff9640 10 client.4428 ms_handle_connect on v2:172.21.15.117:6833/4158016355 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4241995391 <== mds.0 v2:172.21.15.117:6833/4158016355 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55f2ea6ce4e0 con 0x55f2ea7a42b0 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb7e0ff9640 10 client.4428 handle_command_reply: tid=0 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 client.4428 shutdown 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 msgr2=0x55f2ea7b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc07ae50 tx=0x7fb7fc031f90 comp rx=0 tx=0).stop 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 shutdown_connections 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7a42b0 0x55f2ea7a4690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea7ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bdc40 0x55f2ea7eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb7e0ff9640 0 client.4428 ms_handle_reset on v2:172.21.15.117:6833/4158016355 2024-03-30T13:09:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4241995391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8d10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.388+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 >> 172.21.15.66:0/4241995391 conn(0x55f2ea790d40 msgr2=0x55f2ea797630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.389+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 shutdown_connections 2024-03-30T13:09:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.389+0000 7fb80aa07740 1 -- 172.21.15.66:0/4241995391 wait complete. 2024-03-30T13:09:50.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.389+0000 7fb80aa07740 20 client.4428 trim_cache size 0 max 16384 2024-03-30T13:09:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.407+0000 7fb80aa07740 1 Processor -- start 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.407+0000 7fb80aa07740 1 -- start start 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7b9250 0x55f2ea7b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea7dafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb80aa07740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb800201010 con 0x55f2ea7bdc40 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000ce100 con 0x55f2ea7bd860 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb8002009c0 con 0x55f2ea7b9250 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea7dafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea7dafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41540/0 (socket says 172.21.15.66:41540) 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 -- 172.21.15.66:0/1787396004 learned_addr learned my addr 172.21.15.66:0/1787396004 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb805bb7640 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7b9250 0x55f2ea7b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8053b6640 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 -- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7b9250 msgr2=0x55f2ea7b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7b9250 0x55f2ea7b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 -- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7bd860 msgr2=0x55f2ea7b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 -- 172.21.15.66:0/1787396004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ea7b9080 con 0x55f2ea7bdc40 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.408+0000 7fb8063b8640 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea7dafd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80012e0 tx=0x7fb7f8015780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74b82371f7998b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb7e2ffd640 1 -- 172.21.15.66:0/1787396004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8015ea0 con 0x55f2ea7bdc40 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb7e2ffd640 1 -- 172.21.15.66:0/1787396004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f8006d60 con 0x55f2ea7bdc40 2024-03-30T13:09:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb7e2ffd640 1 -- 172.21.15.66:0/1787396004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8003b90 con 0x55f2ea7bdc40 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 -- 172.21.15.66:0/1787396004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 msgr2=0x55f2ea7dafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea7dafd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80012e0 tx=0x7fb7f8015780 comp rx=0 tx=0).stop 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 -- 172.21.15.66:0/1787396004 shutdown_connections 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea7dafd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7bd860 0x55f2ea7b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 --2- 172.21.15.66:0/1787396004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7b9250 0x55f2ea7b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 -- 172.21.15.66:0/1787396004 >> 172.21.15.66:0/1787396004 conn(0x55f2ea790d40 msgr2=0x55f2ea7a4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 -- 172.21.15.66:0/1787396004 shutdown_connections 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.409+0000 7fb80aa07740 1 -- 172.21.15.66:0/1787396004 wait complete. 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.410+0000 7fb80aa07740 1 -- start start 2024-03-30T13:09:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.410+0000 7fb80aa07740 1 Processor -- start 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea856f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bd860 0x55f2ea7dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea857fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea6fa020 con 0x55f2ea7bdc40 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea795110 con 0x55f2ea7b9250 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55f2ea6b59c0 con 0x55f2ea7bd860 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 10 client.0 _ll_register_callbacks cb 0x7fb7f8001850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8063b8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea857fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bd860 0x55f2ea7dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8063b8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea857fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41546/0 (socket says 172.21.15.66:41546) 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8063b8640 1 -- 172.21.15.66:0/4158562746 learned_addr learned my addr 172.21.15.66:0/4158562746 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb805bb7640 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea856f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 -- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 msgr2=0x55f2ea856f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea856f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 -- 172.21.15.66:0/4158562746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 msgr2=0x55f2ea857fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea857fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 -- 172.21.15.66:0/4158562746 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ea7dbfd0 con 0x55f2ea7bd860 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb8053b6640 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bd860 0x55f2ea7dba90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x55f2ea7bef70 tx=0x7fb7f0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c6f460f890a21b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb7e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4158562746 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0031030 con 0x55f2ea7bd860 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f8006d60 con 0x55f2ea7bd860 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.411+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4158562746 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb7f000f040 con 0x55f2ea7bd860 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4158562746 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fb7f003d050 con 0x55f2ea7bd860 2024-03-30T13:09:50.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 10 client.4352 fetch_fsmap learned FSMap version 6 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55f2ea858950 con 0x55f2ea7bd860 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4158562746 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f8006d60 con 0x55f2ea7bd860 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4158562746 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fb7f0040020 con 0x55f2ea7bd860 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 10 client.4352 fetch_fsmap finished waiting for FSMap version 6 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 10 client.4352 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 20 client.4352 populate_metadata read hostname 'smithi066' 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7b7100 0x55f2ea7f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 4 client.4352 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 --> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] -- command(tid 0: {"prefix": "status"}) -- 0x55f2ea707560 con 0x55f2ea7b7100 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb805bb7640 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7b7100 0x55f2ea7f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb805bb7640 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7b7100 0x55f2ea7f5880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb31668763c1c6df server_cookie=1a8727d554742114 in_seq=0 out_seq=0 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.412+0000 7fb7e0ff9640 10 client.4352 ms_handle_connect on v2:172.21.15.117:6833/4158016355 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb7e0ff9640 1 -- 172.21.15.66:0/4158562746 <== mds.0 v2:172.21.15.117:6833/4158016355 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55f2ea707560 con 0x55f2ea7b7100 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb7e0ff9640 10 client.4352 handle_command_reply: tid=0 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 client.4352 shutdown 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bd860 msgr2=0x55f2ea7dba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bd860 0x55f2ea7dba90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x55f2ea7bef70 tx=0x7fb7f0004730 comp rx=0 tx=0).stop 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 shutdown_connections 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f2ea7b7100 0x55f2ea7f5880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f2ea7bdc40 0x55f2ea857fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f2ea7bd860 0x55f2ea7dba90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 --2- 172.21.15.66:0/4158562746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2ea7b9250 0x55f2ea856f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb7e0ff9640 0 client.4352 ms_handle_reset on v2:172.21.15.117:6833/4158016355 2024-03-30T13:09:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 >> 172.21.15.66:0/4158562746 conn(0x55f2ea790d40 msgr2=0x55f2ea7b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 shutdown_connections 2024-03-30T13:09:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 1 -- 172.21.15.66:0/4158562746 wait complete. 2024-03-30T13:09:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb80aa07740 20 client.4352 trim_cache size 0 max 16384 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "03992909-781c-43b4-940b-d3d6d831de0b", 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4290, 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 7.3021544909999996, 2024-03-30T13:09:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 6, 2024-03-30T13:09:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 26, 2024-03-30T13:09:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 25, 2024-03-30T13:09:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 14.19799158 2024-03-30T13:09:50.428 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:09:50.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7e406c530 msgr2=0x7fb7e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7e406c530 0x7fb7e406e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b5970 tx=0x7fb7f0009190 comp rx=0 tx=0).stop 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 msgr2=0x7fb8001f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 0x7fb8001f8260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc000c00 tx=0x7fb7fc0046c0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 shutdown_connections 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7e406c530 0x7fb7e406e9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb800162a80 0x7fb800158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80015e450 0x7fb8001f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.413+0000 7fb807e1b640 1 --2- 172.21.15.66:0/3499226025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8000d38d0 0x7fb8001f8260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.414+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 >> 172.21.15.66:0/3499226025 conn(0x7fb8000678f0 msgr2=0x7fb80006c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.414+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 shutdown_connections 2024-03-30T13:09:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.414+0000 7fb807e1b640 1 -- 172.21.15.66:0/3499226025 wait complete. 2024-03-30T13:09:50.431 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:09:50.431 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:50.432 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:09:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.535+0000 7f71dae7b640 1 Processor -- start 2024-03-30T13:09:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.535+0000 7f71dae7b640 1 -- start start 2024-03-30T13:09:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71dae7b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f71d4145090 0x7f71d40b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71dae7b640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d40b5f80 con 0x7f71d4160540 2024-03-30T13:09:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71dae7b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d40b6260 con 0x7f71d40b5aa0 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71dae7b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f71d40b6540 con 0x7f71d4145090 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71d9418640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f71d4160540 0x7f71d40d11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:34180/0 (socket says 172.21.15.66:34180) 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71d9418640 1 -- 172.21.15.66:0/3228692172 learned_addr learned my addr 172.21.15.66:0/3228692172 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.536+0000 7f71d8c17640 1 --2- 172.21.15.66:0/3228692172 >> v2:172.21.15.117:3300/0 conn(0x7f71d4145090 0x7f71d40b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.537+0000 7f71d37fe640 1 -- 172.21.15.66:0/3228692172 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 814014032 0 0) 0x7f71d40b5f80 con 0x7f71d4160540 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.537+0000 7f71d37fe640 1 -- 172.21.15.66:0/3228692172 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71c0003520 con 0x7f71d4160540 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.537+0000 7f71d8c17640 1 -- 172.21.15.66:0/3228692172 >> v1:172.21.15.117:6789/0 conn(0x7f71d40b5aa0 legacy=0x7f71d415fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.537+0000 7f71d8c17640 1 -- 172.21.15.66:0/3228692172 >> v1:172.21.15.66:6789/0 conn(0x7f71d4160540 legacy=0x7f71d40d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.537+0000 7f71d37fe640 1 -- 172.21.15.66:0/3228692172 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244553934 0 0) 0x7f71d40b6260 con 0x7f71d40b5aa0 2024-03-30T13:09:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.537+0000 7f71d8c17640 1 -- 172.21.15.66:0/3228692172 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d40b6820 con 0x7f71d4145090 2024-03-30T13:09:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71d8c17640 1 --2- 172.21.15.66:0/3228692172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4145090 0x7f71d40b1720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009d90 tx=0x7f71cc02bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38eee053b1d891fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71d37fe640 1 -- 172.21.15.66:0/3228692172 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc039070 con 0x7f71d4145090 2024-03-30T13:09:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71d37fe640 1 -- 172.21.15.66:0/3228692172 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71cc02c7d0 con 0x7f71d4145090 2024-03-30T13:09:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71d37fe640 1 -- 172.21.15.66:0/3228692172 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc001670 con 0x7f71d4145090 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 -- 172.21.15.66:0/3228692172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4145090 msgr2=0x7f71d40b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3228692172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4145090 0x7f71d40b1720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009d90 tx=0x7f71cc02bca0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 -- 172.21.15.66:0/3228692172 shutdown_connections 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3228692172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4145090 0x7f71d40b1720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 -- 172.21.15.66:0/3228692172 >> 172.21.15.66:0/3228692172 conn(0x7f71d406f8b0 msgr2=0x7f71d406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 -- 172.21.15.66:0/3228692172 shutdown_connections 2024-03-30T13:09:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.538+0000 7f71dae7b640 1 -- 172.21.15.66:0/3228692172 wait complete. 2024-03-30T13:09:50.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.539+0000 7f71dae7b640 1 Processor -- start 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 -- start start 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 0x7f71d406d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d4145090 0x7f71d413cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4160540 0x7f71d406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4145470 con 0x7f71d40b5aa0 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4146000 con 0x7f71d4160540 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.540+0000 7f71dae7b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f71d41455f0 con 0x7f71d4145090 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 0x7f71d406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 0x7f71d406d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41556/0 (socket says 172.21.15.66:41556) 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 -- 172.21.15.66:0/3301819360 learned_addr learned my addr 172.21.15.66:0/3301819360 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d9418640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4160540 0x7f71d406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d3fff640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d4145090 0x7f71d413cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 -- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d4145090 msgr2=0x7f71d413cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d4145090 0x7f71d413cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 -- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4160540 msgr2=0x7f71d406dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4160540 0x7f71d406dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 -- 172.21.15.66:0/3301819360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d406d640 con 0x7f71d40b5aa0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d8c17640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 0x7f71d406d010 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f71cc037040 tx=0x7f71cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc044020 con 0x7f71d40b5aa0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71cc034b70 con 0x7f71d40b5aa0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d415cad0 con 0x7f71d40b5aa0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc039040 con 0x7f71d40b5aa0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.541+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d415d090 con 0x7f71d40b5aa0 2024-03-30T13:09:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.542+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d4075a10 con 0x7f71d40b5aa0 2024-03-30T13:09:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.543+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f71cc047020 con 0x7f71d40b5aa0 2024-03-30T13:09:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.543+0000 7f71d17fa640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f71b406c560 0x7f71b406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.543+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f71cc0add30 con 0x7f71d40b5aa0 2024-03-30T13:09:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.543+0000 7f71d3fff640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f71b406c560 0x7f71b406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.544+0000 7f71d3fff640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f71b406c560 0x7f71b406ea20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71d4143ee0 tx=0x7f71c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:50.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.548+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f71cc04e070 con 0x7f71d40b5aa0 2024-03-30T13:09:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.770+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71d40b4420 con 0x7f71d40b5aa0 2024-03-30T13:09:50.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.770+0000 7f71d17fa640 1 -- 172.21.15.66:0/3301819360 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7f71cc07b940 con 0x7f71d40b5aa0 2024-03-30T13:09:50.774 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:09:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f71b406c560 msgr2=0x7f71b406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f71b406c560 0x7f71b406ea20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71d4143ee0 tx=0x7f71c4009040 comp rx=0 tx=0).stop 2024-03-30T13:09:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 msgr2=0x7f71d406d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:50.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 0x7f71d406d010 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f71cc037040 tx=0x7f71cc004a40 comp rx=0 tx=0).stop 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 shutdown_connections 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f71b406c560 0x7f71b406ea20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4160540 0x7f71d406dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d4145090 0x7f71d413cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 --2- 172.21.15.66:0/3301819360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d40b5aa0 0x7f71d406d010 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.772+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 >> 172.21.15.66:0/3301819360 conn(0x7f71d406f8b0 msgr2=0x7f71d40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.773+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 shutdown_connections 2024-03-30T13:09:50.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50.773+0000 7f71dae7b640 1 -- 172.21.15.66:0/3301819360 wait complete. 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T13:09:50.787 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:09:50.788 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:09:50.788 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T13:09:50.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:50.788 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T13:09:50.804 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42640sec preferred_lft 42640sec 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T13:09:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:09:50.806 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T13:09:50.806 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:09:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:09:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:51.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:51.028 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:09:51.061 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T13:09:51.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:51.063 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T13:09:51.129 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T13:09:51.130 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:09:51.130 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:09:51.130 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:09:51.130 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:09:51.130 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:09:51.130 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:09:51.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:09:51.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:09:51.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:09:51.131 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:09:51.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:09:51.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:51.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:51.468 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:09:51.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:51.484 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-03-30T13:09:51.538 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:09:51.539 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:09:51.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:09:51.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-30T13:09:51.539 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:09:51.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:09:51.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:51.668 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-30T13:09:51.669 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:09:51.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:09:51.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:51.826 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:09:51.826 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:09:51.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T13:09:51.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T13:09:51.826 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:09:51.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:09:51.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:51.916 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-30T13:09:51.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:09:51.917 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:51.932 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T13:09:51.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:09:51.932 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:51.987 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T13:09:52.060 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:09:52.116 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-03-30T13:09:52.117 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-03-30T13:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:09:52.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:52.119 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T13:09:52.172 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T13:09:52.173 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-30T13:09:52.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-03-30T13:09:52.245 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:09:52.301 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-03-30T13:09:52.301 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-03-30T13:09:52.301 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-03-30T13:09:52.302 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:09:52.303 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:09:52.304 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-03-30T13:09:52.304 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:09:52.304 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:09:52.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:52.304 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-03-30T13:09:52.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:2024-03-30T13:09:52.306+0000 7f1edc245480 -1 init, newargv = 0x5608e347c230 newargc=15 2024-03-30T13:09:52.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[44257]: starting ceph client 2024-03-30T13:09:52.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[44257]: starting fuse 2024-03-30T13:09:52.325 INFO:teuthology.orchestra.run.smithi066.stdout:41 2024-03-30T13:09:52.325 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-03-30T13:09:52.326 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.admin")) 2024-03-30T13:09:52.327 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-03-30T13:09:52.451 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.admin.44257.asok 2024-03-30T13:09:52.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:52.461 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44257.asok 2024-03-30T13:09:52.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:52.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44257.asok status 2024-03-30T13:09:52.604 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "admin", 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "44257", 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-03-30T13:09:52.605 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4449, 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4449 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 693354920 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:09:52.606 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 693354920 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4449 192.168.0.1:0/693354920", 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/693354920", 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 6, 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 26, 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-03-30T13:09:52.607 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:09:52.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:52.614 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-03-30T13:09:52.614 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:52.629 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-03-30T13:09:52.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:52.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:52.630 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:52.702 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-30T13:09:52.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:09:52.960 INFO:teuthology.orchestra.run.smithi066.stdout:# file: . 2024-03-30T13:09:52.960 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-30T13:09:52.960 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:09:52.966 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:52.967 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:09:52.983 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.127+0000 7fb1f4d32640 1 Processor -- start 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.127+0000 7fb1f4d32640 1 -- start start 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1f4d32640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1f015e950 0x7fb1f00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1f4d32640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f00d36f0 con 0x7fb1f00d30d0 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1f4d32640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f00d39d0 con 0x7fb1f00d48c0 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1f4d32640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb1f00d3cb0 con 0x7fb1f015e950 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1ee59c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1f015e950 0x7fb1f00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1eed9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb1f00d30d0 0x7fb1f00b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46268/0 (socket says 172.21.15.66:46268) 2024-03-30T13:09:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1eed9d640 1 -- 172.21.15.66:0/2517735892 learned_addr learned my addr 172.21.15.66:0/2517735892 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1ee59c640 1 -- 172.21.15.66:0/2517735892 >> v1:172.21.15.117:6789/0 conn(0x7fb1f00d48c0 legacy=0x7fb1f00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1ee59c640 1 -- 172.21.15.66:0/2517735892 >> v1:172.21.15.66:6789/0 conn(0x7fb1f00d30d0 legacy=0x7fb1f00b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1edd9b640 1 -- 172.21.15.66:0/2517735892 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 124392341 0 0) 0x7fb1f00d39d0 con 0x7fb1f00d48c0 2024-03-30T13:09:53.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.128+0000 7fb1ee59c640 1 -- 172.21.15.66:0/2517735892 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f00b7ac0 con 0x7fb1f015e950 2024-03-30T13:09:53.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.129+0000 7fb1ee59c640 1 --2- 172.21.15.66:0/2517735892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f015e950 0x7fb1f00d4380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00097a0 tx=0x7fb1e002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215201a8188e180a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.129+0000 7fb1edd9b640 1 -- 172.21.15.66:0/2517735892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0038070 con 0x7fb1f015e950 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.129+0000 7fb1edd9b640 1 -- 172.21.15.66:0/2517735892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0031700 con 0x7fb1f015e950 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.129+0000 7fb1edd9b640 1 -- 172.21.15.66:0/2517735892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0031ad0 con 0x7fb1f015e950 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.129+0000 7fb1f4d32640 1 -- 172.21.15.66:0/2517735892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f015e950 msgr2=0x7fb1f00d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.129+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/2517735892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f015e950 0x7fb1f00d4380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00097a0 tx=0x7fb1e002b820 comp rx=0 tx=0).stop 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.130+0000 7fb1f4d32640 1 -- 172.21.15.66:0/2517735892 shutdown_connections 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.130+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/2517735892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f015e950 0x7fb1f00d4380 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.130+0000 7fb1f4d32640 1 -- 172.21.15.66:0/2517735892 >> 172.21.15.66:0/2517735892 conn(0x7fb1f00676e0 msgr2=0x7fb1f0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.130+0000 7fb1f4d32640 1 -- 172.21.15.66:0/2517735892 shutdown_connections 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.130+0000 7fb1f4d32640 1 -- 172.21.15.66:0/2517735892 wait complete. 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.131+0000 7fb1f4d32640 1 Processor -- start 2024-03-30T13:09:53.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.131+0000 7fb1f4d32640 1 -- start start 2024-03-30T13:09:53.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1f4d32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 0x7fb1f014b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1f4d32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f00d48c0 0x7fb1f0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1f4d32640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1f015e950 0x7fb1f00c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1f4d32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f015ed30 con 0x7fb1f00d30d0 2024-03-30T13:09:53.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1f4d32640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00d4d00 con 0x7fb1f00d48c0 2024-03-30T13:09:53.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1f4d32640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f015eeb0 con 0x7fb1f015e950 2024-03-30T13:09:53.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 0x7fb1f014b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 0x7fb1f014b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41454/0 (socket says 172.21.15.66:41454) 2024-03-30T13:09:53.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 -- 172.21.15.66:0/408081704 learned_addr learned my addr 172.21.15.66:0/408081704 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1e5d9b640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f00d48c0 0x7fb1f0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1eed9d640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1f015e950 0x7fb1f00c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 -- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1f015e950 msgr2=0x7fb1f00c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1f015e950 0x7fb1f00c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 -- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f00d48c0 msgr2=0x7fb1f0159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f00d48c0 0x7fb1f0159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 -- 172.21.15.66:0/408081704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f0158c30 con 0x7fb1f00d30d0 2024-03-30T13:09:53.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.132+0000 7fb1ee59c640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 0x7fb1f014b6d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0000c00 tx=0x7fb1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.133+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0043070 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.133+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0033a30 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.133+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f0158f10 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.133+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0038040 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.133+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01594d0 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.134+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0075a10 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.134+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb1e0046070 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.134+0000 7fb1e77fe640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1d006c530 0x7fb1d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.135+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb1e00ad820 con 0x7fb1f00d30d0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.135+0000 7fb1e5d9b640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1d006c530 0x7fb1d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.135+0000 7fb1e5d9b640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1d006c530 0x7fb1d006e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00b98b0 tx=0x7fb1d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.139+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb1e0030070 con 0x7fb1f00d30d0 2024-03-30T13:09:53.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.328+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1f00b67a0 con 0x7fb1f00d30d0 2024-03-30T13:09:53.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.329+0000 7fb1e77fe640 1 -- 172.21.15.66:0/408081704 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7fb1e0033e20 con 0x7fb1f00d30d0 2024-03-30T13:09:53.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:53.333 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1d006c530 msgr2=0x7fb1d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1d006c530 0x7fb1d006e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00b98b0 tx=0x7fb1d8009040 comp rx=0 tx=0).stop 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 msgr2=0x7fb1f014b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 0x7fb1f014b6d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0000c00 tx=0x7fb1e00046c0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 shutdown_connections 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1d006c530 0x7fb1d006e9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1f015e950 0x7fb1f00c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f00d48c0 0x7fb1f0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 --2- 172.21.15.66:0/408081704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f00d30d0 0x7fb1f014b6d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 >> 172.21.15.66:0/408081704 conn(0x7fb1f00676e0 msgr2=0x7fb1f00b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 shutdown_connections 2024-03-30T13:09:53.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.331+0000 7fb1f4d32640 1 -- 172.21.15.66:0/408081704 wait complete. 2024-03-30T13:09:53.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:09:53.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.454+0000 7fac7f6ce640 1 Processor -- start 2024-03-30T13:09:53.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.454+0000 7fac7f6ce640 1 -- start start 2024-03-30T13:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7f6ce640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fac780b3760 0x7fac780cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7f6ce640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac780cf860 con 0x7fac780d3ba0 2024-03-30T13:09:53.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7f6ce640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac780cfb40 con 0x7fac780d13d0 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7f6ce640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fac780d33a0 con 0x7fac780b3760 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7cc69640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fac780b3760 0x7fac780cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7dc6b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fac780d3ba0 0x7fac780b7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46278/0 (socket says 172.21.15.66:46278) 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.455+0000 7fac7dc6b640 1 -- 172.21.15.66:0/1484386948 learned_addr learned my addr 172.21.15.66:0/1484386948 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.456+0000 7fac77fff640 1 -- 172.21.15.66:0/1484386948 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587804744 0 0) 0x7fac780cfb40 con 0x7fac780d13d0 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.456+0000 7fac7cc69640 1 -- 172.21.15.66:0/1484386948 >> v1:172.21.15.117:6789/0 conn(0x7fac780d13d0 legacy=0x7fac780d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.456+0000 7fac7cc69640 1 -- 172.21.15.66:0/1484386948 >> v1:172.21.15.66:6789/0 conn(0x7fac780d3ba0 legacy=0x7fac780b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.456+0000 7fac7cc69640 1 -- 172.21.15.66:0/1484386948 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac780d3680 con 0x7fac780b3760 2024-03-30T13:09:53.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.456+0000 7fac7cc69640 1 --2- 172.21.15.66:0/1484386948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780b3760 0x7fac780cecc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0097a0 tx=0x7fac6c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea59d63d369ce727 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.456+0000 7fac77fff640 1 -- 172.21.15.66:0/1484386948 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c037070 con 0x7fac780b3760 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac77fff640 1 -- 172.21.15.66:0/1484386948 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c030710 con 0x7fac780b3760 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac77fff640 1 -- 172.21.15.66:0/1484386948 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c030ae0 con 0x7fac780b3760 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 -- 172.21.15.66:0/1484386948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780b3760 msgr2=0x7fac780cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/1484386948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780b3760 0x7fac780cecc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0097a0 tx=0x7fac6c02b790 comp rx=0 tx=0).stop 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 -- 172.21.15.66:0/1484386948 shutdown_connections 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/1484386948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780b3760 0x7fac780cecc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 -- 172.21.15.66:0/1484386948 >> 172.21.15.66:0/1484386948 conn(0x7fac78067710 msgr2=0x7fac78067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 -- 172.21.15.66:0/1484386948 shutdown_connections 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.457+0000 7fac7f6ce640 1 -- 172.21.15.66:0/1484386948 wait complete. 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.458+0000 7fac7f6ce640 1 Processor -- start 2024-03-30T13:09:53.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.458+0000 7fac7f6ce640 1 -- start start 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7f6ce640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac780b3760 0x7fac780b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7f6ce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 0x7fac781e8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7f6ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac780d3ba0 0x7fac7815d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7f6ce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac7815eab0 con 0x7fac780d13d0 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7f6ce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac7815e930 con 0x7fac780d3ba0 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7f6ce640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fac780cf4c0 con 0x7fac780b3760 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 0x7fac781e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 0x7fac781e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41488/0 (socket says 172.21.15.66:41488) 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 -- 172.21.15.66:0/3852550217 learned_addr learned my addr 172.21.15.66:0/3852550217 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:53.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7d46a640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac780b3760 0x7fac780b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7dc6b640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac780d3ba0 0x7fac7815d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 -- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac780b3760 msgr2=0x7fac780b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac780b3760 0x7fac780b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 -- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac780d3ba0 msgr2=0x7fac7815d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac780d3ba0 0x7fac7815d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.459+0000 7fac7cc69640 1 -- 172.21.15.66:0/3852550217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac780b6300 con 0x7fac780d13d0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.460+0000 7fac7cc69640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 0x7fac781e8090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.460+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c042070 con 0x7fac780d13d0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.460+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c032a40 con 0x7fac780d13d0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.460+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac780b65e0 con 0x7fac780d13d0 2024-03-30T13:09:53.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.460+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c037040 con 0x7fac780d13d0 2024-03-30T13:09:53.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.460+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac780b6ba0 con 0x7fac780d13d0 2024-03-30T13:09:53.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.461+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac78075890 con 0x7fac780d13d0 2024-03-30T13:09:53.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.462+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fac6c045070 con 0x7fac780d13d0 2024-03-30T13:09:53.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.462+0000 7fac75ffb640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fac6406c530 0x7fac6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.462+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fac6c0ad840 con 0x7fac780d13d0 2024-03-30T13:09:53.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.462+0000 7fac7d46a640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fac6406c530 0x7fac6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.463+0000 7fac7d46a640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fac6406c530 0x7fac6406e9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac70000f60 tx=0x7fac70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.467+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fac6c07b5f0 con 0x7fac780d13d0 2024-03-30T13:09:53.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.650+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac78161f20 con 0x7fac780d13d0 2024-03-30T13:09:53.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.650+0000 7fac75ffb640 1 -- 172.21.15.66:0/3852550217 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fac6c07f8f0 con 0x7fac780d13d0 2024-03-30T13:09:53.654 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:09:53.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fac6406c530 msgr2=0x7fac6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fac6406c530 0x7fac6406e9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac70000f60 tx=0x7fac70009040 comp rx=0 tx=0).stop 2024-03-30T13:09:53.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 msgr2=0x7fac781e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 0x7fac781e8090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 shutdown_connections 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fac6406c530 0x7fac6406e9f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac780d3ba0 0x7fac7815d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac780d13d0 0x7fac781e8090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 --2- 172.21.15.66:0/3852550217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac780b3760 0x7fac780b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 >> 172.21.15.66:0/3852550217 conn(0x7fac78067710 msgr2=0x7fac780c9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 shutdown_connections 2024-03-30T13:09:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.652+0000 7fac7f6ce640 1 -- 172.21.15.66:0/3852550217 wait complete. 2024-03-30T13:09:53.667 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 Processor -- start 2024-03-30T13:09:53.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 -- start start 2024-03-30T13:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e740ba090 0x7f2e740d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2e740b73d0 0x7f2e740b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e740d2ff0 con 0x7f2e740b5dc0 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2e740d32d0 con 0x7f2e740ba090 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.774+0000 7f2e7a48f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f2e740d35b0 con 0x7f2e740b73d0 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e737fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e740ba090 0x7f2e740d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e73fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2e740b5dc0 0x7f2e740b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:37900/0 (socket says 172.21.15.66:37900) 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e78a2c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2e740b73d0 0x7f2e740b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e73fff640 1 -- 172.21.15.66:0/2958271605 learned_addr learned my addr 172.21.15.66:0/2958271605 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e737fe640 1 -- 172.21.15.66:0/2958271605 >> v2:172.21.15.117:3300/0 conn(0x7f2e740b73d0 msgr2=0x7f2e740b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e72ffd640 1 -- 172.21.15.66:0/2958271605 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596910785 0 0) 0x7f2e740d2ff0 con 0x7f2e740b5dc0 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e737fe640 1 --2- 172.21.15.66:0/2958271605 >> v2:172.21.15.117:3300/0 conn(0x7f2e740b73d0 0x7f2e740b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e737fe640 1 -- 172.21.15.66:0/2958271605 >> v1:172.21.15.117:6789/0 conn(0x7f2e740b5dc0 legacy=0x7f2e740b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e737fe640 1 -- 172.21.15.66:0/2958271605 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e740d3890 con 0x7f2e740ba090 2024-03-30T13:09:53.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.775+0000 7f2e78a2c640 1 --2- 172.21.15.66:0/2958271605 >> v2:172.21.15.117:3300/0 conn(0x7f2e740b73d0 0x7f2e740b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e737fe640 1 --2- 172.21.15.66:0/2958271605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e740d2680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e680097a0 tx=0x7f2e6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1c64e4ddd8c3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e72ffd640 1 -- 172.21.15.66:0/2958271605 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68035020 con 0x7f2e740ba090 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e72ffd640 1 -- 172.21.15.66:0/2958271605 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e68031830 con 0x7f2e740ba090 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e72ffd640 1 -- 172.21.15.66:0/2958271605 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68031c00 con 0x7f2e740ba090 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 -- 172.21.15.66:0/2958271605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 msgr2=0x7f2e740d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/2958271605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e740d2680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e680097a0 tx=0x7f2e6802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 -- 172.21.15.66:0/2958271605 shutdown_connections 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/2958271605 >> v2:172.21.15.117:3300/0 conn(0x7f2e740b73d0 0x7f2e740b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/2958271605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e740d2680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 -- 172.21.15.66:0/2958271605 >> 172.21.15.66:0/2958271605 conn(0x7f2e740678f0 msgr2=0x7f2e740d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 -- 172.21.15.66:0/2958271605 shutdown_connections 2024-03-30T13:09:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.776+0000 7f2e7a48f640 1 -- 172.21.15.66:0/2958271605 wait complete. 2024-03-30T13:09:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.777+0000 7f2e7a48f640 1 Processor -- start 2024-03-30T13:09:53.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 -- start start 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e740b5dc0 0x7f2e74155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e740b73d0 0x7f2e741574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e7415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7415e880 con 0x7f2e740ba090 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e740d4090 con 0x7f2e740b73d0 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.778+0000 7f2e7a48f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7415ea00 con 0x7f2e740b5dc0 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e7415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e7415f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41508/0 (socket says 172.21.15.66:41508) 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 -- 172.21.15.66:0/4106873878 learned_addr learned my addr 172.21.15.66:0/4106873878 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e737fe640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e740b73d0 0x7f2e741574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e73fff640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e740b5dc0 0x7f2e74155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 -- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e740b5dc0 msgr2=0x7f2e74155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e740b5dc0 0x7f2e74155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 -- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e740b73d0 msgr2=0x7f2e741574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e740b73d0 0x7f2e741574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 -- 172.21.15.66:0/4106873878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e740cb4b0 con 0x7f2e740ba090 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e78a2c640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e7415f590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00efc0 tx=0x7f2e6c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018030 con 0x7f2e740ba090 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6c00cde0 con 0x7f2e740ba090 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e740cb790 con 0x7f2e740ba090 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c003bb0 con 0x7f2e740ba090 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.779+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e74160010 con 0x7f2e740ba090 2024-03-30T13:09:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.780+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e74075890 con 0x7f2e740ba090 2024-03-30T13:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.781+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e6c01b070 con 0x7f2e740ba090 2024-03-30T13:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.781+0000 7f2e70ff9640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e6006c530 0x7f2e6006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.781+0000 7f2e73fff640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e6006c530 0x7f2e6006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.781+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e6c08c350 con 0x7f2e740ba090 2024-03-30T13:09:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.782+0000 7f2e73fff640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e6006c530 0x7f2e6006e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e64000f60 tx=0x7f2e64007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.786+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e6c010040 con 0x7f2e740ba090 2024-03-30T13:09:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.971+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e7406b580 con 0x7f2e740ba090 2024-03-30T13:09:53.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.972+0000 7f2e70ff9640 1 -- 172.21.15.66:0/4106873878 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f2e6c059fe0 con 0x7f2e740ba090 2024-03-30T13:09:53.976 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e6006c530 msgr2=0x7f2e6006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e6006c530 0x7f2e6006e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e64000f60 tx=0x7f2e64007d50 comp rx=0 tx=0).stop 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 msgr2=0x7f2e7415f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e7415f590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00efc0 tx=0x7f2e6c009f90 comp rx=0 tx=0).stop 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 shutdown_connections 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e6006c530 0x7f2e6006e9f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740ba090 0x7f2e7415f590 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e740b73d0 0x7f2e741574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 --2- 172.21.15.66:0/4106873878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e740b5dc0 0x7f2e74155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 >> 172.21.15.66:0/4106873878 conn(0x7f2e740678f0 msgr2=0x7f2e740b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 shutdown_connections 2024-03-30T13:09:53.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:53.974+0000 7f2e7a48f640 1 -- 172.21.15.66:0/4106873878 wait complete. 2024-03-30T13:09:53.988 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:09:54.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.094+0000 7fe04e081640 1 Processor -- start 2024-03-30T13:09:54.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.094+0000 7fe04e081640 1 -- start start 2024-03-30T13:09:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe04e081640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe04815e570 0x7fe0480b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe04e081640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0480cf7f0 0x7fe0480d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe04e081640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe04815e950 con 0x7fe0480ba0b0 2024-03-30T13:09:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe04e081640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe0480b94a0 con 0x7fe04815e570 2024-03-30T13:09:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe04e081640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe0480b8890 con 0x7fe0480cf7f0 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe0477fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe04815e570 0x7fe0480b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.095+0000 7fe047fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0480cf7f0 0x7fe0480d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.096+0000 7fe0477fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe04815e570 0x7fe0480b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:45958/0 (socket says 172.21.15.66:45958) 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.096+0000 7fe0477fe640 1 -- 172.21.15.66:0/1905497226 learned_addr learned my addr 172.21.15.66:0/1905497226 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.096+0000 7fe0477fe640 1 -- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0480cf7f0 msgr2=0x7fe0480d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.096+0000 7fe0477fe640 1 --2- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0480cf7f0 0x7fe0480d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.096+0000 7fe0477fe640 1 -- 172.21.15.66:0/1905497226 >> v1:172.21.15.117:6789/0 conn(0x7fe0480ba0b0 legacy=0x7fe0480cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.096+0000 7fe0477fe640 1 -- 172.21.15.66:0/1905497226 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0480b8b70 con 0x7fe04815e570 2024-03-30T13:09:54.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe0477fe640 1 --2- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe04815e570 0x7fe0480b7f90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0480b9620 tx=0x7fe03802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36a8206dd56f6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe0467fc640 1 -- 172.21.15.66:0/1905497226 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03802cae0 con 0x7fe04815e570 2024-03-30T13:09:54.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe0467fc640 1 -- 172.21.15.66:0/1905497226 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03802cc80 con 0x7fe04815e570 2024-03-30T13:09:54.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe0467fc640 1 -- 172.21.15.66:0/1905497226 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038034610 con 0x7fe04815e570 2024-03-30T13:09:54.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 -- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe04815e570 msgr2=0x7fe0480b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 --2- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe04815e570 0x7fe0480b7f90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0480b9620 tx=0x7fe03802bf40 comp rx=0 tx=0).stop 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 -- 172.21.15.66:0/1905497226 shutdown_connections 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 --2- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0480cf7f0 0x7fe0480d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 --2- 172.21.15.66:0/1905497226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe04815e570 0x7fe0480b7f90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 -- 172.21.15.66:0/1905497226 >> 172.21.15.66:0/1905497226 conn(0x7fe0480676e0 msgr2=0x7fe048067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 -- 172.21.15.66:0/1905497226 shutdown_connections 2024-03-30T13:09:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.097+0000 7fe04e081640 1 -- 172.21.15.66:0/1905497226 wait complete. 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.098+0000 7fe04e081640 1 Processor -- start 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 -- start start 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 0x7fe0480cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0480cf7f0 0x7fe0480d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe04815e570 0x7fe0480d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0480b84d0 con 0x7fe0480ba0b0 2024-03-30T13:09:54.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0480b9130 con 0x7fe0480cf7f0 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.099+0000 7fe04e081640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe0480b8650 con 0x7fe04815e570 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 0x7fe0480cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 0x7fe0480cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41520/0 (socket says 172.21.15.66:41520) 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 -- 172.21.15.66:0/3856812880 learned_addr learned my addr 172.21.15.66:0/3856812880 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe046ffd640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0480cf7f0 0x7fe0480d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe047fff640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe04815e570 0x7fe0480d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 -- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe04815e570 msgr2=0x7fe0480d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe04815e570 0x7fe0480d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 -- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0480cf7f0 msgr2=0x7fe0480d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0480cf7f0 0x7fe0480d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 -- 172.21.15.66:0/3856812880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0480cba20 con 0x7fe0480ba0b0 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe046ffd640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0480cf7f0 0x7fe0480d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe0477fe640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 0x7fe0480cb4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe038009f40 tx=0x7fe038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038043020 con 0x7fe0480ba0b0 2024-03-30T13:09:54.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03803e450 con 0x7fe0480ba0b0 2024-03-30T13:09:54.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe048148a90 con 0x7fe0480ba0b0 2024-03-30T13:09:54.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038009510 con 0x7fe0480ba0b0 2024-03-30T13:09:54.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.100+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe048146180 con 0x7fe0480ba0b0 2024-03-30T13:09:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.101+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe048075890 con 0x7fe0480ba0b0 2024-03-30T13:09:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.102+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe038046020 con 0x7fe0480ba0b0 2024-03-30T13:09:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.102+0000 7fe03ffff640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe02806c530 0x7fe02806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.102+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe0380ae060 con 0x7fe0480ba0b0 2024-03-30T13:09:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.102+0000 7fe046ffd640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe02806c530 0x7fe02806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.103+0000 7fe046ffd640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe02806c530 0x7fe02806e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe04815d5f0 tx=0x7fe030009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.106+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe03804d030 con 0x7fe0480ba0b0 2024-03-30T13:09:54.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.291+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe04815c240 con 0x7fe0480ba0b0 2024-03-30T13:09:54.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.291+0000 7fe03ffff640 1 -- 172.21.15.66:0/3856812880 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe03807bd80 con 0x7fe0480ba0b0 2024-03-30T13:09:54.296 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:09:54.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe02806c530 msgr2=0x7fe02806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe02806c530 0x7fe02806e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe04815d5f0 tx=0x7fe030009040 comp rx=0 tx=0).stop 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 msgr2=0x7fe0480cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 0x7fe0480cb4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe038009f40 tx=0x7fe038004a40 comp rx=0 tx=0).stop 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 shutdown_connections 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe02806c530 0x7fe02806e9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe04815e570 0x7fe0480d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0480cf7f0 0x7fe0480d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 --2- 172.21.15.66:0/3856812880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0480ba0b0 0x7fe0480cb4e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 >> 172.21.15.66:0/3856812880 conn(0x7fe0480676e0 msgr2=0x7fe0480d4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 shutdown_connections 2024-03-30T13:09:54.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.294+0000 7fe04e081640 1 -- 172.21.15.66:0/3856812880 wait complete. 2024-03-30T13:09:54.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:09:54.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.415+0000 7f0006ff4640 1 Processor -- start 2024-03-30T13:09:54.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.415+0000 7f0006ff4640 1 -- start start 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0006ff4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f00000d13d0 0x7f00000cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0006ff4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f00000b3760 0x7f00000d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0006ff4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00000d1cf0 con 0x7f00000d3ba0 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0006ff4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f00000cf860 con 0x7f00000d13d0 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0006ff4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f00000cfb40 con 0x7f00000b3760 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f00000d13d0 0x7f00000cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f00000d13d0 0x7f00000cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41526/0 (socket says 172.21.15.66:41526) 2024-03-30T13:09:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 -- 172.21.15.66:0/3642362312 learned_addr learned my addr 172.21.15.66:0/3642362312 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7efff7fff640 1 --2- 172.21.15.66:0/3642362312 >> v2:172.21.15.117:3300/0 conn(0x7f00000b3760 0x7f00000d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 -- 172.21.15.66:0/3642362312 >> v2:172.21.15.117:3300/0 conn(0x7f00000b3760 msgr2=0x7f00000d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 --2- 172.21.15.66:0/3642362312 >> v2:172.21.15.117:3300/0 conn(0x7f00000b3760 0x7f00000d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 -- 172.21.15.66:0/3642362312 >> v1:172.21.15.117:6789/0 conn(0x7f00000d3ba0 legacy=0x7f00000b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.416+0000 7f0004d90640 1 -- 172.21.15.66:0/3642362312 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00000d33a0 con 0x7f00000d13d0 2024-03-30T13:09:54.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.417+0000 7f0004d90640 1 --2- 172.21.15.66:0/3642362312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cecc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00000d0ce0 tx=0x7efffc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26b847997e2426b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.417+0000 7efff77fe640 1 -- 172.21.15.66:0/3642362312 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc034070 con 0x7f00000d13d0 2024-03-30T13:09:54.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.417+0000 7efff77fe640 1 -- 172.21.15.66:0/3642362312 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efffc030860 con 0x7f00000d13d0 2024-03-30T13:09:54.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.417+0000 7efff77fe640 1 -- 172.21.15.66:0/3642362312 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc030c30 con 0x7f00000d13d0 2024-03-30T13:09:54.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 -- 172.21.15.66:0/3642362312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 msgr2=0x7f00000cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 --2- 172.21.15.66:0/3642362312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cecc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00000d0ce0 tx=0x7efffc02b9a0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 -- 172.21.15.66:0/3642362312 shutdown_connections 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 --2- 172.21.15.66:0/3642362312 >> v2:172.21.15.117:3300/0 conn(0x7f00000b3760 0x7f00000d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 --2- 172.21.15.66:0/3642362312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cecc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 -- 172.21.15.66:0/3642362312 >> 172.21.15.66:0/3642362312 conn(0x7f0000067710 msgr2=0x7f0000067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 -- 172.21.15.66:0/3642362312 shutdown_connections 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.418+0000 7f0006ff4640 1 -- 172.21.15.66:0/3642362312 wait complete. 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.419+0000 7f0006ff4640 1 Processor -- start 2024-03-30T13:09:54.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.419+0000 7f0006ff4640 1 -- start start 2024-03-30T13:09:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0006ff4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000b3760 0x7f00000cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0006ff4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0006ff4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00000d3ba0 0x7f000015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0006ff4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f000015e930 con 0x7f00000d13d0 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0006ff4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00000d3090 con 0x7f00000b3760 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0006ff4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f000015eab0 con 0x7f00000d3ba0 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41532/0 (socket says 172.21.15.66:41532) 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0004d90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000b3760 0x7f00000cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 -- 172.21.15.66:0/2668339225 learned_addr learned my addr 172.21.15.66:0/2668339225 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0005591640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00000d3ba0 0x7f000015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 -- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00000d3ba0 msgr2=0x7f000015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00000d3ba0 0x7f000015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 -- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000b3760 msgr2=0x7f00000cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000b3760 0x7f00000cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 -- 172.21.15.66:0/2668339225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00000cb6c0 con 0x7f00000d13d0 2024-03-30T13:09:54.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7f0004d90640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000b3760 0x7f00000cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff7fff640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cdac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efff000e9e0 tx=0x7efff000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.420+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0017070 con 0x7f00000d13d0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.421+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff000c9d0 con 0x7f00000d13d0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.421+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00000b3ec0 con 0x7f00000d13d0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.421+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff000cdc0 con 0x7f00000d13d0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.421+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00000b4480 con 0x7f00000d13d0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.422+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0000075890 con 0x7f00000d13d0 2024-03-30T13:09:54.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.422+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efff001a070 con 0x7f00000d13d0 2024-03-30T13:09:54.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.422+0000 7efff57fa640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7effec06c530 0x7effec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.423+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efff008b980 con 0x7f00000d13d0 2024-03-30T13:09:54.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.423+0000 7f0004d90640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7effec06c530 0x7effec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.423+0000 7f0004d90640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7effec06c530 0x7effec06e9f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efffc036000 tx=0x7efffc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.427+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efff0059610 con 0x7f00000d13d0 2024-03-30T13:09:54.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.610+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f00000d52c0 con 0x7f00000d13d0 2024-03-30T13:09:54.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.610+0000 7efff57fa640 1 -- 172.21.15.66:0/2668339225 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7efff005d910 con 0x7f00000d13d0 2024-03-30T13:09:54.614 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7effec06c530 msgr2=0x7effec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7effec06c530 0x7effec06e9f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efffc036000 tx=0x7efffc008030 comp rx=0 tx=0).stop 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 msgr2=0x7f00000cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cdac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efff000e9e0 tx=0x7efff000eea0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 shutdown_connections 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7effec06c530 0x7effec06e9f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00000d3ba0 0x7f000015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00000d13d0 0x7f00000cdac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 --2- 172.21.15.66:0/2668339225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000b3760 0x7f00000cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 >> 172.21.15.66:0/2668339225 conn(0x7f0000067710 msgr2=0x7f00000c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 shutdown_connections 2024-03-30T13:09:54.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.612+0000 7f0006ff4640 1 -- 172.21.15.66:0/2668339225 wait complete. 2024-03-30T13:09:54.627 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:09:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.731+0000 7fa879c7e640 1 Processor -- start 2024-03-30T13:09:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.731+0000 7fa879c7e640 1 -- start start 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa879c7e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa87415e7a0 0x7fa8740cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa879c7e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8740d19c0 0x7fa8740b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa879c7e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8740d3120 con 0x7fa8740cec60 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa879c7e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa8740cfc40 con 0x7fa8740d19c0 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa879c7e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740d3400 con 0x7fa87415e7a0 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa873fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8740d19c0 0x7fa8740b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa873fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8740d19c0 0x7fa8740b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41536/0 (socket says 172.21.15.66:41536) 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa873fff640 1 -- 172.21.15.66:0/3106521523 learned_addr learned my addr 172.21.15.66:0/3106521523 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:54.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.732+0000 7fa8737fe640 1 --2- 172.21.15.66:0/3106521523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa87415e7a0 0x7fa8740cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa873fff640 1 -- 172.21.15.66:0/3106521523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa87415e7a0 msgr2=0x7fa8740cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa873fff640 1 --2- 172.21.15.66:0/3106521523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa87415e7a0 0x7fa8740cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa873fff640 1 -- 172.21.15.66:0/3106521523 >> v1:172.21.15.117:6789/0 conn(0x7fa8740cec60 legacy=0x7fa8740d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa8727fc640 1 -- 172.21.15.66:0/3106521523 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081015 0 0) 0x7fa8740d3120 con 0x7fa8740cec60 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa873fff640 1 -- 172.21.15.66:0/3106521523 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8740d36e0 con 0x7fa8740d19c0 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa873fff640 1 --2- 172.21.15.66:0/3106521523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740d19c0 0x7fa8740b75a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1248d1a30c535686 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.733+0000 7fa8727fc640 1 -- 172.21.15.66:0/3106521523 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c035020 con 0x7fa8740d19c0 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa8727fc640 1 -- 172.21.15.66:0/3106521523 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c031640 con 0x7fa8740d19c0 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa8727fc640 1 -- 172.21.15.66:0/3106521523 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c02d040 con 0x7fa8740d19c0 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 -- 172.21.15.66:0/3106521523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740d19c0 msgr2=0x7fa8740b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 --2- 172.21.15.66:0/3106521523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740d19c0 0x7fa8740b75a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02b8f0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 -- 172.21.15.66:0/3106521523 shutdown_connections 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 --2- 172.21.15.66:0/3106521523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740d19c0 0x7fa8740b75a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 --2- 172.21.15.66:0/3106521523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa87415e7a0 0x7fa8740cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 -- 172.21.15.66:0/3106521523 >> 172.21.15.66:0/3106521523 conn(0x7fa8740676b0 msgr2=0x7fa874067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 -- 172.21.15.66:0/3106521523 shutdown_connections 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.734+0000 7fa879c7e640 1 -- 172.21.15.66:0/3106521523 wait complete. 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.735+0000 7fa879c7e640 1 Processor -- start 2024-03-30T13:09:54.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.735+0000 7fa879c7e640 1 -- start start 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa879c7e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8740cec60 0x7fa8740ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa879c7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8740d19c0 0x7fa8740b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa879c7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 0x7fa8740c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa879c7e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740d3ee0 con 0x7fa8740b5750 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa879c7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740cf930 con 0x7fa8740d19c0 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa879c7e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740d4060 con 0x7fa8740cec60 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa873fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 0x7fa8740c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa873fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 0x7fa8740c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41538/0 (socket says 172.21.15.66:41538) 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa873fff640 1 -- 172.21.15.66:0/2582424621 learned_addr learned my addr 172.21.15.66:0/2582424621 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa872ffd640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8740d19c0 0x7fa8740b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.736+0000 7fa8737fe640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8740cec60 0x7fa8740ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa873fff640 1 -- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8740cec60 msgr2=0x7fa8740ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa873fff640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8740cec60 0x7fa8740ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa873fff640 1 -- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8740d19c0 msgr2=0x7fa8740b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa873fff640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8740d19c0 0x7fa8740b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa873fff640 1 -- 172.21.15.66:0/2582424621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8740cd3c0 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa872ffd640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8740d19c0 0x7fa8740b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa873fff640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 0x7fa8740c9c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034040 tx=0x7fa86c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c040070 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c004030 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8740cd6a0 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c035040 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.737+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8740cdc60 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.738+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa874075900 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.739+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa86c038020 con 0x7fa8740b5750 2024-03-30T13:09:54.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.739+0000 7fa86bfff640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa85406c530 0x7fa85406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.739+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa86c0ad110 con 0x7fa8740b5750 2024-03-30T13:09:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.739+0000 7fa8737fe640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa85406c530 0x7fa85406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:54.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.739+0000 7fa8737fe640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa85406c530 0x7fa85406e9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa864011340 tx=0x7fa86400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:54.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.743+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa86c07ae30 con 0x7fa8740b5750 2024-03-30T13:09:54.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.939+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa874158e70 con 0x7fa8740b5750 2024-03-30T13:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.939+0000 7fa86bfff640 1 -- 172.21.15.66:0/2582424621 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa86c07f130 con 0x7fa8740b5750 2024-03-30T13:09:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa85406c530 msgr2=0x7fa85406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa85406c530 0x7fa85406e9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa864011340 tx=0x7fa86400b040 comp rx=0 tx=0).stop 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 msgr2=0x7fa8740c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 0x7fa8740c9c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034040 tx=0x7fa86c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 shutdown_connections 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa85406c530 0x7fa85406e9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8740b5750 0x7fa8740c9c80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8740d19c0 0x7fa8740b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 --2- 172.21.15.66:0/2582424621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8740cec60 0x7fa8740ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 >> 172.21.15.66:0/2582424621 conn(0x7fa8740676b0 msgr2=0x7fa8740c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 shutdown_connections 2024-03-30T13:09:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:54.941+0000 7fa879c7e640 1 -- 172.21.15.66:0/2582424621 wait complete. 2024-03-30T13:09:54.955 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:09:54.955 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:09:54.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:09:55.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.115+0000 7f4800fa8640 1 Processor -- start 2024-03-30T13:09:55.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.115+0000 7f4800fa8640 1 -- start start 2024-03-30T13:09:55.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f4800fa8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f4800fa8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f47fc0cb580 0x7f47fc0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f4800fa8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47fc0724e0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f4800fa8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f47fc0730f0 con 0x7f47fc0cb580 2024-03-30T13:09:55.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f4800fa8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc0727c0 con 0x7f47fc15a960 2024-03-30T13:09:55.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f47fad9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f47fc0cb580 0x7f47fc0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f47fad9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f47fc0cb580 0x7f47fc0cddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41554/0 (socket says 172.21.15.66:41554) 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f47fad9d640 1 -- 172.21.15.66:0/3500405081 learned_addr learned my addr 172.21.15.66:0/3500405081 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.116+0000 7f47fa59c640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47fad9d640 1 -- 172.21.15.66:0/3500405081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 msgr2=0x7f47fc071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47fad9d640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47fad9d640 1 -- 172.21.15.66:0/3500405081 >> v1:172.21.15.117:6789/0 conn(0x7f47fc0cd3b0 legacy=0x7f47fc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47f959a640 1 -- 172.21.15.66:0/3500405081 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524199382 0 0) 0x7f47fc0724e0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47fad9d640 1 -- 172.21.15.66:0/3500405081 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc072aa0 con 0x7f47fc0cb580 2024-03-30T13:09:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47fa59c640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47fad9d640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cb580 0x7f47fc0cddd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ec13cc9c6004b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47f959a640 1 -- 172.21.15.66:0/3500405081 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec02cb00 con 0x7f47fc0cb580 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47f959a640 1 -- 172.21.15.66:0/3500405081 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec02cca0 con 0x7f47fc0cb580 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.117+0000 7f47f959a640 1 -- 172.21.15.66:0/3500405081 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec034610 con 0x7f47fc0cb580 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 -- 172.21.15.66:0/3500405081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cb580 msgr2=0x7f47fc0cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cb580 0x7f47fc0cddd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec02bf60 comp rx=0 tx=0).stop 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 -- 172.21.15.66:0/3500405081 shutdown_connections 2024-03-30T13:09:55.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cb580 0x7f47fc0cddd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 --2- 172.21.15.66:0/3500405081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 -- 172.21.15.66:0/3500405081 >> 172.21.15.66:0/3500405081 conn(0x7f47fc0b1c90 msgr2=0x7f47fc0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:55.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 -- 172.21.15.66:0/3500405081 shutdown_connections 2024-03-30T13:09:55.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.118+0000 7f4800fa8640 1 -- 172.21.15.66:0/3500405081 wait complete. 2024-03-30T13:09:55.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 Processor -- start 2024-03-30T13:09:55.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 -- start start 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47fc0cb580 0x7f47fc153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 0x7f47fc143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc071fb0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc15ad40 con 0x7f47fc0cb580 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f4800fa8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc072130 con 0x7f47fc15a960 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f47f9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 0x7f47fc143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f47f9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 0x7f47fc143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41558/0 (socket says 172.21.15.66:41558) 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f47fa59c640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47fc0cb580 0x7f47fc153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.120+0000 7f47f9d9b640 1 -- 172.21.15.66:0/2045373443 learned_addr learned my addr 172.21.15.66:0/2045373443 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47fad9d640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f9d9b640 1 -- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 msgr2=0x7f47fc141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f9d9b640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f9d9b640 1 -- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47fc0cb580 msgr2=0x7f47fc153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f9d9b640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47fc0cb580 0x7f47fc153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f9d9b640 1 -- 172.21.15.66:0/2045373443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc1441b0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f9d9b640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 0x7f47fc143a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c4b0 tx=0x7f47e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4017070 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e4005480 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47fc144490 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4003830 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.121+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47fc141650 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.128 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.122+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47fc075a10 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.123+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f47e401a070 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.123+0000 7f47f2ffd640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47dc06c530 0x7f47dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.123+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f47e408ade0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.123+0000 7f47fa59c640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47dc06c530 0x7f47dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.123+0000 7f47fa59c640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47dc06c530 0x7f47dc06e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47f400a570 tx=0x7f47f400a2a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.127+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f47e40589f0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.318+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47fc06e2e0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.319+0000 7f47f2ffd640 1 -- 172.21.15.66:0/2045373443 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f47e405ccf0 con 0x7f47fc0cd3b0 2024-03-30T13:09:55.323 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:09:55.323 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:09:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47dc06c530 msgr2=0x7f47dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47dc06c530 0x7f47dc06e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47f400a570 tx=0x7f47f400a2a0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 msgr2=0x7f47fc143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 0x7f47fc143a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c4b0 tx=0x7f47e400c970 comp rx=0 tx=0).stop 2024-03-30T13:09:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 shutdown_connections 2024-03-30T13:09:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47dc06c530 0x7f47dc06e9f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47fc15a960 0x7f47fc141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47fc0cd3b0 0x7f47fc143a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 --2- 172.21.15.66:0/2045373443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47fc0cb580 0x7f47fc153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.321+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 >> 172.21.15.66:0/2045373443 conn(0x7f47fc0b1c90 msgr2=0x7f47fc0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.322+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 shutdown_connections 2024-03-30T13:09:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.322+0000 7f4800fa8640 1 -- 172.21.15.66:0/2045373443 wait complete. 2024-03-30T13:09:55.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:09:55.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:09:55.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.444+0000 7fcd63c7b640 1 Processor -- start 2024-03-30T13:09:55.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.444+0000 7fcd63c7b640 1 -- start start 2024-03-30T13:09:55.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd63c7b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcd5c0d13d0 0x7fcd5c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd63c7b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd5c0d3ba0 0x7fcd5c0b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd63c7b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd5c0cf860 con 0x7fcd5c0b3760 2024-03-30T13:09:55.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd63c7b640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcd5c0cfb40 con 0x7fcd5c0d3ba0 2024-03-30T13:09:55.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd63c7b640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fcd5c0d33a0 con 0x7fcd5c0d13d0 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd62218640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd5c0d3ba0 0x7fcd5c0b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd62218640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd5c0d3ba0 0x7fcd5c0b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41568/0 (socket says 172.21.15.66:41568) 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd62218640 1 -- 172.21.15.66:0/2397083490 learned_addr learned my addr 172.21.15.66:0/2397083490 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.445+0000 7fcd61a17640 1 --2- 172.21.15.66:0/2397083490 >> v2:172.21.15.117:3300/0 conn(0x7fcd5c0d13d0 0x7fcd5c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.446+0000 7fcd62218640 1 -- 172.21.15.66:0/2397083490 >> v2:172.21.15.117:3300/0 conn(0x7fcd5c0d13d0 msgr2=0x7fcd5c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.446+0000 7fcd62218640 1 --2- 172.21.15.66:0/2397083490 >> v2:172.21.15.117:3300/0 conn(0x7fcd5c0d13d0 0x7fcd5c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.446+0000 7fcd62218640 1 -- 172.21.15.66:0/2397083490 >> v1:172.21.15.117:6789/0 conn(0x7fcd5c0b3760 legacy=0x7fcd5c0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.446+0000 7fcd62218640 1 -- 172.21.15.66:0/2397083490 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c0d3680 con 0x7fcd5c0d3ba0 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.446+0000 7fcd62218640 1 --2- 172.21.15.66:0/2397083490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b7740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c06c8c0 tx=0x7fcd4c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b94a33028a43f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd60a15640 1 -- 172.21.15.66:0/2397083490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c02cb20 con 0x7fcd5c0d3ba0 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd60a15640 1 -- 172.21.15.66:0/2397083490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c02ccc0 con 0x7fcd5c0d3ba0 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd60a15640 1 -- 172.21.15.66:0/2397083490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c034610 con 0x7fcd5c0d3ba0 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 -- 172.21.15.66:0/2397083490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0d3ba0 msgr2=0x7fcd5c0b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/2397083490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b7740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c06c8c0 tx=0x7fcd4c02bf80 comp rx=0 tx=0).stop 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 -- 172.21.15.66:0/2397083490 shutdown_connections 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/2397083490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b7740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/2397083490 >> v2:172.21.15.117:3300/0 conn(0x7fcd5c0d13d0 0x7fcd5c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 -- 172.21.15.66:0/2397083490 >> 172.21.15.66:0/2397083490 conn(0x7fcd5c067710 msgr2=0x7fcd5c067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:55.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 -- 172.21.15.66:0/2397083490 shutdown_connections 2024-03-30T13:09:55.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.447+0000 7fcd63c7b640 1 -- 172.21.15.66:0/2397083490 wait complete. 2024-03-30T13:09:55.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.448+0000 7fcd63c7b640 1 Processor -- start 2024-03-30T13:09:55.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.448+0000 7fcd63c7b640 1 -- start start 2024-03-30T13:09:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd63c7b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 0x7fcd5c1ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd63c7b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd5c0d13d0 0x7fcd5c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd63c7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd63c7b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c15eab0 con 0x7fcd5c0b3760 2024-03-30T13:09:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd63c7b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c15e930 con 0x7fcd5c0d3ba0 2024-03-30T13:09:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd63c7b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c0cf4c0 con 0x7fcd5c0d13d0 2024-03-30T13:09:55.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd61a17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 0x7fcd5c1ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd61a17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 0x7fcd5c1ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41574/0 (socket says 172.21.15.66:41574) 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd61a17640 1 -- 172.21.15.66:0/3174380618 learned_addr learned my addr 172.21.15.66:0/3174380618 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd62218640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.449+0000 7fcd61216640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd5c0d13d0 0x7fcd5c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd61a17640 1 -- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd5c0d13d0 msgr2=0x7fcd5c0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd61a17640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd5c0d13d0 0x7fcd5c0b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd61a17640 1 -- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd5c0d3ba0 msgr2=0x7fcd5c0b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd61a17640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd61a17640 1 -- 172.21.15.66:0/3174380618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c1e8ee0 con 0x7fcd5c0b3760 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd61a17640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 0x7fcd5c1ea9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400e9b0 tx=0x7fcd5400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54017070 con 0x7fcd5c0b3760 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd5400c9a0 con 0x7fcd5c0b3760 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400cd90 con 0x7fcd5c0b3760 2024-03-30T13:09:55.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c1e91c0 con 0x7fcd5c0b3760 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.450+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1e9780 con 0x7fcd5c0b3760 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.451+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c075890 con 0x7fcd5c0b3760 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.451+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd5401a070 con 0x7fcd5c0b3760 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.452+0000 7fcd5a7fc640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcd4806c560 0x7fcd4806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.452+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcd5408bc30 con 0x7fcd5c0b3760 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.452+0000 7fcd61216640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcd4806c560 0x7fcd4806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.452+0000 7fcd61216640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcd4806c560 0x7fcd4806ea20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0ce780 tx=0x7fcd50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.456+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcd54059950 con 0x7fcd5c0b3760 2024-03-30T13:09:55.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.645+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd5c161f20 con 0x7fcd5c0b3760 2024-03-30T13:09:55.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.645+0000 7fcd5a7fc640 1 -- 172.21.15.66:0/3174380618 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fcd5405dc50 con 0x7fcd5c0b3760 2024-03-30T13:09:55.649 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcd4806c560 msgr2=0x7fcd4806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcd4806c560 0x7fcd4806ea20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0ce780 tx=0x7fcd50009040 comp rx=0 tx=0).stop 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 msgr2=0x7fcd5c1ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 0x7fcd5c1ea9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400e9b0 tx=0x7fcd5400ee70 comp rx=0 tx=0).stop 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 shutdown_connections 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcd4806c560 0x7fcd4806ea20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd5c0d3ba0 0x7fcd5c0b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd5c0d13d0 0x7fcd5c0b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 --2- 172.21.15.66:0/3174380618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd5c0b3760 0x7fcd5c1ea9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 >> 172.21.15.66:0/3174380618 conn(0x7fcd5c067710 msgr2=0x7fcd5c15ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:55.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 shutdown_connections 2024-03-30T13:09:55.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.647+0000 7fcd63c7b640 1 -- 172.21.15.66:0/3174380618 wait complete. 2024-03-30T13:09:55.661 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:09:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.767+0000 7fd127b31640 1 Processor -- start 2024-03-30T13:09:55.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.767+0000 7fd127b31640 1 -- start start 2024-03-30T13:09:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd127b31640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1200b5dc0 0x7fd1200b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd127b31640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1200b66e0 con 0x7fd1200b73d0 2024-03-30T13:09:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd127b31640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1200d2ff0 con 0x7fd1200ba090 2024-03-30T13:09:55.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd127b31640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd1200d2d90 con 0x7fd1200b5dc0 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd1258cd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1200b5dc0 0x7fd1200b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd1260ce640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd1200b73d0 0x7fd1200b7800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46284/0 (socket says 172.21.15.66:46284) 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.768+0000 7fd1260ce640 1 -- 172.21.15.66:0/201926691 learned_addr learned my addr 172.21.15.66:0/201926691 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.769+0000 7fd1258cd640 1 -- 172.21.15.66:0/201926691 >> v1:172.21.15.117:6789/0 conn(0x7fd1200ba090 legacy=0x7fd1200d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.769+0000 7fd1258cd640 1 -- 172.21.15.66:0/201926691 >> v1:172.21.15.66:6789/0 conn(0x7fd1200b73d0 legacy=0x7fd1200b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.769+0000 7fd1248cb640 1 -- 172.21.15.66:0/201926691 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158939029 0 0) 0x7fd1200d2ff0 con 0x7fd1200ba090 2024-03-30T13:09:55.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.769+0000 7fd1258cd640 1 -- 172.21.15.66:0/201926691 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200d32d0 con 0x7fd1200b5dc0 2024-03-30T13:09:55.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.769+0000 7fd1258cd640 1 --2- 172.21.15.66:0/201926691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b5dc0 0x7fd1200b61a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd1180097a0 tx=0x7fd11802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39efd3d1f8b86f6e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd1248cb640 1 -- 172.21.15.66:0/201926691 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118037070 con 0x7fd1200b5dc0 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd1248cb640 1 -- 172.21.15.66:0/201926691 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118030710 con 0x7fd1200b5dc0 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd1248cb640 1 -- 172.21.15.66:0/201926691 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118030ae0 con 0x7fd1200b5dc0 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 -- 172.21.15.66:0/201926691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b5dc0 msgr2=0x7fd1200b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 --2- 172.21.15.66:0/201926691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b5dc0 0x7fd1200b61a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd1180097a0 tx=0x7fd11802b790 comp rx=0 tx=0).stop 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 -- 172.21.15.66:0/201926691 shutdown_connections 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 --2- 172.21.15.66:0/201926691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b5dc0 0x7fd1200b61a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 -- 172.21.15.66:0/201926691 >> 172.21.15.66:0/201926691 conn(0x7fd1200678f0 msgr2=0x7fd1200d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 -- 172.21.15.66:0/201926691 shutdown_connections 2024-03-30T13:09:55.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.770+0000 7fd127b31640 1 -- 172.21.15.66:0/201926691 wait complete. 2024-03-30T13:09:55.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.771+0000 7fd127b31640 1 Processor -- start 2024-03-30T13:09:55.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 -- start start 2024-03-30T13:09:55.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b5dc0 0x7fd120155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 0x7fd12015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200ba090 0x7fd120158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200d3b70 con 0x7fd1200b73d0 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200d3890 con 0x7fd1200ba090 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd127b31640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd1200d4090 con 0x7fd1200b5dc0 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd1250cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 0x7fd12015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd1250cc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 0x7fd12015e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41578/0 (socket says 172.21.15.66:41578) 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd1250cc640 1 -- 172.21.15.66:0/4035551240 learned_addr learned my addr 172.21.15.66:0/4035551240 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd1260ce640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200ba090 0x7fd120158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.772+0000 7fd1258cd640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b5dc0 0x7fd120155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1250cc640 1 -- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b5dc0 msgr2=0x7fd120155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1250cc640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b5dc0 0x7fd120155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1250cc640 1 -- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200ba090 msgr2=0x7fd120158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1250cc640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200ba090 0x7fd120158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1250cc640 1 -- 172.21.15.66:0/4035551240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200cdff0 con 0x7fd1200b73d0 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1260ce640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200ba090 0x7fd120158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd1250cc640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 0x7fd12015e880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c410 tx=0x7fd11400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114017070 con 0x7fd1200b73d0 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1140035f0 con 0x7fd1200b73d0 2024-03-30T13:09:55.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1200cb5b0 con 0x7fd1200b73d0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140039e0 con 0x7fd1200b73d0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.773+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1200cbb70 con 0x7fd1200b73d0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.774+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd120075890 con 0x7fd1200b73d0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.775+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd11401a070 con 0x7fd1200b73d0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.775+0000 7fd11e7fc640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10c06c530 0x7fd10c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.775+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd11408ad10 con 0x7fd1200b73d0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.775+0000 7fd1258cd640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10c06c530 0x7fd10c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:55.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.776+0000 7fd1258cd640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10c06c530 0x7fd10c06e9f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd118034000 tx=0x7fd11802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:55.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.780+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd1200cdff0 con 0x7fd1200b73d0 2024-03-30T13:09:55.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.963+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd1200cfb80 con 0x7fd1200b73d0 2024-03-30T13:09:55.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.964+0000 7fd11e7fc640 1 -- 172.21.15.66:0/4035551240 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd114058a30 con 0x7fd1200b73d0 2024-03-30T13:09:55.968 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10c06c530 msgr2=0x7fd10c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10c06c530 0x7fd10c06e9f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd118034000 tx=0x7fd11802f000 comp rx=0 tx=0).stop 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 msgr2=0x7fd12015e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 0x7fd12015e880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c410 tx=0x7fd11400c8d0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 shutdown_connections 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd10c06c530 0x7fd10c06e9f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1200ba090 0x7fd120158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1200b73d0 0x7fd12015e880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 --2- 172.21.15.66:0/4035551240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1200b5dc0 0x7fd120155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:55.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 >> 172.21.15.66:0/4035551240 conn(0x7fd1200678f0 msgr2=0x7fd1200ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:55.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 shutdown_connections 2024-03-30T13:09:55.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:55.966+0000 7fd127b31640 1 -- 172.21.15.66:0/4035551240 wait complete. 2024-03-30T13:09:55.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:09:56.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.087+0000 7f98abb6d640 1 Processor -- start 2024-03-30T13:09:56.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.087+0000 7f98abb6d640 1 -- start start 2024-03-30T13:09:56.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98abb6d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98a40d13d0 0x7f98a40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98abb6d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98a40b3760 0x7f98a40d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98abb6d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a40d1cf0 con 0x7f98a40d3ba0 2024-03-30T13:09:56.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98abb6d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40cf860 con 0x7f98a40d13d0 2024-03-30T13:09:56.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98abb6d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40cfb40 con 0x7f98a40b3760 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98a40d13d0 0x7f98a40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98a40d13d0 0x7f98a40cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41590/0 (socket says 172.21.15.66:41590) 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 -- 172.21.15.66:0/2023241402 learned_addr learned my addr 172.21.15.66:0/2023241402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9108640 1 --2- 172.21.15.66:0/2023241402 >> v2:172.21.15.117:3300/0 conn(0x7f98a40b3760 0x7f98a40d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 -- 172.21.15.66:0/2023241402 >> v2:172.21.15.117:3300/0 conn(0x7f98a40b3760 msgr2=0x7f98a40d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 --2- 172.21.15.66:0/2023241402 >> v2:172.21.15.117:3300/0 conn(0x7f98a40b3760 0x7f98a40d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 -- 172.21.15.66:0/2023241402 >> v1:172.21.15.117:6789/0 conn(0x7f98a40d3ba0 legacy=0x7f98a40b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.088+0000 7f98a9909640 1 -- 172.21.15.66:0/2023241402 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40d33a0 con 0x7f98a40d13d0 2024-03-30T13:09:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.089+0000 7f98a9909640 1 --2- 172.21.15.66:0/2023241402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d13d0 0x7f98a40cecc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f98a40d0ce0 tx=0x7f98a002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e71ddfc31b5d9e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.089+0000 7f98a8907640 1 -- 172.21.15.66:0/2023241402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0034070 con 0x7f98a40d13d0 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.089+0000 7f98a8907640 1 -- 172.21.15.66:0/2023241402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a0030860 con 0x7f98a40d13d0 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.089+0000 7f98a8907640 1 -- 172.21.15.66:0/2023241402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0030c30 con 0x7f98a40d13d0 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 -- 172.21.15.66:0/2023241402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d13d0 msgr2=0x7f98a40cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 --2- 172.21.15.66:0/2023241402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d13d0 0x7f98a40cecc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f98a40d0ce0 tx=0x7f98a002b9a0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 -- 172.21.15.66:0/2023241402 shutdown_connections 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 --2- 172.21.15.66:0/2023241402 >> v2:172.21.15.117:3300/0 conn(0x7f98a40b3760 0x7f98a40d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 --2- 172.21.15.66:0/2023241402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d13d0 0x7f98a40cecc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 -- 172.21.15.66:0/2023241402 >> 172.21.15.66:0/2023241402 conn(0x7f98a4067710 msgr2=0x7f98a4067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 -- 172.21.15.66:0/2023241402 shutdown_connections 2024-03-30T13:09:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.090+0000 7f98abb6d640 1 -- 172.21.15.66:0/2023241402 wait complete. 2024-03-30T13:09:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.091+0000 7f98abb6d640 1 Processor -- start 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.091+0000 7f98abb6d640 1 -- start start 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98abb6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98a40b3760 0x7f98a4152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98abb6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a40d13d0 0x7f98a40b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98abb6d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 0x7f98a40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98abb6d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98a415e930 con 0x7f98a40d3ba0 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98abb6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40d3090 con 0x7f98a40d13d0 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98abb6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f98a415eab0 con 0x7f98a40b3760 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 0x7f98a40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98a9108640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a40d13d0 0x7f98a40b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 0x7f98a40cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41604/0 (socket says 172.21.15.66:41604) 2024-03-30T13:09:56.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 -- 172.21.15.66:0/364481194 learned_addr learned my addr 172.21.15.66:0/364481194 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98a9909640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98a40b3760 0x7f98a4152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 -- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98a40b3760 msgr2=0x7f98a4152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98a40b3760 0x7f98a4152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 -- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a40d13d0 msgr2=0x7f98a40b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a40d13d0 0x7f98a40b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98aa10a640 1 -- 172.21.15.66:0/364481194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40b4bb0 con 0x7f98a40d3ba0 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.092+0000 7f98a9108640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a40d13d0 0x7f98a40b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.093+0000 7f98aa10a640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 0x7f98a40cd600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c6b0 tx=0x7f989c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.093+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c003b70 con 0x7f98a40d3ba0 2024-03-30T13:09:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.093+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c003d30 con 0x7f98a40d3ba0 2024-03-30T13:09:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.093+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a40b4e90 con 0x7f98a40d3ba0 2024-03-30T13:09:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.093+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c005310 con 0x7f98a40d3ba0 2024-03-30T13:09:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.093+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a40b5450 con 0x7f98a40d3ba0 2024-03-30T13:09:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.094+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4075890 con 0x7f98a40d3ba0 2024-03-30T13:09:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.094+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f989c0054b0 con 0x7f98a40d3ba0 2024-03-30T13:09:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.095+0000 7f989a7fc640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f988806c530 0x7f988806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.095+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f989c08b720 con 0x7f98a40d3ba0 2024-03-30T13:09:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.095+0000 7f98a9909640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f988806c530 0x7f988806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.095+0000 7f98a9909640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f988806c530 0x7f988806e9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98a0036000 tx=0x7f98a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.099+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f989c0593b0 con 0x7f98a40d3ba0 2024-03-30T13:09:56.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.284+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f98a4161f20 con 0x7f98a40d3ba0 2024-03-30T13:09:56.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.284+0000 7f989a7fc640 1 -- 172.21.15.66:0/364481194 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f989c05d6b0 con 0x7f98a40d3ba0 2024-03-30T13:09:56.288 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f988806c530 msgr2=0x7f988806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f988806c530 0x7f988806e9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98a0036000 tx=0x7f98a0008030 comp rx=0 tx=0).stop 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 msgr2=0x7f98a40cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 0x7f98a40cd600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c6b0 tx=0x7f989c00cb70 comp rx=0 tx=0).stop 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 shutdown_connections 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f988806c530 0x7f988806e9f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a40d3ba0 0x7f98a40cd600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a40d13d0 0x7f98a40b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 --2- 172.21.15.66:0/364481194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98a40b3760 0x7f98a4152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 >> 172.21.15.66:0/364481194 conn(0x7f98a4067710 msgr2=0x7f98a4160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:56.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 shutdown_connections 2024-03-30T13:09:56.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.286+0000 7f98abb6d640 1 -- 172.21.15.66:0/364481194 wait complete. 2024-03-30T13:09:56.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:09:56.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.406+0000 7fd07bbf4640 1 Processor -- start 2024-03-30T13:09:56.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.406+0000 7fd07bbf4640 1 -- start start 2024-03-30T13:09:56.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07bbf4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd07415c000 0x7fd0740d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07bbf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740b9cf0 0x7fd0740d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07bbf4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0740b3c30 con 0x7fd0740ba150 2024-03-30T13:09:56.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07bbf4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd074162110 con 0x7fd07415c000 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07bbf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd0741623d0 con 0x7fd0740b9cf0 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd079990640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd07415c000 0x7fd0740d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07a191640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd0740ba150 0x7fd07415edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:46290/0 (socket says 172.21.15.66:46290) 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07a191640 1 -- 172.21.15.66:0/786848826 learned_addr learned my addr 172.21.15.66:0/786848826 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.407+0000 7fd07918f640 1 --2- 172.21.15.66:0/786848826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740b9cf0 0x7fd0740d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd079990640 1 -- 172.21.15.66:0/786848826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740b9cf0 msgr2=0x7fd0740d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd079990640 1 --2- 172.21.15.66:0/786848826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740b9cf0 0x7fd0740d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd079990640 1 -- 172.21.15.66:0/786848826 >> v1:172.21.15.66:6789/0 conn(0x7fd0740ba150 legacy=0x7fd07415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd07898e640 1 -- 172.21.15.66:0/786848826 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722931802 0 0) 0x7fd0740b3c30 con 0x7fd0740ba150 2024-03-30T13:09:56.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd079990640 1 -- 172.21.15.66:0/786848826 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741626b0 con 0x7fd07415c000 2024-03-30T13:09:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd079990640 1 --2- 172.21.15.66:0/786848826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740d27a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097a0 tx=0x7fd06c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8659a3ec34cd7b92 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.408+0000 7fd07898e640 1 -- 172.21.15.66:0/786848826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0315a0 con 0x7fd07415c000 2024-03-30T13:09:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07898e640 1 -- 172.21.15.66:0/786848826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c037070 con 0x7fd07415c000 2024-03-30T13:09:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07898e640 1 -- 172.21.15.66:0/786848826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c031960 con 0x7fd07415c000 2024-03-30T13:09:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 -- 172.21.15.66:0/786848826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 msgr2=0x7fd0740d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/786848826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740d27a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097a0 tx=0x7fd06c02b840 comp rx=0 tx=0).stop 2024-03-30T13:09:56.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 -- 172.21.15.66:0/786848826 shutdown_connections 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/786848826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740b9cf0 0x7fd0740d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/786848826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740d27a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 -- 172.21.15.66:0/786848826 >> 172.21.15.66:0/786848826 conn(0x7fd0740676e0 msgr2=0x7fd074067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 -- 172.21.15.66:0/786848826 shutdown_connections 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.409+0000 7fd07bbf4640 1 -- 172.21.15.66:0/786848826 wait complete. 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.410+0000 7fd07bbf4640 1 Processor -- start 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.410+0000 7fd07bbf4640 1 -- start start 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07bbf4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0740b9cf0 0x7fd0740ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07bbf4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740ba150 0x7fd0740b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07bbf4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07bbf4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd07415c3e0 con 0x7fd07415c000 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07bbf4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd074162c70 con 0x7fd0740b9cf0 2024-03-30T13:09:56.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07bbf4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd07415c560 con 0x7fd0740ba150 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07a191640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07918f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740ba150 0x7fd0740b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd079990640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0740b9cf0 0x7fd0740ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07a191640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41620/0 (socket says 172.21.15.66:41620) 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.411+0000 7fd07a191640 1 -- 172.21.15.66:0/1955256006 learned_addr learned my addr 172.21.15.66:0/1955256006 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07a191640 1 -- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740ba150 msgr2=0x7fd0740b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07a191640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740ba150 0x7fd0740b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07a191640 1 -- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0740b9cf0 msgr2=0x7fd0740ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07a191640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0740b9cf0 0x7fd0740ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07a191640 1 -- 172.21.15.66:0/1955256006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0740c9930 con 0x7fd07415c000 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07918f640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740ba150 0x7fd0740b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07a191640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740c8910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c780 tx=0x7fd06400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064018070 con 0x7fd07415c000 2024-03-30T13:09:56.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06400a640 con 0x7fd07415c000 2024-03-30T13:09:56.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0740c9c10 con 0x7fd07415c000 2024-03-30T13:09:56.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06400aab0 con 0x7fd07415c000 2024-03-30T13:09:56.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.412+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0740ca1d0 con 0x7fd07415c000 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.413+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd074075890 con 0x7fd07415c000 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.413+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd06401b070 con 0x7fd07415c000 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.414+0000 7fd0727fc640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd06006c560 0x7fd06006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.414+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd06408af50 con 0x7fd07415c000 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.414+0000 7fd079990640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd06006c560 0x7fd06006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.414+0000 7fd079990640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd06006c560 0x7fd06006ea20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036000 tx=0x7fd06c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.418+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd064058be0 con 0x7fd07415c000 2024-03-30T13:09:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.602+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0740d2070 con 0x7fd07415c000 2024-03-30T13:09:56.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.603+0000 7fd0727fc640 1 -- 172.21.15.66:0/1955256006 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd06405cee0 con 0x7fd07415c000 2024-03-30T13:09:56.606 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd06006c560 msgr2=0x7fd06006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd06006c560 0x7fd06006ea20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036000 tx=0x7fd06c03d040 comp rx=0 tx=0).stop 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 msgr2=0x7fd0740c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740c8910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c780 tx=0x7fd06400cc40 comp rx=0 tx=0).stop 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 shutdown_connections 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd06006c560 0x7fd06006ea20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd07415c000 0x7fd0740c8910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0740ba150 0x7fd0740b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 --2- 172.21.15.66:0/1955256006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0740b9cf0 0x7fd0740ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 >> 172.21.15.66:0/1955256006 conn(0x7fd0740676e0 msgr2=0x7fd0740d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:56.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 shutdown_connections 2024-03-30T13:09:56.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.605+0000 7fd07bbf4640 1 -- 172.21.15.66:0/1955256006 wait complete. 2024-03-30T13:09:56.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:09:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.723+0000 7f757ede2640 1 Processor -- start 2024-03-30T13:09:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.723+0000 7f757ede2640 1 -- start start 2024-03-30T13:09:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757ede2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75780d0560 0x7f7578162310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757ede2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75780d1f20 0x7f75780b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757ede2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75781628f0 con 0x7f757815e770 2024-03-30T13:09:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757ede2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f75780b5200 con 0x7f75780d1f20 2024-03-30T13:09:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757ede2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7578162bd0 con 0x7f75780d0560 2024-03-30T13:09:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757d37f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75780d1f20 0x7f75780b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757cb7e640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f757815e770 0x7f75780cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:37972/0 (socket says 172.21.15.66:37972) 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757cb7e640 1 -- 172.21.15.66:0/3198796199 learned_addr learned my addr 172.21.15.66:0/3198796199 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f7577fff640 1 --2- 172.21.15.66:0/3198796199 >> v2:172.21.15.117:3300/0 conn(0x7f75780d0560 0x7f7578162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f75777fe640 1 -- 172.21.15.66:0/3198796199 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727164140 0 0) 0x7f75781628f0 con 0x7f757815e770 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f75777fe640 1 -- 172.21.15.66:0/3198796199 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7560003520 con 0x7f757815e770 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757d37f640 1 -- 172.21.15.66:0/3198796199 >> v2:172.21.15.117:3300/0 conn(0x7f75780d0560 msgr2=0x7f7578162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757d37f640 1 --2- 172.21.15.66:0/3198796199 >> v2:172.21.15.117:3300/0 conn(0x7f75780d0560 0x7f7578162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757d37f640 1 -- 172.21.15.66:0/3198796199 >> v1:172.21.15.117:6789/0 conn(0x7f757815e770 legacy=0x7f75780cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.724+0000 7f757d37f640 1 -- 172.21.15.66:0/3198796199 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7578162eb0 con 0x7f75780d1f20 2024-03-30T13:09:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.725+0000 7f757d37f640 1 --2- 172.21.15.66:0/3198796199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f75780b5690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f75780711e0 tx=0x7f757002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b02091a59419bb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.725+0000 7f75777fe640 1 -- 172.21.15.66:0/3198796199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570039070 con 0x7f75780d1f20 2024-03-30T13:09:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.725+0000 7f75777fe640 1 -- 172.21.15.66:0/3198796199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757002cc80 con 0x7f75780d1f20 2024-03-30T13:09:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.725+0000 7f75777fe640 1 -- 172.21.15.66:0/3198796199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570034660 con 0x7f75780d1f20 2024-03-30T13:09:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 -- 172.21.15.66:0/3198796199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 msgr2=0x7f75780b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 --2- 172.21.15.66:0/3198796199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f75780b5690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f75780711e0 tx=0x7f757002bed0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 -- 172.21.15.66:0/3198796199 shutdown_connections 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 --2- 172.21.15.66:0/3198796199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f75780b5690 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 --2- 172.21.15.66:0/3198796199 >> v2:172.21.15.117:3300/0 conn(0x7f75780d0560 0x7f7578162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 -- 172.21.15.66:0/3198796199 >> 172.21.15.66:0/3198796199 conn(0x7f757806f8f0 msgr2=0x7f757806fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 -- 172.21.15.66:0/3198796199 shutdown_connections 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.726+0000 7f757ede2640 1 -- 172.21.15.66:0/3198796199 wait complete. 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.727+0000 7f757ede2640 1 Processor -- start 2024-03-30T13:09:56.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.727+0000 7f757ede2640 1 -- start start 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757ede2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75780d0560 0x7f75780cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757ede2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f757814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757ede2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f757815e770 0x7f75780c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757ede2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f757806c790 con 0x7f75780d1f20 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757ede2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f757815eb50 con 0x7f75780d0560 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757ede2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f757806c910 con 0x7f757815e770 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f757814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f757814a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41628/0 (socket says 172.21.15.66:41628) 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 -- 172.21.15.66:0/832097926 learned_addr learned my addr 172.21.15.66:0/832097926 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757cb7e640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75780d0560 0x7f75780cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f757d37f640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f757815e770 0x7f75780c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 -- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f757815e770 msgr2=0x7f75780c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f757815e770 0x7f75780c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 -- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75780d0560 msgr2=0x7f75780cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75780d0560 0x7f75780cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 -- 172.21.15.66:0/832097926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75780cdec0 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.728+0000 7f7577fff640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f757814a610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e930 tx=0x7f756c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.729+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c017070 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.729+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c00ca00 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.729+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75780ce1a0 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.729+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c00cdb0 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.729+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75780ce760 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.730+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578075900 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.730+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f756c01a070 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.731+0000 7f75757fa640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f756406c560 0x7f756406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.731+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f756c08bc60 con 0x7f75780d1f20 2024-03-30T13:09:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.731+0000 7f757cb7e640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f756406c560 0x7f756406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:09:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.731+0000 7f757cb7e640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f756406c560 0x7f756406ea20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7568005880 tx=0x7f7568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:09:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.735+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f756c059a40 con 0x7f75780d1f20 2024-03-30T13:09:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.932+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f75780b8200 con 0x7f75780d1f20 2024-03-30T13:09:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.933+0000 7f75757fa640 1 -- 172.21.15.66:0/832097926 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f756c05dd40 con 0x7f75780d1f20 2024-03-30T13:09:56.938 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:09:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f756406c560 msgr2=0x7f756406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f756406c560 0x7f756406ea20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7568005880 tx=0x7f7568009040 comp rx=0 tx=0).stop 2024-03-30T13:09:56.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 msgr2=0x7f757814a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f757814a610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e930 tx=0x7f756c00edf0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 shutdown_connections 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f756406c560 0x7f756406ea20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f757815e770 0x7f75780c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75780d1f20 0x7f757814a610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 --2- 172.21.15.66:0/832097926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75780d0560 0x7f75780cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 >> 172.21.15.66:0/832097926 conn(0x7f757806f8f0 msgr2=0x7f757815ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 shutdown_connections 2024-03-30T13:09:56.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:09:56.935+0000 7f757ede2640 1 -- 172.21.15.66:0/832097926 wait complete. 2024-03-30T13:09:56.950 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-03-30T13:09:56.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:09:56.950 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:09:56.979 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[44257]: fuse finished with error 0 and tester_r 0 2024-03-30T13:09:56.981 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:10:03.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:10:03.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:03.083 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-30T13:10:03.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:03.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.268+0000 7f03f5f6d640 1 Processor -- start 2024-03-30T13:10:03.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.268+0000 7f03f5f6d640 1 -- start start 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03f5f6d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f03f0072230 0x7f03f006bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03f5f6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f006db10 0x7f03f006def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03f5f6d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f006c4d0 con 0x7f03f0072610 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03f5f6d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f03f0158f10 con 0x7f03f0072230 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03f5f6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f03f015ef70 con 0x7f03f006db10 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03eeffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f03f0072610 0x7f03f015f400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53770/0 (socket says 172.21.15.66:53770) 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03eeffd640 1 -- 172.21.15.66:0/318170875 learned_addr learned my addr 172.21.15.66:0/318170875 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03ef7fe640 1 --2- 172.21.15.66:0/318170875 >> v2:172.21.15.117:3300/0 conn(0x7f03f0072230 0x7f03f006bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03effff640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f006db10 0x7f03f006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03ee7fc640 1 -- 172.21.15.66:0/318170875 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278959821 0 0) 0x7f03e4001110 con 0x7f03f0072610 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.269+0000 7f03ee7fc640 1 -- 172.21.15.66:0/318170875 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03d8003520 con 0x7f03f0072610 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.270+0000 7f03ef7fe640 1 -- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f006db10 msgr2=0x7f03f006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:03.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.270+0000 7f03ef7fe640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f006db10 0x7f03f006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.270+0000 7f03ef7fe640 1 -- 172.21.15.66:0/318170875 >> v1:172.21.15.66:6789/0 conn(0x7f03f0072610 legacy=0x7f03f015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.270+0000 7f03ef7fe640 1 -- 172.21.15.66:0/318170875 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f00ce840 con 0x7f03f0072230 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.270+0000 7f03effff640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f006db10 0x7f03f006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.270+0000 7f03ef7fe640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072230 0x7f03f006bf90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f03e00097a0 tx=0x7f03e002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b9c0228d047b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03ee7fc640 1 -- 172.21.15.66:0/318170875 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0035020 con 0x7f03f0072230 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03ee7fc640 1 -- 172.21.15.66:0/318170875 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e0031830 con 0x7f03f0072230 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03ee7fc640 1 -- 172.21.15.66:0/318170875 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0031c00 con 0x7f03f0072230 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 -- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072230 msgr2=0x7f03f006bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072230 0x7f03f006bf90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f03e00097a0 tx=0x7f03e002b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 -- 172.21.15.66:0/318170875 shutdown_connections 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f006db10 0x7f03f006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/318170875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072230 0x7f03f006bf90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 -- 172.21.15.66:0/318170875 >> 172.21.15.66:0/318170875 conn(0x7f03f00d0cb0 msgr2=0x7f03f00d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:03.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 -- 172.21.15.66:0/318170875 shutdown_connections 2024-03-30T13:10:03.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.271+0000 7f03f5f6d640 1 -- 172.21.15.66:0/318170875 wait complete. 2024-03-30T13:10:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.273+0000 7f03f5f6d640 1 Processor -- start 2024-03-30T13:10:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.273+0000 7f03f5f6d640 1 -- start start 2024-03-30T13:10:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.273+0000 7f03f5f6d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 0x7f03f014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03f5f6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f0072230 0x7f03f0160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03f5f6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072610 0x7f03f0161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03f5f6d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0072ac0 con 0x7f03f006db10 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03f5f6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03f015ec00 con 0x7f03f0072610 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03f5f6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f03f006fa90 con 0x7f03f0072230 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 0x7f03f014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 0x7f03f014cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34054/0 (socket says 172.21.15.66:34054) 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 -- 172.21.15.66:0/2541269465 learned_addr learned my addr 172.21.15.66:0/2541269465 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03eeffd640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f0072230 0x7f03f0160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03effff640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072610 0x7f03f0161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 -- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f0072230 msgr2=0x7f03f0160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f0072230 0x7f03f0160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 -- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072610 msgr2=0x7f03f0161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072610 0x7f03f0161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 -- 172.21.15.66:0/2541269465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f00b5f70 con 0x7f03f006db10 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03ef7fe640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 0x7f03f014cf30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009890 tx=0x7f03e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0035040 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e003b860 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f00b6210 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.274+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e003bc50 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.275+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f01624b0 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.275+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f0075a10 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.276+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f03e0038020 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.276+0000 7f03d7fff640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f03dc06c530 0x7f03dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.276+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f03e00ad8d0 con 0x7f03f006db10 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.276+0000 7f03eeffd640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f03dc06c530 0x7f03dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.277+0000 7f03eeffd640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f03dc06c530 0x7f03dc06e9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03f014d470 tx=0x7f03e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.281+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f03e007bc40 con 0x7f03f006db10 2024-03-30T13:10:03.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.466+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03f014da10 con 0x7f03f006db10 2024-03-30T13:10:03.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.467+0000 7f03d7fff640 1 -- 172.21.15.66:0/2541269465 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f03e007b540 con 0x7f03f006db10 2024-03-30T13:10:03.472 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:03.472 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f03dc06c530 msgr2=0x7f03dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f03dc06c530 0x7f03dc06e9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03f014d470 tx=0x7f03e4009040 comp rx=0 tx=0).stop 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 msgr2=0x7f03f014cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 0x7f03f014cf30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009890 tx=0x7f03e00046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 shutdown_connections 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f03dc06c530 0x7f03dc06e9f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03f0072610 0x7f03f0161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03f0072230 0x7f03f0160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 --2- 172.21.15.66:0/2541269465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03f006db10 0x7f03f014cf30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 >> 172.21.15.66:0/2541269465 conn(0x7f03f00d0cb0 msgr2=0x7f03f015c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.469+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 shutdown_connections 2024-03-30T13:10:03.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.470+0000 7f03f5f6d640 1 -- 172.21.15.66:0/2541269465 wait complete. 2024-03-30T13:10:03.485 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:03.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.589+0000 7f5ad02c8640 1 Processor -- start 2024-03-30T13:10:03.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.589+0000 7f5ad02c8640 1 -- start start 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ad02c8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ac8155990 0x7f5ac806c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ad02c8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac81624f0 con 0x7f5ac80b4050 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ad02c8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac81627d0 con 0x7f5ac80d04d0 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ad02c8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac81622a0 con 0x7f5ac8155990 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ace064640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ac8155990 0x7f5ac806c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5acd863640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5ac80b4050 0x7f5ac80cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53774/0 (socket says 172.21.15.66:53774) 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5acd863640 1 -- 172.21.15.66:0/1806163283 learned_addr learned my addr 172.21.15.66:0/1806163283 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:03.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ace064640 1 -- 172.21.15.66:0/1806163283 >> v1:172.21.15.117:6789/0 conn(0x7f5ac80d04d0 legacy=0x7f5ac815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.590+0000 7f5ace064640 1 -- 172.21.15.66:0/1806163283 >> v1:172.21.15.66:6789/0 conn(0x7f5ac80b4050 legacy=0x7f5ac80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.591+0000 7f5ace064640 1 -- 172.21.15.66:0/1806163283 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8162ab0 con 0x7f5ac8155990 2024-03-30T13:10:03.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.591+0000 7f5acd062640 1 -- 172.21.15.66:0/1806163283 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308492634 0 0) 0x7f5ac81624f0 con 0x7f5ac80b4050 2024-03-30T13:10:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.591+0000 7f5ace064640 1 --2- 172.21.15.66:0/1806163283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac8155990 0x7f5ac806c690 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097a0 tx=0x7f5ac002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48cbb50b48c6566 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.591+0000 7f5acd062640 1 -- 172.21.15.66:0/1806163283 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0035020 con 0x7f5ac8155990 2024-03-30T13:10:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.591+0000 7f5acd062640 1 -- 172.21.15.66:0/1806163283 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0031830 con 0x7f5ac8155990 2024-03-30T13:10:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.591+0000 7f5acd062640 1 -- 172.21.15.66:0/1806163283 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0031c00 con 0x7f5ac8155990 2024-03-30T13:10:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 -- 172.21.15.66:0/1806163283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac8155990 msgr2=0x7f5ac806c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/1806163283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac8155990 0x7f5ac806c690 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097a0 tx=0x7f5ac002b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 -- 172.21.15.66:0/1806163283 shutdown_connections 2024-03-30T13:10:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/1806163283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac8155990 0x7f5ac806c690 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 -- 172.21.15.66:0/1806163283 >> 172.21.15.66:0/1806163283 conn(0x7f5ac80672c0 msgr2=0x7f5ac80676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 -- 172.21.15.66:0/1806163283 shutdown_connections 2024-03-30T13:10:03.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.592+0000 7f5ad02c8640 1 -- 172.21.15.66:0/1806163283 wait complete. 2024-03-30T13:10:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.593+0000 7f5ad02c8640 1 Processor -- start 2024-03-30T13:10:03.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.593+0000 7f5ad02c8640 1 -- start start 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ad02c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 0x7f5ac80c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ad02c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac80d04d0 0x7f5ac81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ad02c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac8155990 0x7f5ac80c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ad02c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80cfb20 con 0x7f5ac80b4050 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ad02c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80cf9a0 con 0x7f5ac80d04d0 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ad02c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac80b4500 con 0x7f5ac8155990 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ace064640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 0x7f5ac80c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ace064640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 0x7f5ac80c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34070/0 (socket says 172.21.15.66:34070) 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ace064640 1 -- 172.21.15.66:0/2359690962 learned_addr learned my addr 172.21.15.66:0/2359690962 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5acd863640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac80d04d0 0x7f5ac81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ace865640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac8155990 0x7f5ac80c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ace064640 1 -- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac8155990 msgr2=0x7f5ac80c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.594+0000 7f5ace064640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac8155990 0x7f5ac80c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ace064640 1 -- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac80d04d0 msgr2=0x7f5ac81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ace064640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac80d04d0 0x7f5ac81537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ace064640 1 -- 172.21.15.66:0/2359690962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8153ce0 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5acd863640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac80d04d0 0x7f5ac81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ace064640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 0x7f5ac80c5d20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097a0 tx=0x7f5ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0035050 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0004230 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac80c63b0 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac003b740 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.595+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac80c6c50 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.596+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac8075a10 con 0x7f5ac80b4050 2024-03-30T13:10:03.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.596+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5ac0038020 con 0x7f5ac80b4050 2024-03-30T13:10:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.597+0000 7f5ac6ffd640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5ab406c530 0x7f5ab406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.597+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5ac00ad810 con 0x7f5ac80b4050 2024-03-30T13:10:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.597+0000 7f5acd863640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5ab406c530 0x7f5ab406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.597+0000 7f5acd863640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5ab406c530 0x7f5ab406e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80b7ae0 tx=0x7f5abc007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.601+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5ac8153ce0 con 0x7f5ac80b4050 2024-03-30T13:10:03.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.788+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ac80cf5c0 con 0x7f5ac80b4050 2024-03-30T13:10:03.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.789+0000 7f5ac6ffd640 1 -- 172.21.15.66:0/2359690962 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f5ac00abe20 con 0x7f5ac80b4050 2024-03-30T13:10:03.794 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:03.794 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:03.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.791+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5ab406c530 msgr2=0x7f5ab406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.791+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5ab406c530 0x7f5ab406e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80b7ae0 tx=0x7f5abc007e40 comp rx=0 tx=0).stop 2024-03-30T13:10:03.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.791+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 msgr2=0x7f5ac80c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.791+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 0x7f5ac80c5d20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097a0 tx=0x7f5ac00046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 shutdown_connections 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5ab406c530 0x7f5ab406e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac8155990 0x7f5ac80c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac80d04d0 0x7f5ac81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 --2- 172.21.15.66:0/2359690962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ac80b4050 0x7f5ac80c5d20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 >> 172.21.15.66:0/2359690962 conn(0x7f5ac80672c0 msgr2=0x7f5ac80b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 shutdown_connections 2024-03-30T13:10:03.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.792+0000 7f5ad02c8640 1 -- 172.21.15.66:0/2359690962 wait complete. 2024-03-30T13:10:03.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T13:10:03.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.913+0000 7fa7f8274640 1 Processor -- start 2024-03-30T13:10:03.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.913+0000 7fa7f8274640 1 -- start start 2024-03-30T13:10:03.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f8274640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7f00b3760 0x7fa7f00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f8274640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f00cf860 con 0x7fa7f00d3ba0 2024-03-30T13:10:03.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f8274640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f00cfb40 con 0x7fa7f00d13d0 2024-03-30T13:10:03.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f8274640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f00d33a0 con 0x7fa7f00b3760 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f580f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7f00b3760 0x7fa7f00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f6811640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa7f00d3ba0 0x7fa7f00b7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53790/0 (socket says 172.21.15.66:53790) 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f6811640 1 -- 172.21.15.66:0/2635186722 learned_addr learned my addr 172.21.15.66:0/2635186722 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f500e640 1 -- 172.21.15.66:0/2635186722 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4238732822 0 0) 0x7fa7f00cf860 con 0x7fa7f00d3ba0 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f500e640 1 -- 172.21.15.66:0/2635186722 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7d8003520 con 0x7fa7f00d3ba0 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f500e640 1 -- 172.21.15.66:0/2635186722 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914115601 0 0) 0x7fa7f00cfb40 con 0x7fa7f00d13d0 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f580f640 1 -- 172.21.15.66:0/2635186722 >> v1:172.21.15.117:6789/0 conn(0x7fa7f00d13d0 legacy=0x7fa7f00d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f580f640 1 -- 172.21.15.66:0/2635186722 >> v1:172.21.15.66:6789/0 conn(0x7fa7f00d3ba0 legacy=0x7fa7f00b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.914+0000 7fa7f580f640 1 -- 172.21.15.66:0/2635186722 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f00d3680 con 0x7fa7f00b3760 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.915+0000 7fa7f580f640 1 --2- 172.21.15.66:0/2635186722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00b3760 0x7fa7f00cecc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40097a0 tx=0x7fa7e402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117c97674f7e0e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.915+0000 7fa7f500e640 1 -- 172.21.15.66:0/2635186722 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4034070 con 0x7fa7f00b3760 2024-03-30T13:10:03.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.915+0000 7fa7f500e640 1 -- 172.21.15.66:0/2635186722 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e4030610 con 0x7fa7f00b3760 2024-03-30T13:10:03.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.915+0000 7fa7f500e640 1 -- 172.21.15.66:0/2635186722 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40309e0 con 0x7fa7f00b3760 2024-03-30T13:10:03.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 -- 172.21.15.66:0/2635186722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00b3760 msgr2=0x7fa7f00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 --2- 172.21.15.66:0/2635186722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00b3760 0x7fa7f00cecc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40097a0 tx=0x7fa7e402b960 comp rx=0 tx=0).stop 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 -- 172.21.15.66:0/2635186722 shutdown_connections 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 --2- 172.21.15.66:0/2635186722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00b3760 0x7fa7f00cecc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 -- 172.21.15.66:0/2635186722 >> 172.21.15.66:0/2635186722 conn(0x7fa7f0067710 msgr2=0x7fa7f0067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 -- 172.21.15.66:0/2635186722 shutdown_connections 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.916+0000 7fa7f8274640 1 -- 172.21.15.66:0/2635186722 wait complete. 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.917+0000 7fa7f8274640 1 Processor -- start 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.917+0000 7fa7f8274640 1 -- start start 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f8274640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 0x7fa7f01e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f8274640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00d13d0 0x7fa7f00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f8274640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f00d3ba0 0x7fa7f01f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f8274640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f015eab0 con 0x7fa7f00b3760 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f8274640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f015e930 con 0x7fa7f00d13d0 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f8274640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f00cf4c0 con 0x7fa7f00d3ba0 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 0x7fa7f01e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 0x7fa7f01e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34072/0 (socket says 172.21.15.66:34072) 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 -- 172.21.15.66:0/211445893 learned_addr learned my addr 172.21.15.66:0/211445893 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f580f640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00d13d0 0x7fa7f00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6811640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f00d3ba0 0x7fa7f01f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 -- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f00d3ba0 msgr2=0x7fa7f01f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f00d3ba0 0x7fa7f01f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 -- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00d13d0 msgr2=0x7fa7f00b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00d13d0 0x7fa7f00b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f6010640 1 -- 172.21.15.66:0/211445893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f00b5840 con 0x7fa7f00b3760 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.918+0000 7fa7f580f640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00d13d0 0x7fa7f00b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.919+0000 7fa7f6010640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 0x7fa7f01e8050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800c4b0 tx=0x7fa7e800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.919+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8017070 con 0x7fa7f00b3760 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.919+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e8005480 con 0x7fa7f00b3760 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.919+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8003830 con 0x7fa7f00b3760 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.919+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f00b5b20 con 0x7fa7f00b3760 2024-03-30T13:10:03.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.919+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f00b60e0 con 0x7fa7f00b3760 2024-03-30T13:10:03.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.920+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f00d1c70 con 0x7fa7f00b3760 2024-03-30T13:10:03.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.920+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa7e801a070 con 0x7fa7f00b3760 2024-03-30T13:10:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.921+0000 7fa7eeffd640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7dc06c560 0x7fa7dc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.921+0000 7fa7f580f640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7dc06c560 0x7fa7dc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.921+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa7e808ad10 con 0x7fa7f00b3760 2024-03-30T13:10:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.921+0000 7fa7f580f640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7dc06c560 0x7fa7dc06ea20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4036000 tx=0x7fa7e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:03.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:03.925+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa7f00d1c70 con 0x7fa7f00b3760 2024-03-30T13:10:04.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.147+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa7f0075890 con 0x7fa7f00b3760 2024-03-30T13:10:04.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.158+0000 7fa7eeffd640 1 -- 172.21.15.66:0/211445893 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fa7e80589a0 con 0x7fa7f00b3760 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7dc06c560 msgr2=0x7fa7dc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7dc06c560 0x7fa7dc06ea20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4036000 tx=0x7fa7e4008030 comp rx=0 tx=0).stop 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 msgr2=0x7fa7f01e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 0x7fa7f01e8050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800c4b0 tx=0x7fa7e800c970 comp rx=0 tx=0).stop 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 shutdown_connections 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7dc06c560 0x7fa7dc06ea20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f00d3ba0 0x7fa7f01f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f00d13d0 0x7fa7f00b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 --2- 172.21.15.66:0/211445893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7f00b3760 0x7fa7f01e8050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 >> 172.21.15.66:0/211445893 conn(0x7fa7f0067710 msgr2=0x7fa7f00ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:04.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 shutdown_connections 2024-03-30T13:10:04.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.160+0000 7fa7f8274640 1 -- 172.21.15.66:0/211445893 wait complete. 2024-03-30T13:10:04.176 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-30T13:10:04.176 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T13:10:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.281+0000 7fa6649db640 1 Processor -- start 2024-03-30T13:10:04.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.282+0000 7fa6649db640 1 -- start start 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.282+0000 7fa6649db640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa6601690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.282+0000 7fa6649db640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66015e950 con 0x7fa6600ba0b0 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.282+0000 7fa6649db640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6600b8890 con 0x7fa66015e570 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.282+0000 7fa6649db640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa6600b94a0 con 0x7fa6600cf7f0 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.282+0000 7fa657fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa6600ba0b0 0x7fa6600cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53806/0 (socket says 172.21.15.66:53806) 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa657fff640 1 -- 172.21.15.66:0/499003000 learned_addr learned my addr 172.21.15.66:0/499003000 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65ed9d640 1 --2- 172.21.15.66:0/499003000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa6601690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 241951651 0 0) 0x7fa66015e950 con 0x7fa6600ba0b0 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa64c003520 con 0x7fa6600ba0b0 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632280059 0 0) 0x7fa6600b8890 con 0x7fa66015e570 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66015e950 con 0x7fa66015e570 2024-03-30T13:10:04.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65ed9d640 1 -- 172.21.15.66:0/499003000 >> v1:172.21.15.117:6789/0 conn(0x7fa66015e570 legacy=0x7fa6600b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65ed9d640 1 -- 172.21.15.66:0/499003000 >> v1:172.21.15.66:6789/0 conn(0x7fa6600ba0b0 legacy=0x7fa6600cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.283+0000 7fa65ed9d640 1 -- 172.21.15.66:0/499003000 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6600b8b70 con 0x7fa6600cf7f0 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa65ed9d640 1 --2- 172.21.15.66:0/499003000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa6601690b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6600b9620 tx=0x7fa65802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45116907e00339a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658039070 con 0x7fa6600cf7f0 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65802ccd0 con 0x7fa6600cf7f0 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa65dd9b640 1 -- 172.21.15.66:0/499003000 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658034660 con 0x7fa6600cf7f0 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 -- 172.21.15.66:0/499003000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 msgr2=0x7fa6601690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 --2- 172.21.15.66:0/499003000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa6601690b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6600b9620 tx=0x7fa65802bf20 comp rx=0 tx=0).stop 2024-03-30T13:10:04.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 -- 172.21.15.66:0/499003000 shutdown_connections 2024-03-30T13:10:04.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 --2- 172.21.15.66:0/499003000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa6601690b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 -- 172.21.15.66:0/499003000 >> 172.21.15.66:0/499003000 conn(0x7fa6600676e0 msgr2=0x7fa660067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:04.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 -- 172.21.15.66:0/499003000 shutdown_connections 2024-03-30T13:10:04.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.284+0000 7fa6649db640 1 -- 172.21.15.66:0/499003000 wait complete. 2024-03-30T13:10:04.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 Processor -- start 2024-03-30T13:10:04.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 -- start start 2024-03-30T13:10:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 0x7fa6600c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa66014b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66015e570 0x7fa660155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6600d3740 con 0x7fa6600ba0b0 2024-03-30T13:10:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6600b9130 con 0x7fa66015e570 2024-03-30T13:10:04.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.286+0000 7fa6649db640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa6600d38c0 con 0x7fa6600cf7f0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 0x7fa6600c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 0x7fa6600c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34086/0 (socket says 172.21.15.66:34086) 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 -- 172.21.15.66:0/1813357949 learned_addr learned my addr 172.21.15.66:0/1813357949 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65ed9d640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66015e570 0x7fa660155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa657fff640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa66014b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 -- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 msgr2=0x7fa66014b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa66014b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 -- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66015e570 msgr2=0x7fa660155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66015e570 0x7fa660155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 -- 172.21.15.66:0/1813357949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa660155ab0 con 0x7fa6600ba0b0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa65e59c640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 0x7fa6600c9070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa65000c430 tx=0x7fa65000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650017070 con 0x7fa6600ba0b0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6500052f0 con 0x7fa6600ba0b0 2024-03-30T13:10:04.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650003830 con 0x7fa6600ba0b0 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa660155d90 con 0x7fa6600ba0b0 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.287+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa660156350 con 0x7fa6600ba0b0 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.288+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa660075890 con 0x7fa6600ba0b0 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.289+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa65001a070 con 0x7fa6600ba0b0 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.289+0000 7fa656ffd640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa64006c560 0x7fa64006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.289+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa65008acb0 con 0x7fa6600ba0b0 2024-03-30T13:10:04.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.289+0000 7fa657fff640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa64006c560 0x7fa64006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:04.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.290+0000 7fa657fff640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa64006c560 0x7fa64006ea20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6600c9810 tx=0x7fa648007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:04.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.294+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa6500589d0 con 0x7fa6600ba0b0 2024-03-30T13:10:04.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:04.519+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa66015bd70 con 0x7fa6600ba0b0 2024-03-30T13:10:05.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.168+0000 7fa656ffd640 1 -- 172.21.15.66:0/1813357949 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fa65005ccd0 con 0x7fa6600ba0b0 2024-03-30T13:10:05.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa64006c560 msgr2=0x7fa64006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa64006c560 0x7fa64006ea20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6600c9810 tx=0x7fa648007d30 comp rx=0 tx=0).stop 2024-03-30T13:10:05.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 msgr2=0x7fa6600c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 0x7fa6600c9070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa65000c430 tx=0x7fa65000c8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 shutdown_connections 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa64006c560 0x7fa64006ea20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66015e570 0x7fa660155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6600cf7f0 0x7fa66014b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 --2- 172.21.15.66:0/1813357949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6600ba0b0 0x7fa6600c9070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 >> 172.21.15.66:0/1813357949 conn(0x7fa6600676e0 msgr2=0x7fa66015c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 shutdown_connections 2024-03-30T13:10:05.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.170+0000 7fa6649db640 1 -- 172.21.15.66:0/1813357949 wait complete. 2024-03-30T13:10:05.189 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T13:10:05.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:05.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 Processor -- start 2024-03-30T13:10:05.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 -- start start 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae6c1445e0 0x7fae6c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae6c072340 0x7fae6c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c0cd610 con 0x7fae6c1445e0 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c1449c0 con 0x7fae6c072340 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.297+0000 7fae72fba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0cf5b0 con 0x7fae6c0727d0 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae6c1445e0 0x7fae6c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae6c1445e0 0x7fae6c0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34092/0 (socket says 172.21.15.66:34092) 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 -- 172.21.15.66:0/2546479993 learned_addr learned my addr 172.21.15.66:0/2546479993 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae6bfff640 1 --2- 172.21.15.66:0/2546479993 >> v2:172.21.15.117:3300/0 conn(0x7fae6c072340 0x7fae6c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae71557640 1 --2- 172.21.15.66:0/2546479993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 -- 172.21.15.66:0/2546479993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 msgr2=0x7fae6c0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 --2- 172.21.15.66:0/2546479993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c0cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 -- 172.21.15.66:0/2546479993 >> v2:172.21.15.117:3300/0 conn(0x7fae6c072340 msgr2=0x7fae6c0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 --2- 172.21.15.66:0/2546479993 >> v2:172.21.15.117:3300/0 conn(0x7fae6c072340 0x7fae6c0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae70d56640 1 -- 172.21.15.66:0/2546479993 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c144e10 con 0x7fae6c1445e0 2024-03-30T13:10:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.298+0000 7fae6bfff640 1 --2- 172.21.15.66:0/2546479993 >> v2:172.21.15.117:3300/0 conn(0x7fae6c072340 0x7fae6c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae70d56640 1 --2- 172.21.15.66:0/2546479993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c0cf070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0714b0 tx=0x7fae6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fc26b12cb163a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae6b7fe640 1 -- 172.21.15.66:0/2546479993 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6402ca70 con 0x7fae6c1445e0 2024-03-30T13:10:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae6b7fe640 1 -- 172.21.15.66:0/2546479993 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6402cc10 con 0x7fae6c1445e0 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae6b7fe640 1 -- 172.21.15.66:0/2546479993 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640343f0 con 0x7fae6c1445e0 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 -- 172.21.15.66:0/2546479993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 msgr2=0x7fae6c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 --2- 172.21.15.66:0/2546479993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c0cf070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0714b0 tx=0x7fae6402bed0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 -- 172.21.15.66:0/2546479993 shutdown_connections 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 --2- 172.21.15.66:0/2546479993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 --2- 172.21.15.66:0/2546479993 >> v2:172.21.15.117:3300/0 conn(0x7fae6c072340 0x7fae6c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 --2- 172.21.15.66:0/2546479993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c0cf070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.299+0000 7fae72fba640 1 -- 172.21.15.66:0/2546479993 >> 172.21.15.66:0/2546479993 conn(0x7fae6c0b1c90 msgr2=0x7fae6c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.300+0000 7fae72fba640 1 -- 172.21.15.66:0/2546479993 shutdown_connections 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.300+0000 7fae72fba640 1 -- 172.21.15.66:0/2546479993 wait complete. 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.301+0000 7fae72fba640 1 Processor -- start 2024-03-30T13:10:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.301+0000 7fae72fba640 1 -- start start 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae6c072340 0x7fae6c15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0cad60 con 0x7fae6c1445e0 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c15edb0 con 0x7fae6c072340 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0cb040 con 0x7fae6c0727d0 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34108/0 (socket says 172.21.15.66:34108) 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 -- 172.21.15.66:0/86627407 learned_addr learned my addr 172.21.15.66:0/86627407 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae70d56640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae6c072340 0x7fae6c15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae6bfff640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 -- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 msgr2=0x7fae6c15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c15b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 -- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae6c072340 msgr2=0x7fae6c15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae6c072340 0x7fae6c15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 -- 172.21.15.66:0/86627407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c15ca70 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae71557640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c15be10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e960 tx=0x7fae5c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c017070 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c00c980 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c15c4b0 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00cd70 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.302+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1544f0 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.304+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c075a10 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.304+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fae5c01a070 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.304+0000 7fae697fa640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae4c06c530 0x7fae4c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.304+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fae5c08bc70 con 0x7fae6c1445e0 2024-03-30T13:10:05.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.304+0000 7fae70d56640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae4c06c530 0x7fae4c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.305+0000 7fae70d56640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae4c06c530 0x7fae4c06e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae64035000 tx=0x7fae64030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.309+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fae5c059990 con 0x7fae6c1445e0 2024-03-30T13:10:05.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.530+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae6c0c9f50 con 0x7fae6c1445e0 2024-03-30T13:10:05.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.531+0000 7fae697fa640 1 -- 172.21.15.66:0/86627407 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fae5c05ee20 con 0x7fae6c1445e0 2024-03-30T13:10:05.536 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:05.537 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:05.539 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:05.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae4c06c530 msgr2=0x7fae4c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae4c06c530 0x7fae4c06e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae64035000 tx=0x7fae64030000 comp rx=0 tx=0).stop 2024-03-30T13:10:05.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 msgr2=0x7fae6c15be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c15be10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e960 tx=0x7fae5c00ee20 comp rx=0 tx=0).stop 2024-03-30T13:10:05.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 shutdown_connections 2024-03-30T13:10:05.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae4c06c530 0x7fae4c06e9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae6c1445e0 0x7fae6c15be10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae6c0727d0 0x7fae6c15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 --2- 172.21.15.66:0/86627407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae6c072340 0x7fae6c15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 >> 172.21.15.66:0/86627407 conn(0x7fae6c0b1c90 msgr2=0x7fae6c15f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:05.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 shutdown_connections 2024-03-30T13:10:05.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.533+0000 7fae72fba640 1 -- 172.21.15.66:0/86627407 wait complete. 2024-03-30T13:10:05.549 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:05.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.652+0000 7f3838ac8640 1 Processor -- start 2024-03-30T13:10:05.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.652+0000 7f3838ac8640 1 -- start start 2024-03-30T13:10:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3838ac8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f383415e4f0 0x7f38340cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3838ac8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38340b7530 0x7f38340d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3838ac8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38340b9d20 con 0x7f38340b7990 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3838ac8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f38340ba000 con 0x7f38340b7530 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3838ac8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f38340ba2e0 con 0x7f383415e4f0 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3832d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f38340b7990 0x7f38340b5e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53810/0 (socket says 172.21.15.66:53810) 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3832d9d640 1 -- 172.21.15.66:0/18651807 learned_addr learned my addr 172.21.15.66:0/18651807 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f383259c640 1 --2- 172.21.15.66:0/18651807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f383415e4f0 0x7f38340cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f382bfff640 1 --2- 172.21.15.66:0/18651807 >> v2:172.21.15.117:3300/0 conn(0x7f38340b7530 0x7f38340d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f383259c640 1 -- 172.21.15.66:0/18651807 >> v2:172.21.15.117:3300/0 conn(0x7f38340b7530 msgr2=0x7f38340d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f383259c640 1 --2- 172.21.15.66:0/18651807 >> v2:172.21.15.117:3300/0 conn(0x7f38340b7530 0x7f38340d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f383259c640 1 -- 172.21.15.66:0/18651807 >> v1:172.21.15.66:6789/0 conn(0x7f38340b7990 legacy=0x7f38340b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f3831d9b640 1 -- 172.21.15.66:0/18651807 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962968396 0 0) 0x7f38340b9d20 con 0x7f38340b7990 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f383259c640 1 -- 172.21.15.66:0/18651807 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340ceaf0 con 0x7f383415e4f0 2024-03-30T13:10:05.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.653+0000 7f382bfff640 1 --2- 172.21.15.66:0/18651807 >> v2:172.21.15.117:3300/0 conn(0x7f38340b7530 0x7f38340d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.654+0000 7f383259c640 1 --2- 172.21.15.66:0/18651807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f383415e4f0 0x7f38340cf460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38240097a0 tx=0x7f382402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e88705ac4ef7cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.654+0000 7f3831d9b640 1 -- 172.21.15.66:0/18651807 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824031780 con 0x7f383415e4f0 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.654+0000 7f3831d9b640 1 -- 172.21.15.66:0/18651807 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3824037070 con 0x7f383415e4f0 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.654+0000 7f3831d9b640 1 -- 172.21.15.66:0/18651807 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824031b40 con 0x7f383415e4f0 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 -- 172.21.15.66:0/18651807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f383415e4f0 msgr2=0x7f38340cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 --2- 172.21.15.66:0/18651807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f383415e4f0 0x7f38340cf460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38240097a0 tx=0x7f382402b820 comp rx=0 tx=0).stop 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 -- 172.21.15.66:0/18651807 shutdown_connections 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 --2- 172.21.15.66:0/18651807 >> v2:172.21.15.117:3300/0 conn(0x7f38340b7530 0x7f38340d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 --2- 172.21.15.66:0/18651807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f383415e4f0 0x7f38340cf460 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 -- 172.21.15.66:0/18651807 >> 172.21.15.66:0/18651807 conn(0x7f38340676e0 msgr2=0x7f3834067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 -- 172.21.15.66:0/18651807 shutdown_connections 2024-03-30T13:10:05.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.655+0000 7f3838ac8640 1 -- 172.21.15.66:0/18651807 wait complete. 2024-03-30T13:10:05.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.656+0000 7f3838ac8640 1 Processor -- start 2024-03-30T13:10:05.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.656+0000 7f3838ac8640 1 -- start start 2024-03-30T13:10:05.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3838ac8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38340b7530 0x7f38340b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3838ac8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 0x7f383414d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3838ac8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f383415e4f0 0x7f38340b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3838ac8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f383415ea50 con 0x7f38340b7990 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3838ac8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f383415e8d0 con 0x7f383415e4f0 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3838ac8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f38340d2ef0 con 0x7f38340b7530 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 0x7f383414d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 0x7f383414d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34122/0 (socket says 172.21.15.66:34122) 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 -- 172.21.15.66:0/1846251760 learned_addr learned my addr 172.21.15.66:0/1846251760 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f383259c640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38340b7530 0x7f38340b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f3832d9d640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f383415e4f0 0x7f38340b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 -- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38340b7530 msgr2=0x7f38340b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38340b7530 0x7f38340b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 -- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f383415e4f0 msgr2=0x7f38340b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f383415e4f0 0x7f38340b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 -- 172.21.15.66:0/1846251760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340b4080 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f383259c640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38340b7530 0x7f38340b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.657+0000 7f382bfff640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 0x7f383414d930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f381c00ec10 tx=0x7f381c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.658+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c017070 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.658+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c00cbd0 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.658+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38340b4360 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.658+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c013470 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.658+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38340b4920 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.659+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38340b3e10 con 0x7f38340b7990 2024-03-30T13:10:05.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.659+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f381c01a070 con 0x7f38340b7990 2024-03-30T13:10:05.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.659+0000 7f382affd640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f381406c530 0x7f381406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.660+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f381c08bcd0 con 0x7f38340b7990 2024-03-30T13:10:05.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.660+0000 7f383259c640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f381406c530 0x7f381406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.660+0000 7f383259c640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f381406c530 0x7f381406e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3824036000 tx=0x7f382403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.664+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f381c0598e0 con 0x7f38340b7990 2024-03-30T13:10:05.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.850+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3834075a10 con 0x7f38340b7990 2024-03-30T13:10:05.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.851+0000 7f382affd640 1 -- 172.21.15.66:0/1846251760 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f381c071e20 con 0x7f38340b7990 2024-03-30T13:10:05.856 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:05.856 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f381406c530 msgr2=0x7f381406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f381406c530 0x7f381406e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3824036000 tx=0x7f382403d040 comp rx=0 tx=0).stop 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 msgr2=0x7f383414d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 0x7f383414d930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f381c00ec10 tx=0x7f381c00c520 comp rx=0 tx=0).stop 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 shutdown_connections 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f381406c530 0x7f381406e9f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f383415e4f0 0x7f38340b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340b7990 0x7f383414d930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 --2- 172.21.15.66:0/1846251760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38340b7530 0x7f38340b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 >> 172.21.15.66:0/1846251760 conn(0x7f38340676e0 msgr2=0x7f3834160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 shutdown_connections 2024-03-30T13:10:05.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.853+0000 7f3838ac8640 1 -- 172.21.15.66:0/1846251760 wait complete. 2024-03-30T13:10:05.870 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T13:10:05.870 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:10:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.972+0000 7f4a8b093640 1 Processor -- start 2024-03-30T13:10:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.972+0000 7f4a8b093640 1 -- start start 2024-03-30T13:10:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a8b093640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4a84072610 0x7f4a8415f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a8b093640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4a8406db10 0x7f4a8406df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a8b093640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a840ce840 con 0x7f4a84072230 2024-03-30T13:10:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a8b093640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4a84158f10 con 0x7f4a84072610 2024-03-30T13:10:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a8b093640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8415ef70 con 0x7f4a8406db10 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4a84072610 0x7f4a8415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4a84072610 0x7f4a8415f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34132/0 (socket says 172.21.15.66:34132) 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 -- 172.21.15.66:0/1707407419 learned_addr learned my addr 172.21.15.66:0/1707407419 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a89630640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4a8406db10 0x7f4a8406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 -- 172.21.15.66:0/1707407419 >> v2:172.21.15.117:3300/0 conn(0x7f4a8406db10 msgr2=0x7f4a8406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 --2- 172.21.15.66:0/1707407419 >> v2:172.21.15.117:3300/0 conn(0x7f4a8406db10 0x7f4a8406df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 -- 172.21.15.66:0/1707407419 >> v1:172.21.15.117:6789/0 conn(0x7f4a84072230 legacy=0x7f4a8406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a83fff640 1 -- 172.21.15.66:0/1707407419 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a840ceb20 con 0x7f4a84072610 2024-03-30T13:10:05.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.973+0000 7f4a837fe640 1 -- 172.21.15.66:0/1707407419 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632783554 0 0) 0x7f4a840ce840 con 0x7f4a84072230 2024-03-30T13:10:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.974+0000 7f4a83fff640 1 --2- 172.21.15.66:0/1707407419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415f400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a8415eee0 tx=0x7f4a7802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a185796bfdd5c22 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.974+0000 7f4a837fe640 1 -- 172.21.15.66:0/1707407419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78034070 con 0x7f4a84072610 2024-03-30T13:10:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.974+0000 7f4a837fe640 1 -- 172.21.15.66:0/1707407419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a78030860 con 0x7f4a84072610 2024-03-30T13:10:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.974+0000 7f4a837fe640 1 -- 172.21.15.66:0/1707407419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78030c30 con 0x7f4a84072610 2024-03-30T13:10:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 -- 172.21.15.66:0/1707407419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 msgr2=0x7f4a8415f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 --2- 172.21.15.66:0/1707407419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415f400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a8415eee0 tx=0x7f4a7802b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 -- 172.21.15.66:0/1707407419 shutdown_connections 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 --2- 172.21.15.66:0/1707407419 >> v2:172.21.15.117:3300/0 conn(0x7f4a8406db10 0x7f4a8406df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 --2- 172.21.15.66:0/1707407419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415f400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 -- 172.21.15.66:0/1707407419 >> 172.21.15.66:0/1707407419 conn(0x7f4a840d0cb0 msgr2=0x7f4a840d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 -- 172.21.15.66:0/1707407419 shutdown_connections 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.975+0000 7f4a8b093640 1 -- 172.21.15.66:0/1707407419 wait complete. 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.976+0000 7f4a8b093640 1 Processor -- start 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.976+0000 7f4a8b093640 1 -- start start 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a8b093640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a8406db10 0x7f4a840b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a8b093640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a84072230 0x7f4a840b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a8b093640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a8b093640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84072ac0 con 0x7f4a84072610 2024-03-30T13:10:05.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a8b093640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8415f940 con 0x7f4a8406db10 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a8b093640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8406fa90 con 0x7f4a84072230 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34134/0 (socket says 172.21.15.66:34134) 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 -- 172.21.15.66:0/945911690 learned_addr learned my addr 172.21.15.66:0/945911690 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a88e2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a8406db10 0x7f4a840b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a83fff640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a84072230 0x7f4a840b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 -- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a84072230 msgr2=0x7f4a840b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a84072230 0x7f4a840b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 -- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a8406db10 msgr2=0x7f4a840b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a8406db10 0x7f4a840b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.977+0000 7f4a89630640 1 -- 172.21.15.66:0/945911690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a840b5e00 con 0x7f4a84072610 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.978+0000 7f4a89630640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415b670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00efc0 tx=0x7f4a7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.978+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c018020 con 0x7f4a84072610 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.978+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c014480 con 0x7f4a84072610 2024-03-30T13:10:05.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.978+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a840b3de0 con 0x7f4a84072610 2024-03-30T13:10:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.978+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c014870 con 0x7f4a84072610 2024-03-30T13:10:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.978+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a84160360 con 0x7f4a84072610 2024-03-30T13:10:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.979+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a84075a10 con 0x7f4a84072610 2024-03-30T13:10:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.979+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4a7c005d10 con 0x7f4a84072610 2024-03-30T13:10:05.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.980+0000 7f4a817fa640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a7006c530 0x7f4a7006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:05.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.980+0000 7f4a88e2f640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a7006c530 0x7f4a7006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:05.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.980+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4a7c08c450 con 0x7f4a84072610 2024-03-30T13:10:05.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.980+0000 7f4a88e2f640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a7006c530 0x7f4a7006e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000f60 tx=0x7f4a74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:05.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:05.984+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4a7c022070 con 0x7f4a84072610 2024-03-30T13:10:06.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.165+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a8415b200 con 0x7f4a7006c530 2024-03-30T13:10:06.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.174+0000 7f4a817fa640 1 -- 172.21.15.66:0/945911690 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4a8415b200 con 0x7f4a7006c530 2024-03-30T13:10:06.184 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:06.185 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a7006c530 msgr2=0x7f4a7006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a7006c530 0x7f4a7006e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000f60 tx=0x7f4a74009040 comp rx=0 tx=0).stop 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 msgr2=0x7f4a8415b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415b670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00efc0 tx=0x7f4a7c00c490 comp rx=0 tx=0).stop 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 shutdown_connections 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a7006c530 0x7f4a7006e9f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a84072610 0x7f4a8415b670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a84072230 0x7f4a840b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 --2- 172.21.15.66:0/945911690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a8406db10 0x7f4a840b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 >> 172.21.15.66:0/945911690 conn(0x7f4a840d0cb0 msgr2=0x7f4a840b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 shutdown_connections 2024-03-30T13:10:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.178+0000 7f4a8b093640 1 -- 172.21.15.66:0/945911690 wait complete. 2024-03-30T13:10:06.193 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-30T13:10:04.798334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219500,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755220,"statfs":{"total":773094113280,"available":772869345280,"internally_reserved":0,"allocated":3416064,"data_stored":2064256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002545"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939160+0000","last_change":"2024-03-30T13:09:42.124458+0000","last_active":"2024-03-30T13:09:49.939160+0000","last_peered":"2024-03-30T13:09:49.939160+0000","last_clean":"2024-03-30T13:09:49.939160+0000","last_became_active":"2024-03-30T13:09:42.124158+0000","last_became_peered":"2024-03-30T13:09:42.124158+0000","last_unstale":"2024-03-30T13:09:49.939160+0000","last_undegraded":"2024-03-30T13:09:49.939160+0000","last_fullsized":"2024-03-30T13:09:49.939160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132823+0000","last_change":"2024-03-30T13:09:43.127433+0000","last_active":"2024-03-30T13:09:49.132823+0000","last_peered":"2024-03-30T13:09:49.132823+0000","last_clean":"2024-03-30T13:09:49.132823+0000","last_became_active":"2024-03-30T13:09:43.127127+0000","last_became_peered":"2024-03-30T13:09:43.127127+0000","last_unstale":"2024-03-30T13:09:49.132823+0000","last_undegraded":"2024-03-30T13:09:49.132823+0000","last_fullsized":"2024-03-30T13:09:49.132823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132184+0000","last_change":"2024-03-30T13:09:47.572430+0000","last_active":"2024-03-30T13:09:49.132184+0000","last_peered":"2024-03-30T13:09:49.132184+0000","last_clean":"2024-03-30T13:09:49.132184+0000","last_became_active":"2024-03-30T13:09:47.572059+0000","last_became_peered":"2024-03-30T13:09:47.572059+0000","last_unstale":"2024-03-30T13:09:49.132184+0000","last_undegraded":"2024-03-30T13:09:49.132184+0000","last_fullsized":"2024-03-30T13:09:49.132184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:31:37.218184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132943+0000","last_change":"2024-03-30T13:09:42.126685+0000","last_active":"2024-03-30T13:09:49.132943+0000","last_peered":"2024-03-30T13:09:49.132943+0000","last_clean":"2024-03-30T13:09:49.132943+0000","last_became_active":"2024-03-30T13:09:42.126383+0000","last_became_peered":"2024-03-30T13:09:42.126383+0000","last_unstale":"2024-03-30T13:09:49.132943+0000","last_undegraded":"2024-03-30T13:09:49.132943+0000","last_fullsized":"2024-03-30T13:09:49.132943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938945+0000","last_change":"2024-03-30T13:09:43.132452+0000","last_active":"2024-03-30T13:09:49.938945+0000","last_peered":"2024-03-30T13:09:49.938945+0000","last_clean":"2024-03-30T13:09:49.938945+0000","last_became_active":"2024-03-30T13:09:43.132158+0000","last_became_peered":"2024-03-30T13:09:43.132158+0000","last_unstale":"2024-03-30T13:09:49.938945+0000","last_undegraded":"2024-03-30T13:09:49.938945+0000","last_fullsized":"2024-03-30T13:09:49.938945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131666+0000","last_change":"2024-03-30T13:09:47.157090+0000","last_active":"2024-03-30T13:09:49.131666+0000","last_peered":"2024-03-30T13:09:49.131666+0000","last_clean":"2024-03-30T13:09:49.131666+0000","last_became_active":"2024-03-30T13:09:47.156676+0000","last_became_peered":"2024-03-30T13:09:47.156676+0000","last_unstale":"2024-03-30T13:09:49.131666+0000","last_undegraded":"2024-03-30T13:09:49.131666+0000","last_fullsized":"2024-03-30T13:09:49.131666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938826+0000","last_change":"2024-03-30T13:09:42.127138+0000","last_active":"2024-03-30T13:09:49.938826+0000","last_peered":"2024-03-30T13:09:49.938826+0000","last_clean":"2024-03-30T13:09:49.938826+0000","last_became_active":"2024-03-30T13:09:42.126853+0000","last_became_peered":"2024-03-30T13:09:42.126853+0000","last_unstale":"2024-03-30T13:09:49.938826+0000","last_undegraded":"2024-03-30T13:09:49.938826+0000","last_fullsized":"2024-03-30T13:09:49.938826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938126+0000","last_change":"2024-03-30T13:09:43.137336+0000","last_active":"2024-03-30T13:09:49.938126+0000","last_peered":"2024-03-30T13:09:49.938126+0000","last_clean":"2024-03-30T13:09:49.938126+0000","last_became_active":"2024-03-30T13:09:43.137057+0000","last_became_peered":"2024-03-30T13:09:43.137057+0000","last_unstale":"2024-03-30T13:09:49.938126+0000","last_undegraded":"2024-03-30T13:09:49.938126+0000","last_fullsized":"2024-03-30T13:09:49.938126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:20.284742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132127+0000","last_change":"2024-03-30T13:09:47.149659+0000","last_active":"2024-03-30T13:09:49.132127+0000","last_peered":"2024-03-30T13:09:49.132127+0000","last_clean":"2024-03-30T13:09:49.132127+0000","last_became_active":"2024-03-30T13:09:47.149298+0000","last_became_peered":"2024-03-30T13:09:47.149298+0000","last_unstale":"2024-03-30T13:09:49.132127+0000","last_undegraded":"2024-03-30T13:09:49.132127+0000","last_fullsized":"2024-03-30T13:09:49.132127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:48.863738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938057+0000","last_change":"2024-03-30T13:09:42.126941+0000","last_active":"2024-03-30T13:09:49.938057+0000","last_peered":"2024-03-30T13:09:49.938057+0000","last_clean":"2024-03-30T13:09:49.938057+0000","last_became_active":"2024-03-30T13:09:42.126646+0000","last_became_peered":"2024-03-30T13:09:42.126646+0000","last_unstale":"2024-03-30T13:09:49.938057+0000","last_undegraded":"2024-03-30T13:09:49.938057+0000","last_fullsized":"2024-03-30T13:09:49.938057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938364+0000","last_change":"2024-03-30T13:09:43.129721+0000","last_active":"2024-03-30T13:09:49.938364+0000","last_peered":"2024-03-30T13:09:49.938364+0000","last_clean":"2024-03-30T13:09:49.938364+0000","last_became_active":"2024-03-30T13:09:43.128999+0000","last_became_peered":"2024-03-30T13:09:43.128999+0000","last_unstale":"2024-03-30T13:09:49.938364+0000","last_undegraded":"2024-03-30T13:09:49.938364+0000","last_fullsized":"2024-03-30T13:09:49.938364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:28:48.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131877+0000","last_change":"2024-03-30T13:09:47.149841+0000","last_active":"2024-03-30T13:09:49.131877+0000","last_peered":"2024-03-30T13:09:49.131877+0000","last_clean":"2024-03-30T13:09:49.131877+0000","last_became_active":"2024-03-30T13:09:47.148965+0000","last_became_peered":"2024-03-30T13:09:47.148965+0000","last_unstale":"2024-03-30T13:09:49.131877+0000","last_undegraded":"2024-03-30T13:09:49.131877+0000","last_fullsized":"2024-03-30T13:09:49.131877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:08.872490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295198+0000","last_change":"2024-03-30T13:09:43.127058+0000","last_active":"2024-03-30T13:09:49.295198+0000","last_peered":"2024-03-30T13:09:49.295198+0000","last_clean":"2024-03-30T13:09:49.295198+0000","last_became_active":"2024-03-30T13:09:43.126777+0000","last_became_peered":"2024-03-30T13:09:43.126777+0000","last_unstale":"2024-03-30T13:09:49.295198+0000","last_undegraded":"2024-03-30T13:09:49.295198+0000","last_fullsized":"2024-03-30T13:09:49.295198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938548+0000","last_change":"2024-03-30T13:09:42.124505+0000","last_active":"2024-03-30T13:09:49.938548+0000","last_peered":"2024-03-30T13:09:49.938548+0000","last_clean":"2024-03-30T13:09:49.938548+0000","last_became_active":"2024-03-30T13:09:42.124243+0000","last_became_peered":"2024-03-30T13:09:42.124243+0000","last_unstale":"2024-03-30T13:09:49.938548+0000","last_undegraded":"2024-03-30T13:09:49.938548+0000","last_fullsized":"2024-03-30T13:09:49.938548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:47.055563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132399+0000","last_change":"2024-03-30T13:09:47.146122+0000","last_active":"2024-03-30T13:09:49.132399+0000","last_peered":"2024-03-30T13:09:49.132399+0000","last_clean":"2024-03-30T13:09:49.132399+0000","last_became_active":"2024-03-30T13:09:47.145870+0000","last_became_peered":"2024-03-30T13:09:47.145870+0000","last_unstale":"2024-03-30T13:09:49.132399+0000","last_undegraded":"2024-03-30T13:09:49.132399+0000","last_fullsized":"2024-03-30T13:09:49.132399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132985+0000","last_change":"2024-03-30T13:09:43.130941+0000","last_active":"2024-03-30T13:09:49.132985+0000","last_peered":"2024-03-30T13:09:49.132985+0000","last_clean":"2024-03-30T13:09:49.132985+0000","last_became_active":"2024-03-30T13:09:43.130335+0000","last_became_peered":"2024-03-30T13:09:43.130335+0000","last_unstale":"2024-03-30T13:09:49.132985+0000","last_undegraded":"2024-03-30T13:09:49.132985+0000","last_fullsized":"2024-03-30T13:09:49.132985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938369+0000","last_change":"2024-03-30T13:09:42.124102+0000","last_active":"2024-03-30T13:09:49.938369+0000","last_peered":"2024-03-30T13:09:49.938369+0000","last_clean":"2024-03-30T13:09:49.938369+0000","last_became_active":"2024-03-30T13:09:42.123746+0000","last_became_peered":"2024-03-30T13:09:42.123746+0000","last_unstale":"2024-03-30T13:09:49.938369+0000","last_undegraded":"2024-03-30T13:09:49.938369+0000","last_fullsized":"2024-03-30T13:09:49.938369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132249+0000","last_change":"2024-03-30T13:09:47.153343+0000","last_active":"2024-03-30T13:09:49.132249+0000","last_peered":"2024-03-30T13:09:49.132249+0000","last_clean":"2024-03-30T13:09:49.132249+0000","last_became_active":"2024-03-30T13:09:47.152894+0000","last_became_peered":"2024-03-30T13:09:47.152894+0000","last_unstale":"2024-03-30T13:09:49.132249+0000","last_undegraded":"2024-03-30T13:09:49.132249+0000","last_fullsized":"2024-03-30T13:09:49.132249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132418+0000","last_change":"2024-03-30T13:09:43.117715+0000","last_active":"2024-03-30T13:09:49.132418+0000","last_peered":"2024-03-30T13:09:49.132418+0000","last_clean":"2024-03-30T13:09:49.132418+0000","last_became_active":"2024-03-30T13:09:43.116768+0000","last_became_peered":"2024-03-30T13:09:43.116768+0000","last_unstale":"2024-03-30T13:09:49.132418+0000","last_undegraded":"2024-03-30T13:09:49.132418+0000","last_fullsized":"2024-03-30T13:09:49.132418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938237+0000","last_change":"2024-03-30T13:09:42.125545+0000","last_active":"2024-03-30T13:09:49.938237+0000","last_peered":"2024-03-30T13:09:49.938237+0000","last_clean":"2024-03-30T13:09:49.938237+0000","last_became_active":"2024-03-30T13:09:42.125280+0000","last_became_peered":"2024-03-30T13:09:42.125280+0000","last_unstale":"2024-03-30T13:09:49.938237+0000","last_undegraded":"2024-03-30T13:09:49.938237+0000","last_fullsized":"2024-03-30T13:09:49.938237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938821+0000","last_change":"2024-03-30T13:09:47.571186+0000","last_active":"2024-03-30T13:09:49.938821+0000","last_peered":"2024-03-30T13:09:49.938821+0000","last_clean":"2024-03-30T13:09:49.938821+0000","last_became_active":"2024-03-30T13:09:47.570755+0000","last_became_peered":"2024-03-30T13:09:47.570755+0000","last_unstale":"2024-03-30T13:09:49.938821+0000","last_undegraded":"2024-03-30T13:09:49.938821+0000","last_fullsized":"2024-03-30T13:09:49.938821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131859+0000","last_change":"2024-03-30T13:09:43.115171+0000","last_active":"2024-03-30T13:09:49.131859+0000","last_peered":"2024-03-30T13:09:49.131859+0000","last_clean":"2024-03-30T13:09:49.131859+0000","last_became_active":"2024-03-30T13:09:43.113821+0000","last_became_peered":"2024-03-30T13:09:43.113821+0000","last_unstale":"2024-03-30T13:09:49.131859+0000","last_undegraded":"2024-03-30T13:09:49.131859+0000","last_fullsized":"2024-03-30T13:09:49.131859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:04.366501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938219+0000","last_change":"2024-03-30T13:09:42.125753+0000","last_active":"2024-03-30T13:09:49.938219+0000","last_peered":"2024-03-30T13:09:49.938219+0000","last_clean":"2024-03-30T13:09:49.938219+0000","last_became_active":"2024-03-30T13:09:42.125505+0000","last_became_peered":"2024-03-30T13:09:42.125505+0000","last_unstale":"2024-03-30T13:09:49.938219+0000","last_undegraded":"2024-03-30T13:09:49.938219+0000","last_fullsized":"2024-03-30T13:09:49.938219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:43:24.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132292+0000","last_change":"2024-03-30T13:09:47.149806+0000","last_active":"2024-03-30T13:09:49.132292+0000","last_peered":"2024-03-30T13:09:49.132292+0000","last_clean":"2024-03-30T13:09:49.132292+0000","last_became_active":"2024-03-30T13:09:47.149427+0000","last_became_peered":"2024-03-30T13:09:47.149427+0000","last_unstale":"2024-03-30T13:09:49.132292+0000","last_undegraded":"2024-03-30T13:09:49.132292+0000","last_fullsized":"2024-03-30T13:09:49.132292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939301+0000","last_change":"2024-03-30T13:09:43.122135+0000","last_active":"2024-03-30T13:09:49.939301+0000","last_peered":"2024-03-30T13:09:49.939301+0000","last_clean":"2024-03-30T13:09:49.939301+0000","last_became_active":"2024-03-30T13:09:43.121877+0000","last_became_peered":"2024-03-30T13:09:43.121877+0000","last_unstale":"2024-03-30T13:09:49.939301+0000","last_undegraded":"2024-03-30T13:09:49.939301+0000","last_fullsized":"2024-03-30T13:09:49.939301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295852+0000","last_change":"2024-03-30T13:09:42.124452+0000","last_active":"2024-03-30T13:09:49.295852+0000","last_peered":"2024-03-30T13:09:49.295852+0000","last_clean":"2024-03-30T13:09:49.295852+0000","last_became_active":"2024-03-30T13:09:42.124188+0000","last_became_peered":"2024-03-30T13:09:42.124188+0000","last_unstale":"2024-03-30T13:09:49.295852+0000","last_undegraded":"2024-03-30T13:09:49.295852+0000","last_fullsized":"2024-03-30T13:09:49.295852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131805+0000","last_change":"2024-03-30T13:09:47.571007+0000","last_active":"2024-03-30T13:09:49.131805+0000","last_peered":"2024-03-30T13:09:49.131805+0000","last_clean":"2024-03-30T13:09:49.131805+0000","last_became_active":"2024-03-30T13:09:47.570641+0000","last_became_peered":"2024-03-30T13:09:47.570641+0000","last_unstale":"2024-03-30T13:09:49.131805+0000","last_undegraded":"2024-03-30T13:09:49.131805+0000","last_fullsized":"2024-03-30T13:09:49.131805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133129+0000","last_change":"2024-03-30T13:09:43.123324+0000","last_active":"2024-03-30T13:09:49.133129+0000","last_peered":"2024-03-30T13:09:49.133129+0000","last_clean":"2024-03-30T13:09:49.133129+0000","last_became_active":"2024-03-30T13:09:43.122970+0000","last_became_peered":"2024-03-30T13:09:43.122970+0000","last_unstale":"2024-03-30T13:09:49.133129+0000","last_undegraded":"2024-03-30T13:09:49.133129+0000","last_fullsized":"2024-03-30T13:09:49.133129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938610+0000","last_change":"2024-03-30T13:09:42.112467+0000","last_active":"2024-03-30T13:09:49.938610+0000","last_peered":"2024-03-30T13:09:49.938610+0000","last_clean":"2024-03-30T13:09:49.938610+0000","last_became_active":"2024-03-30T13:09:42.112119+0000","last_became_peered":"2024-03-30T13:09:42.112119+0000","last_unstale":"2024-03-30T13:09:49.938610+0000","last_undegraded":"2024-03-30T13:09:49.938610+0000","last_fullsized":"2024-03-30T13:09:49.938610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:45.606451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132205+0000","last_change":"2024-03-30T13:09:47.571907+0000","last_active":"2024-03-30T13:09:49.132205+0000","last_peered":"2024-03-30T13:09:49.132205+0000","last_clean":"2024-03-30T13:09:49.132205+0000","last_became_active":"2024-03-30T13:09:47.571466+0000","last_became_peered":"2024-03-30T13:09:47.571466+0000","last_unstale":"2024-03-30T13:09:49.132205+0000","last_undegraded":"2024-03-30T13:09:49.132205+0000","last_fullsized":"2024-03-30T13:09:49.132205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:36:14.998133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938844+0000","last_change":"2024-03-30T13:09:43.114536+0000","last_active":"2024-03-30T13:09:49.938844+0000","last_peered":"2024-03-30T13:09:49.938844+0000","last_clean":"2024-03-30T13:09:49.938844+0000","last_became_active":"2024-03-30T13:09:43.114268+0000","last_became_peered":"2024-03-30T13:09:43.114268+0000","last_unstale":"2024-03-30T13:09:49.938844+0000","last_undegraded":"2024-03-30T13:09:49.938844+0000","last_fullsized":"2024-03-30T13:09:49.938844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938931+0000","last_change":"2024-03-30T13:09:42.125506+0000","last_active":"2024-03-30T13:09:49.938931+0000","last_peered":"2024-03-30T13:09:49.938931+0000","last_clean":"2024-03-30T13:09:49.938931+0000","last_became_active":"2024-03-30T13:09:42.125297+0000","last_became_peered":"2024-03-30T13:09:42.125297+0000","last_unstale":"2024-03-30T13:09:49.938931+0000","last_undegraded":"2024-03-30T13:09:49.938931+0000","last_fullsized":"2024-03-30T13:09:49.938931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:58.545296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938573+0000","last_change":"2024-03-30T13:09:47.570730+0000","last_active":"2024-03-30T13:09:49.938573+0000","last_peered":"2024-03-30T13:09:49.938573+0000","last_clean":"2024-03-30T13:09:49.938573+0000","last_became_active":"2024-03-30T13:09:47.570239+0000","last_became_peered":"2024-03-30T13:09:47.570239+0000","last_unstale":"2024-03-30T13:09:49.938573+0000","last_undegraded":"2024-03-30T13:09:49.938573+0000","last_fullsized":"2024-03-30T13:09:49.938573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938464+0000","last_change":"2024-03-30T13:09:43.127560+0000","last_active":"2024-03-30T13:09:49.938464+0000","last_peered":"2024-03-30T13:09:49.938464+0000","last_clean":"2024-03-30T13:09:49.938464+0000","last_became_active":"2024-03-30T13:09:43.127245+0000","last_became_peered":"2024-03-30T13:09:43.127245+0000","last_unstale":"2024-03-30T13:09:49.938464+0000","last_undegraded":"2024-03-30T13:09:49.938464+0000","last_fullsized":"2024-03-30T13:09:49.938464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:37.548735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938862+0000","last_change":"2024-03-30T13:09:42.116193+0000","last_active":"2024-03-30T13:09:49.938862+0000","last_peered":"2024-03-30T13:09:49.938862+0000","last_clean":"2024-03-30T13:09:49.938862+0000","last_became_active":"2024-03-30T13:09:42.115609+0000","last_became_peered":"2024-03-30T13:09:42.115609+0000","last_unstale":"2024-03-30T13:09:49.938862+0000","last_undegraded":"2024-03-30T13:09:49.938862+0000","last_fullsized":"2024-03-30T13:09:49.938862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938004+0000","last_change":"2024-03-30T13:09:47.159058+0000","last_active":"2024-03-30T13:09:49.938004+0000","last_peered":"2024-03-30T13:09:49.938004+0000","last_clean":"2024-03-30T13:09:49.938004+0000","last_became_active":"2024-03-30T13:09:47.158646+0000","last_became_peered":"2024-03-30T13:09:47.158646+0000","last_unstale":"2024-03-30T13:09:49.938004+0000","last_undegraded":"2024-03-30T13:09:49.938004+0000","last_fullsized":"2024-03-30T13:09:49.938004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:51.919084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132471+0000","last_change":"2024-03-30T13:09:43.121780+0000","last_active":"2024-03-30T13:09:49.132471+0000","last_peered":"2024-03-30T13:09:49.132471+0000","last_clean":"2024-03-30T13:09:49.132471+0000","last_became_active":"2024-03-30T13:09:43.121519+0000","last_became_peered":"2024-03-30T13:09:43.121519+0000","last_unstale":"2024-03-30T13:09:49.132471+0000","last_undegraded":"2024-03-30T13:09:49.132471+0000","last_fullsized":"2024-03-30T13:09:49.132471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:48.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938327+0000","last_change":"2024-03-30T13:09:42.122015+0000","last_active":"2024-03-30T13:09:49.938327+0000","last_peered":"2024-03-30T13:09:49.938327+0000","last_clean":"2024-03-30T13:09:49.938327+0000","last_became_active":"2024-03-30T13:09:42.121721+0000","last_became_peered":"2024-03-30T13:09:42.121721+0000","last_unstale":"2024-03-30T13:09:49.938327+0000","last_undegraded":"2024-03-30T13:09:49.938327+0000","last_fullsized":"2024-03-30T13:09:49.938327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938675+0000","last_change":"2024-03-30T13:09:47.159041+0000","last_active":"2024-03-30T13:09:49.938675+0000","last_peered":"2024-03-30T13:09:49.938675+0000","last_clean":"2024-03-30T13:09:49.938675+0000","last_became_active":"2024-03-30T13:09:47.158466+0000","last_became_peered":"2024-03-30T13:09:47.158466+0000","last_unstale":"2024-03-30T13:09:49.938675+0000","last_undegraded":"2024-03-30T13:09:49.938675+0000","last_fullsized":"2024-03-30T13:09:49.938675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938050+0000","last_change":"2024-03-30T13:09:43.111943+0000","last_active":"2024-03-30T13:09:49.938050+0000","last_peered":"2024-03-30T13:09:49.938050+0000","last_clean":"2024-03-30T13:09:49.938050+0000","last_became_active":"2024-03-30T13:09:43.111677+0000","last_became_peered":"2024-03-30T13:09:43.111677+0000","last_unstale":"2024-03-30T13:09:49.938050+0000","last_undegraded":"2024-03-30T13:09:49.938050+0000","last_fullsized":"2024-03-30T13:09:49.938050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:33:22.810971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821050+0000","last_change":"2024-03-30T13:09:42.121133+0000","last_active":"2024-03-30T13:09:49.821050+0000","last_peered":"2024-03-30T13:09:49.821050+0000","last_clean":"2024-03-30T13:09:49.821050+0000","last_became_active":"2024-03-30T13:09:42.120851+0000","last_became_peered":"2024-03-30T13:09:42.120851+0000","last_unstale":"2024-03-30T13:09:49.821050+0000","last_undegraded":"2024-03-30T13:09:49.821050+0000","last_fullsized":"2024-03-30T13:09:49.821050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938890+0000","last_change":"2024-03-30T13:09:47.155307+0000","last_active":"2024-03-30T13:09:49.938890+0000","last_peered":"2024-03-30T13:09:49.938890+0000","last_clean":"2024-03-30T13:09:49.938890+0000","last_became_active":"2024-03-30T13:09:47.154875+0000","last_became_peered":"2024-03-30T13:09:47.154875+0000","last_unstale":"2024-03-30T13:09:49.938890+0000","last_undegraded":"2024-03-30T13:09:49.938890+0000","last_fullsized":"2024-03-30T13:09:49.938890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:17:44.422352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937941+0000","last_change":"2024-03-30T13:09:43.128352+0000","last_active":"2024-03-30T13:09:49.937941+0000","last_peered":"2024-03-30T13:09:49.937941+0000","last_clean":"2024-03-30T13:09:49.937941+0000","last_became_active":"2024-03-30T13:09:43.128073+0000","last_became_peered":"2024-03-30T13:09:43.128073+0000","last_unstale":"2024-03-30T13:09:49.937941+0000","last_undegraded":"2024-03-30T13:09:49.937941+0000","last_fullsized":"2024-03-30T13:09:49.937941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:41.389848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938326+0000","last_change":"2024-03-30T13:09:42.124601+0000","last_active":"2024-03-30T13:09:49.938326+0000","last_peered":"2024-03-30T13:09:49.938326+0000","last_clean":"2024-03-30T13:09:49.938326+0000","last_became_active":"2024-03-30T13:09:42.124335+0000","last_became_peered":"2024-03-30T13:09:42.124335+0000","last_unstale":"2024-03-30T13:09:49.938326+0000","last_undegraded":"2024-03-30T13:09:49.938326+0000","last_fullsized":"2024-03-30T13:09:49.938326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:02:06.738441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939206+0000","last_change":"2024-03-30T13:09:47.572608+0000","last_active":"2024-03-30T13:09:49.939206+0000","last_peered":"2024-03-30T13:09:49.939206+0000","last_clean":"2024-03-30T13:09:49.939206+0000","last_became_active":"2024-03-30T13:09:47.571945+0000","last_became_peered":"2024-03-30T13:09:47.571945+0000","last_unstale":"2024-03-30T13:09:49.939206+0000","last_undegraded":"2024-03-30T13:09:49.939206+0000","last_fullsized":"2024-03-30T13:09:49.939206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:06:37.115753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132459+0000","last_change":"2024-03-30T13:09:43.125963+0000","last_active":"2024-03-30T13:09:49.132459+0000","last_peered":"2024-03-30T13:09:49.132459+0000","last_clean":"2024-03-30T13:09:49.132459+0000","last_became_active":"2024-03-30T13:09:43.125682+0000","last_became_peered":"2024-03-30T13:09:43.125682+0000","last_unstale":"2024-03-30T13:09:49.132459+0000","last_undegraded":"2024-03-30T13:09:49.132459+0000","last_fullsized":"2024-03-30T13:09:49.132459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938090+0000","last_change":"2024-03-30T13:09:42.119717+0000","last_active":"2024-03-30T13:09:49.938090+0000","last_peered":"2024-03-30T13:09:49.938090+0000","last_clean":"2024-03-30T13:09:49.938090+0000","last_became_active":"2024-03-30T13:09:42.119185+0000","last_became_peered":"2024-03-30T13:09:42.119185+0000","last_unstale":"2024-03-30T13:09:49.938090+0000","last_undegraded":"2024-03-30T13:09:49.938090+0000","last_fullsized":"2024-03-30T13:09:49.938090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938410+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:09:49.938410+0000","last_peered":"2024-03-30T13:09:49.938410+0000","last_clean":"2024-03-30T13:09:49.938410+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:09:49.938410+0000","last_undegraded":"2024-03-30T13:09:49.938410+0000","last_fullsized":"2024-03-30T13:09:49.938410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295610+0000","last_change":"2024-03-30T13:09:47.572345+0000","last_active":"2024-03-30T13:09:49.295610+0000","last_peered":"2024-03-30T13:09:49.295610+0000","last_clean":"2024-03-30T13:09:49.295610+0000","last_became_active":"2024-03-30T13:09:47.571956+0000","last_became_peered":"2024-03-30T13:09:47.571956+0000","last_unstale":"2024-03-30T13:09:49.295610+0000","last_undegraded":"2024-03-30T13:09:49.295610+0000","last_fullsized":"2024-03-30T13:09:49.295610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938718+0000","last_change":"2024-03-30T13:09:43.134581+0000","last_active":"2024-03-30T13:09:49.938718+0000","last_peered":"2024-03-30T13:09:49.938718+0000","last_clean":"2024-03-30T13:09:49.938718+0000","last_became_active":"2024-03-30T13:09:43.131462+0000","last_became_peered":"2024-03-30T13:09:43.131462+0000","last_unstale":"2024-03-30T13:09:49.938718+0000","last_undegraded":"2024-03-30T13:09:49.938718+0000","last_fullsized":"2024-03-30T13:09:49.938718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:49:52.247198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821188+0000","last_change":"2024-03-30T13:09:42.119614+0000","last_active":"2024-03-30T13:09:49.821188+0000","last_peered":"2024-03-30T13:09:49.821188+0000","last_clean":"2024-03-30T13:09:49.821188+0000","last_became_active":"2024-03-30T13:09:42.119252+0000","last_became_peered":"2024-03-30T13:09:42.119252+0000","last_unstale":"2024-03-30T13:09:49.821188+0000","last_undegraded":"2024-03-30T13:09:49.821188+0000","last_fullsized":"2024-03-30T13:09:49.821188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295320+0000","last_change":"2024-03-30T13:09:47.571383+0000","last_active":"2024-03-30T13:09:49.295320+0000","last_peered":"2024-03-30T13:09:49.295320+0000","last_clean":"2024-03-30T13:09:49.295320+0000","last_became_active":"2024-03-30T13:09:47.570998+0000","last_became_peered":"2024-03-30T13:09:47.570998+0000","last_unstale":"2024-03-30T13:09:49.295320+0000","last_undegraded":"2024-03-30T13:09:49.295320+0000","last_fullsized":"2024-03-30T13:09:49.295320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938152+0000","last_change":"2024-03-30T13:09:43.127405+0000","last_active":"2024-03-30T13:09:49.938152+0000","last_peered":"2024-03-30T13:09:49.938152+0000","last_clean":"2024-03-30T13:09:49.938152+0000","last_became_active":"2024-03-30T13:09:43.122914+0000","last_became_peered":"2024-03-30T13:09:43.122914+0000","last_unstale":"2024-03-30T13:09:49.938152+0000","last_undegraded":"2024-03-30T13:09:49.938152+0000","last_fullsized":"2024-03-30T13:09:49.938152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:51.506154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938749+0000","last_change":"2024-03-30T13:09:42.113530+0000","last_active":"2024-03-30T13:09:49.938749+0000","last_peered":"2024-03-30T13:09:49.938749+0000","last_clean":"2024-03-30T13:09:49.938749+0000","last_became_active":"2024-03-30T13:09:42.113114+0000","last_became_peered":"2024-03-30T13:09:42.113114+0000","last_unstale":"2024-03-30T13:09:49.938749+0000","last_undegraded":"2024-03-30T13:09:49.938749+0000","last_fullsized":"2024-03-30T13:09:49.938749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:59:09.359720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938814+0000","last_change":"2024-03-30T13:09:47.156241+0000","last_active":"2024-03-30T13:09:49.938814+0000","last_peered":"2024-03-30T13:09:49.938814+0000","last_clean":"2024-03-30T13:09:49.938814+0000","last_became_active":"2024-03-30T13:09:47.155884+0000","last_became_peered":"2024-03-30T13:09:47.155884+0000","last_unstale":"2024-03-30T13:09:49.938814+0000","last_undegraded":"2024-03-30T13:09:49.938814+0000","last_fullsized":"2024-03-30T13:09:49.938814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939042+0000","last_change":"2024-03-30T13:09:43.126029+0000","last_active":"2024-03-30T13:09:49.939042+0000","last_peered":"2024-03-30T13:09:49.939042+0000","last_clean":"2024-03-30T13:09:49.939042+0000","last_became_active":"2024-03-30T13:09:43.125724+0000","last_became_peered":"2024-03-30T13:09:43.125724+0000","last_unstale":"2024-03-30T13:09:49.939042+0000","last_undegraded":"2024-03-30T13:09:49.939042+0000","last_fullsized":"2024-03-30T13:09:49.939042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:44.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821402+0000","last_change":"2024-03-30T13:09:42.125370+0000","last_active":"2024-03-30T13:09:49.821402+0000","last_peered":"2024-03-30T13:09:49.821402+0000","last_clean":"2024-03-30T13:09:49.821402+0000","last_became_active":"2024-03-30T13:09:42.124450+0000","last_became_peered":"2024-03-30T13:09:42.124450+0000","last_unstale":"2024-03-30T13:09:49.821402+0000","last_undegraded":"2024-03-30T13:09:49.821402+0000","last_fullsized":"2024-03-30T13:09:49.821402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937709+0000","last_change":"2024-03-30T13:09:47.571108+0000","last_active":"2024-03-30T13:09:49.937709+0000","last_peered":"2024-03-30T13:09:49.937709+0000","last_clean":"2024-03-30T13:09:49.937709+0000","last_became_active":"2024-03-30T13:09:47.570725+0000","last_became_peered":"2024-03-30T13:09:47.570725+0000","last_unstale":"2024-03-30T13:09:49.937709+0000","last_undegraded":"2024-03-30T13:09:49.937709+0000","last_fullsized":"2024-03-30T13:09:49.937709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938937+0000","last_change":"2024-03-30T13:09:43.125121+0000","last_active":"2024-03-30T13:09:49.938937+0000","last_peered":"2024-03-30T13:09:49.938937+0000","last_clean":"2024-03-30T13:09:49.938937+0000","last_became_active":"2024-03-30T13:09:43.124795+0000","last_became_peered":"2024-03-30T13:09:43.124795+0000","last_unstale":"2024-03-30T13:09:49.938937+0000","last_undegraded":"2024-03-30T13:09:49.938937+0000","last_fullsized":"2024-03-30T13:09:49.938937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:55.220836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132420+0000","last_change":"2024-03-30T13:09:42.118818+0000","last_active":"2024-03-30T13:09:49.132420+0000","last_peered":"2024-03-30T13:09:49.132420+0000","last_clean":"2024-03-30T13:09:49.132420+0000","last_became_active":"2024-03-30T13:09:42.118529+0000","last_became_peered":"2024-03-30T13:09:42.118529+0000","last_unstale":"2024-03-30T13:09:49.132420+0000","last_undegraded":"2024-03-30T13:09:49.132420+0000","last_fullsized":"2024-03-30T13:09:49.132420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:40:03.615310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131582+0000","last_change":"2024-03-30T13:09:47.149385+0000","last_active":"2024-03-30T13:09:49.131582+0000","last_peered":"2024-03-30T13:09:49.131582+0000","last_clean":"2024-03-30T13:09:49.131582+0000","last_became_active":"2024-03-30T13:09:47.149030+0000","last_became_peered":"2024-03-30T13:09:47.149030+0000","last_unstale":"2024-03-30T13:09:49.131582+0000","last_undegraded":"2024-03-30T13:09:49.131582+0000","last_fullsized":"2024-03-30T13:09:49.131582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.118404+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.117853+0000","last_became_peered":"2024-03-30T13:09:43.117853+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820864+0000","last_change":"2024-03-30T13:09:42.117594+0000","last_active":"2024-03-30T13:09:49.820864+0000","last_peered":"2024-03-30T13:09:49.820864+0000","last_clean":"2024-03-30T13:09:49.820864+0000","last_became_active":"2024-03-30T13:09:42.117276+0000","last_became_peered":"2024-03-30T13:09:42.117276+0000","last_unstale":"2024-03-30T13:09:49.820864+0000","last_undegraded":"2024-03-30T13:09:49.820864+0000","last_fullsized":"2024-03-30T13:09:49.820864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939158+0000","last_change":"2024-03-30T13:09:47.155090+0000","last_active":"2024-03-30T13:09:49.939158+0000","last_peered":"2024-03-30T13:09:49.939158+0000","last_clean":"2024-03-30T13:09:49.939158+0000","last_became_active":"2024-03-30T13:09:47.154070+0000","last_became_peered":"2024-03-30T13:09:47.154070+0000","last_unstale":"2024-03-30T13:09:49.939158+0000","last_undegraded":"2024-03-30T13:09:49.939158+0000","last_fullsized":"2024-03-30T13:09:49.939158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295287+0000","last_change":"2024-03-30T13:09:43.127827+0000","last_active":"2024-03-30T13:09:49.295287+0000","last_peered":"2024-03-30T13:09:49.295287+0000","last_clean":"2024-03-30T13:09:49.295287+0000","last_became_active":"2024-03-30T13:09:43.127217+0000","last_became_peered":"2024-03-30T13:09:43.127217+0000","last_unstale":"2024-03-30T13:09:49.295287+0000","last_undegraded":"2024-03-30T13:09:49.295287+0000","last_fullsized":"2024-03-30T13:09:49.295287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295274+0000","last_change":"2024-03-30T13:09:42.112507+0000","last_active":"2024-03-30T13:09:49.295274+0000","last_peered":"2024-03-30T13:09:49.295274+0000","last_clean":"2024-03-30T13:09:49.295274+0000","last_became_active":"2024-03-30T13:09:42.112227+0000","last_became_peered":"2024-03-30T13:09:42.112227+0000","last_unstale":"2024-03-30T13:09:49.295274+0000","last_undegraded":"2024-03-30T13:09:49.295274+0000","last_fullsized":"2024-03-30T13:09:49.295274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133062+0000","last_change":"2024-03-30T13:09:47.572882+0000","last_active":"2024-03-30T13:09:49.133062+0000","last_peered":"2024-03-30T13:09:49.133062+0000","last_clean":"2024-03-30T13:09:49.133062+0000","last_became_active":"2024-03-30T13:09:47.572527+0000","last_became_peered":"2024-03-30T13:09:47.572527+0000","last_unstale":"2024-03-30T13:09:49.133062+0000","last_undegraded":"2024-03-30T13:09:49.133062+0000","last_fullsized":"2024-03-30T13:09:49.133062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:18.356964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938472+0000","last_change":"2024-03-30T13:09:43.125989+0000","last_active":"2024-03-30T13:09:49.938472+0000","last_peered":"2024-03-30T13:09:49.938472+0000","last_clean":"2024-03-30T13:09:49.938472+0000","last_became_active":"2024-03-30T13:09:43.125685+0000","last_became_peered":"2024-03-30T13:09:43.125685+0000","last_unstale":"2024-03-30T13:09:49.938472+0000","last_undegraded":"2024-03-30T13:09:49.938472+0000","last_fullsized":"2024-03-30T13:09:49.938472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:23.535266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938231+0000","last_change":"2024-03-30T13:09:42.122605+0000","last_active":"2024-03-30T13:09:49.938231+0000","last_peered":"2024-03-30T13:09:49.938231+0000","last_clean":"2024-03-30T13:09:49.938231+0000","last_became_active":"2024-03-30T13:09:42.122338+0000","last_became_peered":"2024-03-30T13:09:42.122338+0000","last_unstale":"2024-03-30T13:09:49.938231+0000","last_undegraded":"2024-03-30T13:09:49.938231+0000","last_fullsized":"2024-03-30T13:09:49.938231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133046+0000","last_change":"2024-03-30T13:09:47.571300+0000","last_active":"2024-03-30T13:09:49.133046+0000","last_peered":"2024-03-30T13:09:49.133046+0000","last_clean":"2024-03-30T13:09:49.133046+0000","last_became_active":"2024-03-30T13:09:47.570892+0000","last_became_peered":"2024-03-30T13:09:47.570892+0000","last_unstale":"2024-03-30T13:09:49.133046+0000","last_undegraded":"2024-03-30T13:09:49.133046+0000","last_fullsized":"2024-03-30T13:09:49.133046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:09.284589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938281+0000","last_change":"2024-03-30T13:09:43.125239+0000","last_active":"2024-03-30T13:09:49.938281+0000","last_peered":"2024-03-30T13:09:49.938281+0000","last_clean":"2024-03-30T13:09:49.938281+0000","last_became_active":"2024-03-30T13:09:43.124815+0000","last_became_peered":"2024-03-30T13:09:43.124815+0000","last_unstale":"2024-03-30T13:09:49.938281+0000","last_undegraded":"2024-03-30T13:09:49.938281+0000","last_fullsized":"2024-03-30T13:09:49.938281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939006+0000","last_change":"2024-03-30T13:09:42.113307+0000","last_active":"2024-03-30T13:09:49.939006+0000","last_peered":"2024-03-30T13:09:49.939006+0000","last_clean":"2024-03-30T13:09:49.939006+0000","last_became_active":"2024-03-30T13:09:42.112968+0000","last_became_peered":"2024-03-30T13:09:42.112968+0000","last_unstale":"2024-03-30T13:09:49.939006+0000","last_undegraded":"2024-03-30T13:09:49.939006+0000","last_fullsized":"2024-03-30T13:09:49.939006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938380+0000","last_change":"2024-03-30T13:09:47.572011+0000","last_active":"2024-03-30T13:09:49.938380+0000","last_peered":"2024-03-30T13:09:49.938380+0000","last_clean":"2024-03-30T13:09:49.938380+0000","last_became_active":"2024-03-30T13:09:47.571632+0000","last_became_peered":"2024-03-30T13:09:47.571632+0000","last_unstale":"2024-03-30T13:09:49.938380+0000","last_undegraded":"2024-03-30T13:09:49.938380+0000","last_fullsized":"2024-03-30T13:09:49.938380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938714+0000","last_change":"2024-03-30T13:09:43.129993+0000","last_active":"2024-03-30T13:09:49.938714+0000","last_peered":"2024-03-30T13:09:49.938714+0000","last_clean":"2024-03-30T13:09:49.938714+0000","last_became_active":"2024-03-30T13:09:43.128265+0000","last_became_peered":"2024-03-30T13:09:43.128265+0000","last_unstale":"2024-03-30T13:09:49.938714+0000","last_undegraded":"2024-03-30T13:09:49.938714+0000","last_fullsized":"2024-03-30T13:09:49.938714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:28:32.691360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:52.955617+0000","last_change":"2024-03-30T13:09:42.120865+0000","last_active":"2024-03-30T13:09:52.955617+0000","last_peered":"2024-03-30T13:09:52.955617+0000","last_clean":"2024-03-30T13:09:52.955617+0000","last_became_active":"2024-03-30T13:09:42.120589+0000","last_became_peered":"2024-03-30T13:09:42.120589+0000","last_unstale":"2024-03-30T13:09:52.955617+0000","last_undegraded":"2024-03-30T13:09:52.955617+0000","last_fullsized":"2024-03-30T13:09:52.955617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:03.359370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938224+0000","last_change":"2024-03-30T13:09:47.155432+0000","last_active":"2024-03-30T13:09:49.938224+0000","last_peered":"2024-03-30T13:09:49.938224+0000","last_clean":"2024-03-30T13:09:49.938224+0000","last_became_active":"2024-03-30T13:09:47.155044+0000","last_became_peered":"2024-03-30T13:09:47.155044+0000","last_unstale":"2024-03-30T13:09:49.938224+0000","last_undegraded":"2024-03-30T13:09:49.938224+0000","last_fullsized":"2024-03-30T13:09:49.938224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:16.148467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132299+0000","last_change":"2024-03-30T13:09:43.115819+0000","last_active":"2024-03-30T13:09:49.132299+0000","last_peered":"2024-03-30T13:09:49.132299+0000","last_clean":"2024-03-30T13:09:49.132299+0000","last_became_active":"2024-03-30T13:09:43.115563+0000","last_became_peered":"2024-03-30T13:09:43.115563+0000","last_unstale":"2024-03-30T13:09:49.132299+0000","last_undegraded":"2024-03-30T13:09:49.132299+0000","last_fullsized":"2024-03-30T13:09:49.132299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:59.552427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938506+0000","last_change":"2024-03-30T13:09:42.121667+0000","last_active":"2024-03-30T13:09:49.938506+0000","last_peered":"2024-03-30T13:09:49.938506+0000","last_clean":"2024-03-30T13:09:49.938506+0000","last_became_active":"2024-03-30T13:09:42.121384+0000","last_became_peered":"2024-03-30T13:09:42.121384+0000","last_unstale":"2024-03-30T13:09:49.938506+0000","last_undegraded":"2024-03-30T13:09:49.938506+0000","last_fullsized":"2024-03-30T13:09:49.938506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132454+0000","last_change":"2024-03-30T13:09:47.147513+0000","last_active":"2024-03-30T13:09:49.132454+0000","last_peered":"2024-03-30T13:09:49.132454+0000","last_clean":"2024-03-30T13:09:49.132454+0000","last_became_active":"2024-03-30T13:09:47.147126+0000","last_became_peered":"2024-03-30T13:09:47.147126+0000","last_unstale":"2024-03-30T13:09:49.132454+0000","last_undegraded":"2024-03-30T13:09:49.132454+0000","last_fullsized":"2024-03-30T13:09:49.132454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939064+0000","last_change":"2024-03-30T13:09:43.130550+0000","last_active":"2024-03-30T13:09:49.939064+0000","last_peered":"2024-03-30T13:09:49.939064+0000","last_clean":"2024-03-30T13:09:49.939064+0000","last_became_active":"2024-03-30T13:09:43.130297+0000","last_became_peered":"2024-03-30T13:09:43.130297+0000","last_unstale":"2024-03-30T13:09:49.939064+0000","last_undegraded":"2024-03-30T13:09:49.939064+0000","last_fullsized":"2024-03-30T13:09:49.939064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938180+0000","last_change":"2024-03-30T13:09:42.122699+0000","last_active":"2024-03-30T13:09:49.938180+0000","last_peered":"2024-03-30T13:09:49.938180+0000","last_clean":"2024-03-30T13:09:49.938180+0000","last_became_active":"2024-03-30T13:09:42.122407+0000","last_became_peered":"2024-03-30T13:09:42.122407+0000","last_unstale":"2024-03-30T13:09:49.938180+0000","last_undegraded":"2024-03-30T13:09:49.938180+0000","last_fullsized":"2024-03-30T13:09:49.938180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132389+0000","last_change":"2024-03-30T13:09:47.160317+0000","last_active":"2024-03-30T13:09:49.132389+0000","last_peered":"2024-03-30T13:09:49.132389+0000","last_clean":"2024-03-30T13:09:49.132389+0000","last_became_active":"2024-03-30T13:09:47.159818+0000","last_became_peered":"2024-03-30T13:09:47.159818+0000","last_unstale":"2024-03-30T13:09:49.132389+0000","last_undegraded":"2024-03-30T13:09:49.132389+0000","last_fullsized":"2024-03-30T13:09:49.132389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:22:15.869872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938158+0000","last_change":"2024-03-30T13:09:43.128528+0000","last_active":"2024-03-30T13:09:49.938158+0000","last_peered":"2024-03-30T13:09:49.938158+0000","last_clean":"2024-03-30T13:09:49.938158+0000","last_became_active":"2024-03-30T13:09:43.128239+0000","last_became_peered":"2024-03-30T13:09:43.128239+0000","last_unstale":"2024-03-30T13:09:49.938158+0000","last_undegraded":"2024-03-30T13:09:49.938158+0000","last_fullsized":"2024-03-30T13:09:49.938158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:38:56.141275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295744+0000","last_change":"2024-03-30T13:09:42.118540+0000","last_active":"2024-03-30T13:09:49.295744+0000","last_peered":"2024-03-30T13:09:49.295744+0000","last_clean":"2024-03-30T13:09:49.295744+0000","last_became_active":"2024-03-30T13:09:42.118090+0000","last_became_peered":"2024-03-30T13:09:42.118090+0000","last_unstale":"2024-03-30T13:09:49.295744+0000","last_undegraded":"2024-03-30T13:09:49.295744+0000","last_fullsized":"2024-03-30T13:09:49.295744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132435+0000","last_change":"2024-03-30T13:09:47.571367+0000","last_active":"2024-03-30T13:09:49.132435+0000","last_peered":"2024-03-30T13:09:49.132435+0000","last_clean":"2024-03-30T13:09:49.132435+0000","last_became_active":"2024-03-30T13:09:47.570973+0000","last_became_peered":"2024-03-30T13:09:47.570973+0000","last_unstale":"2024-03-30T13:09:49.132435+0000","last_undegraded":"2024-03-30T13:09:49.132435+0000","last_fullsized":"2024-03-30T13:09:49.132435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:33:15.965915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939033+0000","last_change":"2024-03-30T13:09:47.572196+0000","last_active":"2024-03-30T13:09:49.939033+0000","last_peered":"2024-03-30T13:09:49.939033+0000","last_clean":"2024-03-30T13:09:49.939033+0000","last_became_active":"2024-03-30T13:09:47.571378+0000","last_became_peered":"2024-03-30T13:09:47.571378+0000","last_unstale":"2024-03-30T13:09:49.939033+0000","last_undegraded":"2024-03-30T13:09:49.939033+0000","last_fullsized":"2024-03-30T13:09:49.939033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938039+0000","last_change":"2024-03-30T13:09:43.125653+0000","last_active":"2024-03-30T13:09:49.938039+0000","last_peered":"2024-03-30T13:09:49.938039+0000","last_clean":"2024-03-30T13:09:49.938039+0000","last_became_active":"2024-03-30T13:09:43.125382+0000","last_became_peered":"2024-03-30T13:09:43.125382+0000","last_unstale":"2024-03-30T13:09:49.938039+0000","last_undegraded":"2024-03-30T13:09:49.938039+0000","last_fullsized":"2024-03-30T13:09:49.938039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:25:10.034049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937816+0000","last_change":"2024-03-30T13:09:42.115855+0000","last_active":"2024-03-30T13:09:49.937816+0000","last_peered":"2024-03-30T13:09:49.937816+0000","last_clean":"2024-03-30T13:09:49.937816+0000","last_became_active":"2024-03-30T13:09:42.115369+0000","last_became_peered":"2024-03-30T13:09:42.115369+0000","last_unstale":"2024-03-30T13:09:49.937816+0000","last_undegraded":"2024-03-30T13:09:49.937816+0000","last_fullsized":"2024-03-30T13:09:49.937816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295715+0000","last_change":"2024-03-30T13:09:47.156344+0000","last_active":"2024-03-30T13:09:49.295715+0000","last_peered":"2024-03-30T13:09:49.295715+0000","last_clean":"2024-03-30T13:09:49.295715+0000","last_became_active":"2024-03-30T13:09:47.155917+0000","last_became_peered":"2024-03-30T13:09:47.155917+0000","last_unstale":"2024-03-30T13:09:49.295715+0000","last_undegraded":"2024-03-30T13:09:49.295715+0000","last_fullsized":"2024-03-30T13:09:49.295715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132295+0000","last_change":"2024-03-30T13:09:43.128241+0000","last_active":"2024-03-30T13:09:49.132295+0000","last_peered":"2024-03-30T13:09:49.132295+0000","last_clean":"2024-03-30T13:09:49.132295+0000","last_became_active":"2024-03-30T13:09:43.127971+0000","last_became_peered":"2024-03-30T13:09:43.127971+0000","last_unstale":"2024-03-30T13:09:49.132295+0000","last_undegraded":"2024-03-30T13:09:49.132295+0000","last_fullsized":"2024-03-30T13:09:49.132295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132655+0000","last_change":"2024-03-30T13:09:42.122509+0000","last_active":"2024-03-30T13:09:49.132655+0000","last_peered":"2024-03-30T13:09:49.132655+0000","last_clean":"2024-03-30T13:09:49.132655+0000","last_became_active":"2024-03-30T13:09:42.122130+0000","last_became_peered":"2024-03-30T13:09:42.122130+0000","last_unstale":"2024-03-30T13:09:49.132655+0000","last_undegraded":"2024-03-30T13:09:49.132655+0000","last_fullsized":"2024-03-30T13:09:49.132655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295373+0000","last_change":"2024-03-30T13:09:42.115453+0000","last_active":"2024-03-30T13:09:49.295373+0000","last_peered":"2024-03-30T13:09:49.295373+0000","last_clean":"2024-03-30T13:09:49.295373+0000","last_became_active":"2024-03-30T13:09:42.115182+0000","last_became_peered":"2024-03-30T13:09:42.115182+0000","last_unstale":"2024-03-30T13:09:49.295373+0000","last_undegraded":"2024-03-30T13:09:49.295373+0000","last_fullsized":"2024-03-30T13:09:49.295373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:47:12.895083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938966+0000","last_change":"2024-03-30T13:09:43.127679+0000","last_active":"2024-03-30T13:09:49.938966+0000","last_peered":"2024-03-30T13:09:49.938966+0000","last_clean":"2024-03-30T13:09:49.938966+0000","last_became_active":"2024-03-30T13:09:43.127421+0000","last_became_peered":"2024-03-30T13:09:43.127421+0000","last_unstale":"2024-03-30T13:09:49.938966+0000","last_undegraded":"2024-03-30T13:09:49.938966+0000","last_fullsized":"2024-03-30T13:09:49.938966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821070+0000","last_change":"2024-03-30T13:09:47.154699+0000","last_active":"2024-03-30T13:09:49.821070+0000","last_peered":"2024-03-30T13:09:49.821070+0000","last_clean":"2024-03-30T13:09:49.821070+0000","last_became_active":"2024-03-30T13:09:47.154291+0000","last_became_peered":"2024-03-30T13:09:47.154291+0000","last_unstale":"2024-03-30T13:09:49.821070+0000","last_undegraded":"2024-03-30T13:09:49.821070+0000","last_fullsized":"2024-03-30T13:09:49.821070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:56:47.829620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133085+0000","last_change":"2024-03-30T13:09:42.124644+0000","last_active":"2024-03-30T13:09:49.133085+0000","last_peered":"2024-03-30T13:09:49.133085+0000","last_clean":"2024-03-30T13:09:49.133085+0000","last_became_active":"2024-03-30T13:09:42.124355+0000","last_became_peered":"2024-03-30T13:09:42.124355+0000","last_unstale":"2024-03-30T13:09:49.133085+0000","last_undegraded":"2024-03-30T13:09:49.133085+0000","last_fullsized":"2024-03-30T13:09:49.133085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132040+0000","last_change":"2024-03-30T13:09:43.113299+0000","last_active":"2024-03-30T13:09:49.132040+0000","last_peered":"2024-03-30T13:09:49.132040+0000","last_clean":"2024-03-30T13:09:49.132040+0000","last_became_active":"2024-03-30T13:09:43.113031+0000","last_became_peered":"2024-03-30T13:09:43.113031+0000","last_unstale":"2024-03-30T13:09:49.132040+0000","last_undegraded":"2024-03-30T13:09:49.132040+0000","last_fullsized":"2024-03-30T13:09:49.132040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:32.715713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132575+0000","last_change":"2024-03-30T13:09:47.156912+0000","last_active":"2024-03-30T13:09:49.132575+0000","last_peered":"2024-03-30T13:09:49.132575+0000","last_clean":"2024-03-30T13:09:49.132575+0000","last_became_active":"2024-03-30T13:09:47.156492+0000","last_became_peered":"2024-03-30T13:09:47.156492+0000","last_unstale":"2024-03-30T13:09:49.132575+0000","last_undegraded":"2024-03-30T13:09:49.132575+0000","last_fullsized":"2024-03-30T13:09:49.132575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:55:36.886194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295186+0000","last_change":"2024-03-30T13:09:43.127904+0000","last_active":"2024-03-30T13:09:49.295186+0000","last_peered":"2024-03-30T13:09:49.295186+0000","last_clean":"2024-03-30T13:09:49.295186+0000","last_became_active":"2024-03-30T13:09:43.127563+0000","last_became_peered":"2024-03-30T13:09:43.127563+0000","last_unstale":"2024-03-30T13:09:49.295186+0000","last_undegraded":"2024-03-30T13:09:49.295186+0000","last_fullsized":"2024-03-30T13:09:49.295186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820892+0000","last_change":"2024-03-30T13:09:43.131561+0000","last_active":"2024-03-30T13:09:49.820892+0000","last_peered":"2024-03-30T13:09:49.820892+0000","last_clean":"2024-03-30T13:09:49.820892+0000","last_became_active":"2024-03-30T13:09:43.129703+0000","last_became_peered":"2024-03-30T13:09:43.129703+0000","last_unstale":"2024-03-30T13:09:49.820892+0000","last_undegraded":"2024-03-30T13:09:49.820892+0000","last_fullsized":"2024-03-30T13:09:49.820892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:46.613583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938131+0000","last_change":"2024-03-30T13:09:43.129678+0000","last_active":"2024-03-30T13:09:49.938131+0000","last_peered":"2024-03-30T13:09:49.938131+0000","last_clean":"2024-03-30T13:09:49.938131+0000","last_became_active":"2024-03-30T13:09:43.129405+0000","last_became_peered":"2024-03-30T13:09:43.129405+0000","last_unstale":"2024-03-30T13:09:49.938131+0000","last_undegraded":"2024-03-30T13:09:49.938131+0000","last_fullsized":"2024-03-30T13:09:49.938131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:13:29.545084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938913+0000","last_change":"2024-03-30T13:09:43.135202+0000","last_active":"2024-03-30T13:09:49.938913+0000","last_peered":"2024-03-30T13:09:49.938913+0000","last_clean":"2024-03-30T13:09:49.938913+0000","last_became_active":"2024-03-30T13:09:43.134895+0000","last_became_peered":"2024-03-30T13:09:43.134895+0000","last_unstale":"2024-03-30T13:09:49.938913+0000","last_undegraded":"2024-03-30T13:09:49.938913+0000","last_fullsized":"2024-03-30T13:09:49.938913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133173+0000","last_change":"2024-03-30T13:09:43.127414+0000","last_active":"2024-03-30T13:09:49.133173+0000","last_peered":"2024-03-30T13:09:49.133173+0000","last_clean":"2024-03-30T13:09:49.133173+0000","last_became_active":"2024-03-30T13:09:43.127077+0000","last_became_peered":"2024-03-30T13:09:43.127077+0000","last_unstale":"2024-03-30T13:09:49.133173+0000","last_undegraded":"2024-03-30T13:09:49.133173+0000","last_fullsized":"2024-03-30T13:09:49.133173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938324+0000","last_change":"2024-03-30T13:09:43.119060+0000","last_active":"2024-03-30T13:09:49.938324+0000","last_peered":"2024-03-30T13:09:49.938324+0000","last_clean":"2024-03-30T13:09:49.938324+0000","last_became_active":"2024-03-30T13:09:43.118585+0000","last_became_peered":"2024-03-30T13:09:43.118585+0000","last_unstale":"2024-03-30T13:09:49.938324+0000","last_undegraded":"2024-03-30T13:09:49.938324+0000","last_fullsized":"2024-03-30T13:09:49.938324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295833+0000","last_change":"2024-03-30T13:09:43.122310+0000","last_active":"2024-03-30T13:09:49.295833+0000","last_peered":"2024-03-30T13:09:49.295833+0000","last_clean":"2024-03-30T13:09:49.295833+0000","last_became_active":"2024-03-30T13:09:43.121999+0000","last_became_peered":"2024-03-30T13:09:43.121999+0000","last_unstale":"2024-03-30T13:09:49.295833+0000","last_undegraded":"2024-03-30T13:09:49.295833+0000","last_fullsized":"2024-03-30T13:09:49.295833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133127+0000","last_change":"2024-03-30T13:09:43.122476+0000","last_active":"2024-03-30T13:09:49.133127+0000","last_peered":"2024-03-30T13:09:49.133127+0000","last_clean":"2024-03-30T13:09:49.133127+0000","last_became_active":"2024-03-30T13:09:43.120315+0000","last_became_peered":"2024-03-30T13:09:43.120315+0000","last_unstale":"2024-03-30T13:09:49.133127+0000","last_undegraded":"2024-03-30T13:09:49.133127+0000","last_fullsized":"2024-03-30T13:09:49.133127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937780+0000","last_change":"2024-03-30T13:09:43.128432+0000","last_active":"2024-03-30T13:09:49.937780+0000","last_peered":"2024-03-30T13:09:49.937780+0000","last_clean":"2024-03-30T13:09:49.937780+0000","last_became_active":"2024-03-30T13:09:43.128013+0000","last_became_peered":"2024-03-30T13:09:43.128013+0000","last_unstale":"2024-03-30T13:09:49.937780+0000","last_undegraded":"2024-03-30T13:09:49.937780+0000","last_fullsized":"2024-03-30T13:09:49.937780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132485+0000","last_change":"2024-03-30T13:09:43.120476+0000","last_active":"2024-03-30T13:09:49.132485+0000","last_peered":"2024-03-30T13:09:49.132485+0000","last_clean":"2024-03-30T13:09:49.132485+0000","last_became_active":"2024-03-30T13:09:43.120212+0000","last_became_peered":"2024-03-30T13:09:43.120212+0000","last_unstale":"2024-03-30T13:09:49.132485+0000","last_undegraded":"2024-03-30T13:09:49.132485+0000","last_fullsized":"2024-03-30T13:09:49.132485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:50:23.461880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938025+0000","last_change":"2024-03-30T13:09:43.137167+0000","last_active":"2024-03-30T13:09:49.938025+0000","last_peered":"2024-03-30T13:09:49.938025+0000","last_clean":"2024-03-30T13:09:49.938025+0000","last_became_active":"2024-03-30T13:09:43.136900+0000","last_became_peered":"2024-03-30T13:09:43.136900+0000","last_unstale":"2024-03-30T13:09:49.938025+0000","last_undegraded":"2024-03-30T13:09:49.938025+0000","last_fullsized":"2024-03-30T13:09:49.938025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:15:29.671082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124147+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.123860+0000","last_became_peered":"2024-03-30T13:09:43.123860+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:32:55.194498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295535+0000","last_change":"2024-03-30T13:09:43.127624+0000","last_active":"2024-03-30T13:09:49.295535+0000","last_peered":"2024-03-30T13:09:49.295535+0000","last_clean":"2024-03-30T13:09:49.295535+0000","last_became_active":"2024-03-30T13:09:43.127340+0000","last_became_peered":"2024-03-30T13:09:43.127340+0000","last_unstale":"2024-03-30T13:09:49.295535+0000","last_undegraded":"2024-03-30T13:09:49.295535+0000","last_fullsized":"2024-03-30T13:09:49.295535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938857+0000","last_change":"2024-03-30T13:09:43.122985+0000","last_active":"2024-03-30T13:09:49.938857+0000","last_peered":"2024-03-30T13:09:49.938857+0000","last_clean":"2024-03-30T13:09:49.938857+0000","last_became_active":"2024-03-30T13:09:43.122689+0000","last_became_peered":"2024-03-30T13:09:43.122689+0000","last_unstale":"2024-03-30T13:09:49.938857+0000","last_undegraded":"2024-03-30T13:09:49.938857+0000","last_fullsized":"2024-03-30T13:09:49.938857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:41.817012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821341+0000","last_change":"2024-03-30T13:09:43.131062+0000","last_active":"2024-03-30T13:09:49.821341+0000","last_peered":"2024-03-30T13:09:49.821341+0000","last_clean":"2024-03-30T13:09:49.821341+0000","last_became_active":"2024-03-30T13:09:43.130727+0000","last_became_peered":"2024-03-30T13:09:43.130727+0000","last_unstale":"2024-03-30T13:09:49.821341+0000","last_undegraded":"2024-03-30T13:09:49.821341+0000","last_fullsized":"2024-03-30T13:09:49.821341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:03:31.612660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938468+0000","last_change":"2024-03-30T13:09:43.137027+0000","last_active":"2024-03-30T13:09:49.938468+0000","last_peered":"2024-03-30T13:09:49.938468+0000","last_clean":"2024-03-30T13:09:49.938468+0000","last_became_active":"2024-03-30T13:09:43.136755+0000","last_became_peered":"2024-03-30T13:09:43.136755+0000","last_unstale":"2024-03-30T13:09:49.938468+0000","last_undegraded":"2024-03-30T13:09:49.938468+0000","last_fullsized":"2024-03-30T13:09:49.938468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:38:26.087735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937690+0000","last_change":"2024-03-30T13:09:43.132197+0000","last_active":"2024-03-30T13:09:49.937690+0000","last_peered":"2024-03-30T13:09:49.937690+0000","last_clean":"2024-03-30T13:09:49.937690+0000","last_became_active":"2024-03-30T13:09:43.131927+0000","last_became_peered":"2024-03-30T13:09:43.131927+0000","last_unstale":"2024-03-30T13:09:49.937690+0000","last_undegraded":"2024-03-30T13:09:49.937690+0000","last_fullsized":"2024-03-30T13:09:49.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938139+0000","last_change":"2024-03-30T13:09:43.109275+0000","last_active":"2024-03-30T13:09:49.938139+0000","last_peered":"2024-03-30T13:09:49.938139+0000","last_clean":"2024-03-30T13:09:49.938139+0000","last_became_active":"2024-03-30T13:09:43.108964+0000","last_became_peered":"2024-03-30T13:09:43.108964+0000","last_unstale":"2024-03-30T13:09:49.938139+0000","last_undegraded":"2024-03-30T13:09:49.938139+0000","last_fullsized":"2024-03-30T13:09:49.938139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:48.847357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938228+0000","last_change":"2024-03-30T13:09:43.121915+0000","last_active":"2024-03-30T13:09:49.938228+0000","last_peered":"2024-03-30T13:09:49.938228+0000","last_clean":"2024-03-30T13:09:49.938228+0000","last_became_active":"2024-03-30T13:09:43.121625+0000","last_became_peered":"2024-03-30T13:09:43.121625+0000","last_unstale":"2024-03-30T13:09:49.938228+0000","last_undegraded":"2024-03-30T13:09:49.938228+0000","last_fullsized":"2024-03-30T13:09:49.938228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132890+0000","last_change":"2024-03-30T13:09:43.131187+0000","last_active":"2024-03-30T13:09:49.132890+0000","last_peered":"2024-03-30T13:09:49.132890+0000","last_clean":"2024-03-30T13:09:49.132890+0000","last_became_active":"2024-03-30T13:09:43.130682+0000","last_became_peered":"2024-03-30T13:09:43.130682+0000","last_unstale":"2024-03-30T13:09:49.132890+0000","last_undegraded":"2024-03-30T13:09:49.132890+0000","last_fullsized":"2024-03-30T13:09:49.132890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132240+0000","last_change":"2024-03-30T13:09:43.123005+0000","last_active":"2024-03-30T13:09:49.132240+0000","last_peered":"2024-03-30T13:09:49.132240+0000","last_clean":"2024-03-30T13:09:49.132240+0000","last_became_active":"2024-03-30T13:09:43.122761+0000","last_became_peered":"2024-03-30T13:09:43.122761+0000","last_unstale":"2024-03-30T13:09:49.132240+0000","last_undegraded":"2024-03-30T13:09:49.132240+0000","last_fullsized":"2024-03-30T13:09:49.132240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:48.062694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295578+0000","last_change":"2024-03-30T13:09:43.122715+0000","last_active":"2024-03-30T13:09:49.295578+0000","last_peered":"2024-03-30T13:09:49.295578+0000","last_clean":"2024-03-30T13:09:49.295578+0000","last_became_active":"2024-03-30T13:09:43.122445+0000","last_became_peered":"2024-03-30T13:09:43.122445+0000","last_unstale":"2024-03-30T13:09:49.295578+0000","last_undegraded":"2024-03-30T13:09:49.295578+0000","last_fullsized":"2024-03-30T13:09:49.295578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295783+0000","last_change":"2024-03-30T13:09:43.130345+0000","last_active":"2024-03-30T13:09:49.295783+0000","last_peered":"2024-03-30T13:09:49.295783+0000","last_clean":"2024-03-30T13:09:49.295783+0000","last_became_active":"2024-03-30T13:09:43.130062+0000","last_became_peered":"2024-03-30T13:09:43.130062+0000","last_unstale":"2024-03-30T13:09:49.295783+0000","last_undegraded":"2024-03-30T13:09:49.295783+0000","last_fullsized":"2024-03-30T13:09:49.295783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939037+0000","last_change":"2024-03-30T13:09:43.135018+0000","last_active":"2024-03-30T13:09:49.939037+0000","last_peered":"2024-03-30T13:09:49.939037+0000","last_clean":"2024-03-30T13:09:49.939037+0000","last_became_active":"2024-03-30T13:09:43.134354+0000","last_became_peered":"2024-03-30T13:09:43.134354+0000","last_unstale":"2024-03-30T13:09:49.939037+0000","last_undegraded":"2024-03-30T13:09:49.939037+0000","last_fullsized":"2024-03-30T13:09:49.939037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:04.849718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132188+0000","last_change":"2024-03-30T13:09:43.123158+0000","last_active":"2024-03-30T13:09:49.132188+0000","last_peered":"2024-03-30T13:09:49.132188+0000","last_clean":"2024-03-30T13:09:49.132188+0000","last_became_active":"2024-03-30T13:09:43.122830+0000","last_became_peered":"2024-03-30T13:09:43.122830+0000","last_unstale":"2024-03-30T13:09:49.132188+0000","last_undegraded":"2024-03-30T13:09:49.132188+0000","last_fullsized":"2024-03-30T13:09:49.132188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124628+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.124364+0000","last_became_peered":"2024-03-30T13:09:43.124364+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937937+0000","last_change":"2024-03-30T13:09:43.132387+0000","last_active":"2024-03-30T13:09:49.937937+0000","last_peered":"2024-03-30T13:09:49.937937+0000","last_clean":"2024-03-30T13:09:49.937937+0000","last_became_active":"2024-03-30T13:09:43.132110+0000","last_became_peered":"2024-03-30T13:09:43.132110+0000","last_unstale":"2024-03-30T13:09:49.937937+0000","last_undegraded":"2024-03-30T13:09:49.937937+0000","last_fullsized":"2024-03-30T13:09:49.937937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938308+0000","last_change":"2024-03-30T13:09:43.124528+0000","last_active":"2024-03-30T13:09:49.938308+0000","last_peered":"2024-03-30T13:09:49.938308+0000","last_clean":"2024-03-30T13:09:49.938308+0000","last_became_active":"2024-03-30T13:09:43.124023+0000","last_became_peered":"2024-03-30T13:09:43.124023+0000","last_unstale":"2024-03-30T13:09:49.938308+0000","last_undegraded":"2024-03-30T13:09:49.938308+0000","last_fullsized":"2024-03-30T13:09:49.938308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938061+0000","last_change":"2024-03-30T13:09:43.127663+0000","last_active":"2024-03-30T13:09:49.938061+0000","last_peered":"2024-03-30T13:09:49.938061+0000","last_clean":"2024-03-30T13:09:49.938061+0000","last_became_active":"2024-03-30T13:09:43.127180+0000","last_became_peered":"2024-03-30T13:09:43.127180+0000","last_unstale":"2024-03-30T13:09:49.938061+0000","last_undegraded":"2024-03-30T13:09:49.938061+0000","last_fullsized":"2024-03-30T13:09:49.938061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:36:33.418013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.121807+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.121498+0000","last_became_peered":"2024-03-30T13:09:43.121498+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938554+0000","last_change":"2024-03-30T13:09:43.124024+0000","last_active":"2024-03-30T13:09:49.938554+0000","last_peered":"2024-03-30T13:09:49.938554+0000","last_clean":"2024-03-30T13:09:49.938554+0000","last_became_active":"2024-03-30T13:09:43.123748+0000","last_became_peered":"2024-03-30T13:09:43.123748+0000","last_unstale":"2024-03-30T13:09:49.938554+0000","last_undegraded":"2024-03-30T13:09:49.938554+0000","last_fullsized":"2024-03-30T13:09:49.938554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938980+0000","last_change":"2024-03-30T13:09:43.122707+0000","last_active":"2024-03-30T13:09:49.938980+0000","last_peered":"2024-03-30T13:09:49.938980+0000","last_clean":"2024-03-30T13:09:49.938980+0000","last_became_active":"2024-03-30T13:09:43.122418+0000","last_became_peered":"2024-03-30T13:09:43.122418+0000","last_unstale":"2024-03-30T13:09:49.938980+0000","last_undegraded":"2024-03-30T13:09:49.938980+0000","last_fullsized":"2024-03-30T13:09:49.938980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:49.970892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":700230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":109406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":109280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":109440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":109370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:10:06.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:10:06.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.336+0000 7fdd2b8eb640 1 Processor -- start 2024-03-30T13:10:06.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.336+0000 7fdd2b8eb640 1 -- start start 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd2b8eb640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdd240d1390 0x7fdd240d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd2b8eb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd240b3570 0x7fdd240b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd2b8eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd2b8eb640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fdd240ba3d0 con 0x7fdd240d1390 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd2b8eb640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fdd240d1cb0 con 0x7fdd240b3570 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd2b8eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240cf6e0 con 0x7fdd240b39d0 2024-03-30T13:10:06.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdd240d1390 0x7fdd240d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdd240d1390 0x7fdd240d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34146/0 (socket says 172.21.15.66:34146) 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 -- 172.21.15.66:0/1625224043 learned_addr learned my addr 172.21.15.66:0/1625224043 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29e88640 1 --2- 172.21.15.66:0/1625224043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd28e86640 1 --2- 172.21.15.66:0/1625224043 >> v2:172.21.15.117:3300/0 conn(0x7fdd240b3570 0x7fdd240b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 -- 172.21.15.66:0/1625224043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 msgr2=0x7fdd240b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 --2- 172.21.15.66:0/1625224043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 -- 172.21.15.66:0/1625224043 >> v2:172.21.15.117:3300/0 conn(0x7fdd240b3570 msgr2=0x7fdd240b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 --2- 172.21.15.66:0/1625224043 >> v2:172.21.15.117:3300/0 conn(0x7fdd240b3570 0x7fdd240b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.337+0000 7fdd29687640 1 -- 172.21.15.66:0/1625224043 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd240b9050 con 0x7fdd240d1390 2024-03-30T13:10:06.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.338+0000 7fdd29687640 1 --2- 172.21.15.66:0/1625224043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd240d1770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd240b63b0 tx=0x7fdd1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1563c67a433d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.338+0000 7fdd23fff640 1 -- 172.21.15.66:0/1625224043 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c034070 con 0x7fdd240d1390 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.338+0000 7fdd23fff640 1 -- 172.21.15.66:0/1625224043 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c030860 con 0x7fdd240d1390 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.338+0000 7fdd23fff640 1 -- 172.21.15.66:0/1625224043 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c030c30 con 0x7fdd240d1390 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/1625224043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 msgr2=0x7fdd240d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/1625224043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd240d1770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd240b63b0 tx=0x7fdd1c02b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/1625224043 shutdown_connections 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/1625224043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/1625224043 >> v2:172.21.15.117:3300/0 conn(0x7fdd240b3570 0x7fdd240b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/1625224043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd240d1770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/1625224043 >> 172.21.15.66:0/1625224043 conn(0x7fdd240676e0 msgr2=0x7fdd24067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/1625224043 shutdown_connections 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.339+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/1625224043 wait complete. 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.340+0000 7fdd2b8eb640 1 Processor -- start 2024-03-30T13:10:06.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.340+0000 7fdd2b8eb640 1 -- start start 2024-03-30T13:10:06.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd2b8eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd240b3570 0x7fdd24152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd2b8eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd2b8eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd24160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd2b8eb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd240b98f0 con 0x7fdd240d1390 2024-03-30T13:10:06.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd2b8eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd240b9610 con 0x7fdd240b3570 2024-03-30T13:10:06.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd2b8eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdd240b9bd0 con 0x7fdd240b39d0 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd24160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd24160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34162/0 (socket says 172.21.15.66:34162) 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 -- 172.21.15.66:0/3968549187 learned_addr learned my addr 172.21.15.66:0/3968549187 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd28e86640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29687640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd240b3570 0x7fdd24152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 -- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 msgr2=0x7fdd240c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 -- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd240b3570 msgr2=0x7fdd24152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd240b3570 0x7fdd24152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.341+0000 7fdd29e88640 1 -- 172.21.15.66:0/3968549187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd240ca470 con 0x7fdd240d1390 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.342+0000 7fdd29e88640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd24160cf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400efc0 tx=0x7fdd1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.342+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14018020 con 0x7fdd240d1390 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.342+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd14014480 con 0x7fdd240d1390 2024-03-30T13:10:06.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.342+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14014870 con 0x7fdd240d1390 2024-03-30T13:10:06.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.342+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd240ca750 con 0x7fdd240d1390 2024-03-30T13:10:06.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.342+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd240cad10 con 0x7fdd240d1390 2024-03-30T13:10:06.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.343+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd240d5190 con 0x7fdd240d1390 2024-03-30T13:10:06.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.343+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdd14005d10 con 0x7fdd240d1390 2024-03-30T13:10:06.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.344+0000 7fdd21ffb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdd1006c560 0x7fdd1006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.344+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdd1408c390 con 0x7fdd240d1390 2024-03-30T13:10:06.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.344+0000 7fdd29687640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdd1006c560 0x7fdd1006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.344+0000 7fdd29687640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdd1006c560 0x7fdd1006ea20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036000 tx=0x7fdd1c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:06.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.348+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdd240d5190 con 0x7fdd240d1390 2024-03-30T13:10:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.531+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd24075a10 con 0x7fdd1006c560 2024-03-30T13:10:06.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.538+0000 7fdd21ffb640 1 -- 172.21.15.66:0/3968549187 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fdd24075a10 con 0x7fdd1006c560 2024-03-30T13:10:06.546 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:06.548 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:10:06.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdd1006c560 msgr2=0x7fdd1006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdd1006c560 0x7fdd1006ea20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036000 tx=0x7fdd1c03b040 comp rx=0 tx=0).stop 2024-03-30T13:10:06.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 msgr2=0x7fdd24160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd24160cf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400efc0 tx=0x7fdd1400c490 comp rx=0 tx=0).stop 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 shutdown_connections 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdd1006c560 0x7fdd1006ea20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd240d1390 0x7fdd24160cf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd240b39d0 0x7fdd240c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 --2- 172.21.15.66:0/3968549187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd240b3570 0x7fdd24152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 >> 172.21.15.66:0/3968549187 conn(0x7fdd240676e0 msgr2=0x7fdd240b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 shutdown_connections 2024-03-30T13:10:06.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.542+0000 7fdd2b8eb640 1 -- 172.21.15.66:0/3968549187 wait complete. 2024-03-30T13:10:06.558 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-30T13:10:04.798334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219500,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755220,"statfs":{"total":773094113280,"available":772869345280,"internally_reserved":0,"allocated":3416064,"data_stored":2064256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002545"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939160+0000","last_change":"2024-03-30T13:09:42.124458+0000","last_active":"2024-03-30T13:09:49.939160+0000","last_peered":"2024-03-30T13:09:49.939160+0000","last_clean":"2024-03-30T13:09:49.939160+0000","last_became_active":"2024-03-30T13:09:42.124158+0000","last_became_peered":"2024-03-30T13:09:42.124158+0000","last_unstale":"2024-03-30T13:09:49.939160+0000","last_undegraded":"2024-03-30T13:09:49.939160+0000","last_fullsized":"2024-03-30T13:09:49.939160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132823+0000","last_change":"2024-03-30T13:09:43.127433+0000","last_active":"2024-03-30T13:09:49.132823+0000","last_peered":"2024-03-30T13:09:49.132823+0000","last_clean":"2024-03-30T13:09:49.132823+0000","last_became_active":"2024-03-30T13:09:43.127127+0000","last_became_peered":"2024-03-30T13:09:43.127127+0000","last_unstale":"2024-03-30T13:09:49.132823+0000","last_undegraded":"2024-03-30T13:09:49.132823+0000","last_fullsized":"2024-03-30T13:09:49.132823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132184+0000","last_change":"2024-03-30T13:09:47.572430+0000","last_active":"2024-03-30T13:09:49.132184+0000","last_peered":"2024-03-30T13:09:49.132184+0000","last_clean":"2024-03-30T13:09:49.132184+0000","last_became_active":"2024-03-30T13:09:47.572059+0000","last_became_peered":"2024-03-30T13:09:47.572059+0000","last_unstale":"2024-03-30T13:09:49.132184+0000","last_undegraded":"2024-03-30T13:09:49.132184+0000","last_fullsized":"2024-03-30T13:09:49.132184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:31:37.218184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132943+0000","last_change":"2024-03-30T13:09:42.126685+0000","last_active":"2024-03-30T13:09:49.132943+0000","last_peered":"2024-03-30T13:09:49.132943+0000","last_clean":"2024-03-30T13:09:49.132943+0000","last_became_active":"2024-03-30T13:09:42.126383+0000","last_became_peered":"2024-03-30T13:09:42.126383+0000","last_unstale":"2024-03-30T13:09:49.132943+0000","last_undegraded":"2024-03-30T13:09:49.132943+0000","last_fullsized":"2024-03-30T13:09:49.132943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938945+0000","last_change":"2024-03-30T13:09:43.132452+0000","last_active":"2024-03-30T13:09:49.938945+0000","last_peered":"2024-03-30T13:09:49.938945+0000","last_clean":"2024-03-30T13:09:49.938945+0000","last_became_active":"2024-03-30T13:09:43.132158+0000","last_became_peered":"2024-03-30T13:09:43.132158+0000","last_unstale":"2024-03-30T13:09:49.938945+0000","last_undegraded":"2024-03-30T13:09:49.938945+0000","last_fullsized":"2024-03-30T13:09:49.938945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131666+0000","last_change":"2024-03-30T13:09:47.157090+0000","last_active":"2024-03-30T13:09:49.131666+0000","last_peered":"2024-03-30T13:09:49.131666+0000","last_clean":"2024-03-30T13:09:49.131666+0000","last_became_active":"2024-03-30T13:09:47.156676+0000","last_became_peered":"2024-03-30T13:09:47.156676+0000","last_unstale":"2024-03-30T13:09:49.131666+0000","last_undegraded":"2024-03-30T13:09:49.131666+0000","last_fullsized":"2024-03-30T13:09:49.131666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938826+0000","last_change":"2024-03-30T13:09:42.127138+0000","last_active":"2024-03-30T13:09:49.938826+0000","last_peered":"2024-03-30T13:09:49.938826+0000","last_clean":"2024-03-30T13:09:49.938826+0000","last_became_active":"2024-03-30T13:09:42.126853+0000","last_became_peered":"2024-03-30T13:09:42.126853+0000","last_unstale":"2024-03-30T13:09:49.938826+0000","last_undegraded":"2024-03-30T13:09:49.938826+0000","last_fullsized":"2024-03-30T13:09:49.938826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938126+0000","last_change":"2024-03-30T13:09:43.137336+0000","last_active":"2024-03-30T13:09:49.938126+0000","last_peered":"2024-03-30T13:09:49.938126+0000","last_clean":"2024-03-30T13:09:49.938126+0000","last_became_active":"2024-03-30T13:09:43.137057+0000","last_became_peered":"2024-03-30T13:09:43.137057+0000","last_unstale":"2024-03-30T13:09:49.938126+0000","last_undegraded":"2024-03-30T13:09:49.938126+0000","last_fullsized":"2024-03-30T13:09:49.938126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:20.284742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132127+0000","last_change":"2024-03-30T13:09:47.149659+0000","last_active":"2024-03-30T13:09:49.132127+0000","last_peered":"2024-03-30T13:09:49.132127+0000","last_clean":"2024-03-30T13:09:49.132127+0000","last_became_active":"2024-03-30T13:09:47.149298+0000","last_became_peered":"2024-03-30T13:09:47.149298+0000","last_unstale":"2024-03-30T13:09:49.132127+0000","last_undegraded":"2024-03-30T13:09:49.132127+0000","last_fullsized":"2024-03-30T13:09:49.132127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:48.863738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938057+0000","last_change":"2024-03-30T13:09:42.126941+0000","last_active":"2024-03-30T13:09:49.938057+0000","last_peered":"2024-03-30T13:09:49.938057+0000","last_clean":"2024-03-30T13:09:49.938057+0000","last_became_active":"2024-03-30T13:09:42.126646+0000","last_became_peered":"2024-03-30T13:09:42.126646+0000","last_unstale":"2024-03-30T13:09:49.938057+0000","last_undegraded":"2024-03-30T13:09:49.938057+0000","last_fullsized":"2024-03-30T13:09:49.938057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938364+0000","last_change":"2024-03-30T13:09:43.129721+0000","last_active":"2024-03-30T13:09:49.938364+0000","last_peered":"2024-03-30T13:09:49.938364+0000","last_clean":"2024-03-30T13:09:49.938364+0000","last_became_active":"2024-03-30T13:09:43.128999+0000","last_became_peered":"2024-03-30T13:09:43.128999+0000","last_unstale":"2024-03-30T13:09:49.938364+0000","last_undegraded":"2024-03-30T13:09:49.938364+0000","last_fullsized":"2024-03-30T13:09:49.938364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:28:48.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131877+0000","last_change":"2024-03-30T13:09:47.149841+0000","last_active":"2024-03-30T13:09:49.131877+0000","last_peered":"2024-03-30T13:09:49.131877+0000","last_clean":"2024-03-30T13:09:49.131877+0000","last_became_active":"2024-03-30T13:09:47.148965+0000","last_became_peered":"2024-03-30T13:09:47.148965+0000","last_unstale":"2024-03-30T13:09:49.131877+0000","last_undegraded":"2024-03-30T13:09:49.131877+0000","last_fullsized":"2024-03-30T13:09:49.131877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:08.872490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295198+0000","last_change":"2024-03-30T13:09:43.127058+0000","last_active":"2024-03-30T13:09:49.295198+0000","last_peered":"2024-03-30T13:09:49.295198+0000","last_clean":"2024-03-30T13:09:49.295198+0000","last_became_active":"2024-03-30T13:09:43.126777+0000","last_became_peered":"2024-03-30T13:09:43.126777+0000","last_unstale":"2024-03-30T13:09:49.295198+0000","last_undegraded":"2024-03-30T13:09:49.295198+0000","last_fullsized":"2024-03-30T13:09:49.295198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938548+0000","last_change":"2024-03-30T13:09:42.124505+0000","last_active":"2024-03-30T13:09:49.938548+0000","last_peered":"2024-03-30T13:09:49.938548+0000","last_clean":"2024-03-30T13:09:49.938548+0000","last_became_active":"2024-03-30T13:09:42.124243+0000","last_became_peered":"2024-03-30T13:09:42.124243+0000","last_unstale":"2024-03-30T13:09:49.938548+0000","last_undegraded":"2024-03-30T13:09:49.938548+0000","last_fullsized":"2024-03-30T13:09:49.938548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:47.055563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132399+0000","last_change":"2024-03-30T13:09:47.146122+0000","last_active":"2024-03-30T13:09:49.132399+0000","last_peered":"2024-03-30T13:09:49.132399+0000","last_clean":"2024-03-30T13:09:49.132399+0000","last_became_active":"2024-03-30T13:09:47.145870+0000","last_became_peered":"2024-03-30T13:09:47.145870+0000","last_unstale":"2024-03-30T13:09:49.132399+0000","last_undegraded":"2024-03-30T13:09:49.132399+0000","last_fullsized":"2024-03-30T13:09:49.132399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132985+0000","last_change":"2024-03-30T13:09:43.130941+0000","last_active":"2024-03-30T13:09:49.132985+0000","last_peered":"2024-03-30T13:09:49.132985+0000","last_clean":"2024-03-30T13:09:49.132985+0000","last_became_active":"2024-03-30T13:09:43.130335+0000","last_became_peered":"2024-03-30T13:09:43.130335+0000","last_unstale":"2024-03-30T13:09:49.132985+0000","last_undegraded":"2024-03-30T13:09:49.132985+0000","last_fullsized":"2024-03-30T13:09:49.132985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938369+0000","last_change":"2024-03-30T13:09:42.124102+0000","last_active":"2024-03-30T13:09:49.938369+0000","last_peered":"2024-03-30T13:09:49.938369+0000","last_clean":"2024-03-30T13:09:49.938369+0000","last_became_active":"2024-03-30T13:09:42.123746+0000","last_became_peered":"2024-03-30T13:09:42.123746+0000","last_unstale":"2024-03-30T13:09:49.938369+0000","last_undegraded":"2024-03-30T13:09:49.938369+0000","last_fullsized":"2024-03-30T13:09:49.938369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132249+0000","last_change":"2024-03-30T13:09:47.153343+0000","last_active":"2024-03-30T13:09:49.132249+0000","last_peered":"2024-03-30T13:09:49.132249+0000","last_clean":"2024-03-30T13:09:49.132249+0000","last_became_active":"2024-03-30T13:09:47.152894+0000","last_became_peered":"2024-03-30T13:09:47.152894+0000","last_unstale":"2024-03-30T13:09:49.132249+0000","last_undegraded":"2024-03-30T13:09:49.132249+0000","last_fullsized":"2024-03-30T13:09:49.132249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132418+0000","last_change":"2024-03-30T13:09:43.117715+0000","last_active":"2024-03-30T13:09:49.132418+0000","last_peered":"2024-03-30T13:09:49.132418+0000","last_clean":"2024-03-30T13:09:49.132418+0000","last_became_active":"2024-03-30T13:09:43.116768+0000","last_became_peered":"2024-03-30T13:09:43.116768+0000","last_unstale":"2024-03-30T13:09:49.132418+0000","last_undegraded":"2024-03-30T13:09:49.132418+0000","last_fullsized":"2024-03-30T13:09:49.132418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938237+0000","last_change":"2024-03-30T13:09:42.125545+0000","last_active":"2024-03-30T13:09:49.938237+0000","last_peered":"2024-03-30T13:09:49.938237+0000","last_clean":"2024-03-30T13:09:49.938237+0000","last_became_active":"2024-03-30T13:09:42.125280+0000","last_became_peered":"2024-03-30T13:09:42.125280+0000","last_unstale":"2024-03-30T13:09:49.938237+0000","last_undegraded":"2024-03-30T13:09:49.938237+0000","last_fullsized":"2024-03-30T13:09:49.938237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938821+0000","last_change":"2024-03-30T13:09:47.571186+0000","last_active":"2024-03-30T13:09:49.938821+0000","last_peered":"2024-03-30T13:09:49.938821+0000","last_clean":"2024-03-30T13:09:49.938821+0000","last_became_active":"2024-03-30T13:09:47.570755+0000","last_became_peered":"2024-03-30T13:09:47.570755+0000","last_unstale":"2024-03-30T13:09:49.938821+0000","last_undegraded":"2024-03-30T13:09:49.938821+0000","last_fullsized":"2024-03-30T13:09:49.938821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131859+0000","last_change":"2024-03-30T13:09:43.115171+0000","last_active":"2024-03-30T13:09:49.131859+0000","last_peered":"2024-03-30T13:09:49.131859+0000","last_clean":"2024-03-30T13:09:49.131859+0000","last_became_active":"2024-03-30T13:09:43.113821+0000","last_became_peered":"2024-03-30T13:09:43.113821+0000","last_unstale":"2024-03-30T13:09:49.131859+0000","last_undegraded":"2024-03-30T13:09:49.131859+0000","last_fullsized":"2024-03-30T13:09:49.131859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:04.366501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938219+0000","last_change":"2024-03-30T13:09:42.125753+0000","last_active":"2024-03-30T13:09:49.938219+0000","last_peered":"2024-03-30T13:09:49.938219+0000","last_clean":"2024-03-30T13:09:49.938219+0000","last_became_active":"2024-03-30T13:09:42.125505+0000","last_became_peered":"2024-03-30T13:09:42.125505+0000","last_unstale":"2024-03-30T13:09:49.938219+0000","last_undegraded":"2024-03-30T13:09:49.938219+0000","last_fullsized":"2024-03-30T13:09:49.938219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:43:24.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132292+0000","last_change":"2024-03-30T13:09:47.149806+0000","last_active":"2024-03-30T13:09:49.132292+0000","last_peered":"2024-03-30T13:09:49.132292+0000","last_clean":"2024-03-30T13:09:49.132292+0000","last_became_active":"2024-03-30T13:09:47.149427+0000","last_became_peered":"2024-03-30T13:09:47.149427+0000","last_unstale":"2024-03-30T13:09:49.132292+0000","last_undegraded":"2024-03-30T13:09:49.132292+0000","last_fullsized":"2024-03-30T13:09:49.132292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939301+0000","last_change":"2024-03-30T13:09:43.122135+0000","last_active":"2024-03-30T13:09:49.939301+0000","last_peered":"2024-03-30T13:09:49.939301+0000","last_clean":"2024-03-30T13:09:49.939301+0000","last_became_active":"2024-03-30T13:09:43.121877+0000","last_became_peered":"2024-03-30T13:09:43.121877+0000","last_unstale":"2024-03-30T13:09:49.939301+0000","last_undegraded":"2024-03-30T13:09:49.939301+0000","last_fullsized":"2024-03-30T13:09:49.939301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295852+0000","last_change":"2024-03-30T13:09:42.124452+0000","last_active":"2024-03-30T13:09:49.295852+0000","last_peered":"2024-03-30T13:09:49.295852+0000","last_clean":"2024-03-30T13:09:49.295852+0000","last_became_active":"2024-03-30T13:09:42.124188+0000","last_became_peered":"2024-03-30T13:09:42.124188+0000","last_unstale":"2024-03-30T13:09:49.295852+0000","last_undegraded":"2024-03-30T13:09:49.295852+0000","last_fullsized":"2024-03-30T13:09:49.295852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131805+0000","last_change":"2024-03-30T13:09:47.571007+0000","last_active":"2024-03-30T13:09:49.131805+0000","last_peered":"2024-03-30T13:09:49.131805+0000","last_clean":"2024-03-30T13:09:49.131805+0000","last_became_active":"2024-03-30T13:09:47.570641+0000","last_became_peered":"2024-03-30T13:09:47.570641+0000","last_unstale":"2024-03-30T13:09:49.131805+0000","last_undegraded":"2024-03-30T13:09:49.131805+0000","last_fullsized":"2024-03-30T13:09:49.131805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133129+0000","last_change":"2024-03-30T13:09:43.123324+0000","last_active":"2024-03-30T13:09:49.133129+0000","last_peered":"2024-03-30T13:09:49.133129+0000","last_clean":"2024-03-30T13:09:49.133129+0000","last_became_active":"2024-03-30T13:09:43.122970+0000","last_became_peered":"2024-03-30T13:09:43.122970+0000","last_unstale":"2024-03-30T13:09:49.133129+0000","last_undegraded":"2024-03-30T13:09:49.133129+0000","last_fullsized":"2024-03-30T13:09:49.133129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938610+0000","last_change":"2024-03-30T13:09:42.112467+0000","last_active":"2024-03-30T13:09:49.938610+0000","last_peered":"2024-03-30T13:09:49.938610+0000","last_clean":"2024-03-30T13:09:49.938610+0000","last_became_active":"2024-03-30T13:09:42.112119+0000","last_became_peered":"2024-03-30T13:09:42.112119+0000","last_unstale":"2024-03-30T13:09:49.938610+0000","last_undegraded":"2024-03-30T13:09:49.938610+0000","last_fullsized":"2024-03-30T13:09:49.938610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:45.606451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132205+0000","last_change":"2024-03-30T13:09:47.571907+0000","last_active":"2024-03-30T13:09:49.132205+0000","last_peered":"2024-03-30T13:09:49.132205+0000","last_clean":"2024-03-30T13:09:49.132205+0000","last_became_active":"2024-03-30T13:09:47.571466+0000","last_became_peered":"2024-03-30T13:09:47.571466+0000","last_unstale":"2024-03-30T13:09:49.132205+0000","last_undegraded":"2024-03-30T13:09:49.132205+0000","last_fullsized":"2024-03-30T13:09:49.132205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:36:14.998133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938844+0000","last_change":"2024-03-30T13:09:43.114536+0000","last_active":"2024-03-30T13:09:49.938844+0000","last_peered":"2024-03-30T13:09:49.938844+0000","last_clean":"2024-03-30T13:09:49.938844+0000","last_became_active":"2024-03-30T13:09:43.114268+0000","last_became_peered":"2024-03-30T13:09:43.114268+0000","last_unstale":"2024-03-30T13:09:49.938844+0000","last_undegraded":"2024-03-30T13:09:49.938844+0000","last_fullsized":"2024-03-30T13:09:49.938844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938931+0000","last_change":"2024-03-30T13:09:42.125506+0000","last_active":"2024-03-30T13:09:49.938931+0000","last_peered":"2024-03-30T13:09:49.938931+0000","last_clean":"2024-03-30T13:09:49.938931+0000","last_became_active":"2024-03-30T13:09:42.125297+0000","last_became_peered":"2024-03-30T13:09:42.125297+0000","last_unstale":"2024-03-30T13:09:49.938931+0000","last_undegraded":"2024-03-30T13:09:49.938931+0000","last_fullsized":"2024-03-30T13:09:49.938931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:58.545296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938573+0000","last_change":"2024-03-30T13:09:47.570730+0000","last_active":"2024-03-30T13:09:49.938573+0000","last_peered":"2024-03-30T13:09:49.938573+0000","last_clean":"2024-03-30T13:09:49.938573+0000","last_became_active":"2024-03-30T13:09:47.570239+0000","last_became_peered":"2024-03-30T13:09:47.570239+0000","last_unstale":"2024-03-30T13:09:49.938573+0000","last_undegraded":"2024-03-30T13:09:49.938573+0000","last_fullsized":"2024-03-30T13:09:49.938573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938464+0000","last_change":"2024-03-30T13:09:43.127560+0000","last_active":"2024-03-30T13:09:49.938464+0000","last_peered":"2024-03-30T13:09:49.938464+0000","last_clean":"2024-03-30T13:09:49.938464+0000","last_became_active":"2024-03-30T13:09:43.127245+0000","last_became_peered":"2024-03-30T13:09:43.127245+0000","last_unstale":"2024-03-30T13:09:49.938464+0000","last_undegraded":"2024-03-30T13:09:49.938464+0000","last_fullsized":"2024-03-30T13:09:49.938464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:37.548735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938862+0000","last_change":"2024-03-30T13:09:42.116193+0000","last_active":"2024-03-30T13:09:49.938862+0000","last_peered":"2024-03-30T13:09:49.938862+0000","last_clean":"2024-03-30T13:09:49.938862+0000","last_became_active":"2024-03-30T13:09:42.115609+0000","last_became_peered":"2024-03-30T13:09:42.115609+0000","last_unstale":"2024-03-30T13:09:49.938862+0000","last_undegraded":"2024-03-30T13:09:49.938862+0000","last_fullsized":"2024-03-30T13:09:49.938862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938004+0000","last_change":"2024-03-30T13:09:47.159058+0000","last_active":"2024-03-30T13:09:49.938004+0000","last_peered":"2024-03-30T13:09:49.938004+0000","last_clean":"2024-03-30T13:09:49.938004+0000","last_became_active":"2024-03-30T13:09:47.158646+0000","last_became_peered":"2024-03-30T13:09:47.158646+0000","last_unstale":"2024-03-30T13:09:49.938004+0000","last_undegraded":"2024-03-30T13:09:49.938004+0000","last_fullsized":"2024-03-30T13:09:49.938004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:51.919084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132471+0000","last_change":"2024-03-30T13:09:43.121780+0000","last_active":"2024-03-30T13:09:49.132471+0000","last_peered":"2024-03-30T13:09:49.132471+0000","last_clean":"2024-03-30T13:09:49.132471+0000","last_became_active":"2024-03-30T13:09:43.121519+0000","last_became_peered":"2024-03-30T13:09:43.121519+0000","last_unstale":"2024-03-30T13:09:49.132471+0000","last_undegraded":"2024-03-30T13:09:49.132471+0000","last_fullsized":"2024-03-30T13:09:49.132471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:48.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938327+0000","last_change":"2024-03-30T13:09:42.122015+0000","last_active":"2024-03-30T13:09:49.938327+0000","last_peered":"2024-03-30T13:09:49.938327+0000","last_clean":"2024-03-30T13:09:49.938327+0000","last_became_active":"2024-03-30T13:09:42.121721+0000","last_became_peered":"2024-03-30T13:09:42.121721+0000","last_unstale":"2024-03-30T13:09:49.938327+0000","last_undegraded":"2024-03-30T13:09:49.938327+0000","last_fullsized":"2024-03-30T13:09:49.938327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938675+0000","last_change":"2024-03-30T13:09:47.159041+0000","last_active":"2024-03-30T13:09:49.938675+0000","last_peered":"2024-03-30T13:09:49.938675+0000","last_clean":"2024-03-30T13:09:49.938675+0000","last_became_active":"2024-03-30T13:09:47.158466+0000","last_became_peered":"2024-03-30T13:09:47.158466+0000","last_unstale":"2024-03-30T13:09:49.938675+0000","last_undegraded":"2024-03-30T13:09:49.938675+0000","last_fullsized":"2024-03-30T13:09:49.938675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938050+0000","last_change":"2024-03-30T13:09:43.111943+0000","last_active":"2024-03-30T13:09:49.938050+0000","last_peered":"2024-03-30T13:09:49.938050+0000","last_clean":"2024-03-30T13:09:49.938050+0000","last_became_active":"2024-03-30T13:09:43.111677+0000","last_became_peered":"2024-03-30T13:09:43.111677+0000","last_unstale":"2024-03-30T13:09:49.938050+0000","last_undegraded":"2024-03-30T13:09:49.938050+0000","last_fullsized":"2024-03-30T13:09:49.938050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:33:22.810971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821050+0000","last_change":"2024-03-30T13:09:42.121133+0000","last_active":"2024-03-30T13:09:49.821050+0000","last_peered":"2024-03-30T13:09:49.821050+0000","last_clean":"2024-03-30T13:09:49.821050+0000","last_became_active":"2024-03-30T13:09:42.120851+0000","last_became_peered":"2024-03-30T13:09:42.120851+0000","last_unstale":"2024-03-30T13:09:49.821050+0000","last_undegraded":"2024-03-30T13:09:49.821050+0000","last_fullsized":"2024-03-30T13:09:49.821050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938890+0000","last_change":"2024-03-30T13:09:47.155307+0000","last_active":"2024-03-30T13:09:49.938890+0000","last_peered":"2024-03-30T13:09:49.938890+0000","last_clean":"2024-03-30T13:09:49.938890+0000","last_became_active":"2024-03-30T13:09:47.154875+0000","last_became_peered":"2024-03-30T13:09:47.154875+0000","last_unstale":"2024-03-30T13:09:49.938890+0000","last_undegraded":"2024-03-30T13:09:49.938890+0000","last_fullsized":"2024-03-30T13:09:49.938890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:17:44.422352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937941+0000","last_change":"2024-03-30T13:09:43.128352+0000","last_active":"2024-03-30T13:09:49.937941+0000","last_peered":"2024-03-30T13:09:49.937941+0000","last_clean":"2024-03-30T13:09:49.937941+0000","last_became_active":"2024-03-30T13:09:43.128073+0000","last_became_peered":"2024-03-30T13:09:43.128073+0000","last_unstale":"2024-03-30T13:09:49.937941+0000","last_undegraded":"2024-03-30T13:09:49.937941+0000","last_fullsized":"2024-03-30T13:09:49.937941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:41.389848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938326+0000","last_change":"2024-03-30T13:09:42.124601+0000","last_active":"2024-03-30T13:09:49.938326+0000","last_peered":"2024-03-30T13:09:49.938326+0000","last_clean":"2024-03-30T13:09:49.938326+0000","last_became_active":"2024-03-30T13:09:42.124335+0000","last_became_peered":"2024-03-30T13:09:42.124335+0000","last_unstale":"2024-03-30T13:09:49.938326+0000","last_undegraded":"2024-03-30T13:09:49.938326+0000","last_fullsized":"2024-03-30T13:09:49.938326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:02:06.738441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939206+0000","last_change":"2024-03-30T13:09:47.572608+0000","last_active":"2024-03-30T13:09:49.939206+0000","last_peered":"2024-03-30T13:09:49.939206+0000","last_clean":"2024-03-30T13:09:49.939206+0000","last_became_active":"2024-03-30T13:09:47.571945+0000","last_became_peered":"2024-03-30T13:09:47.571945+0000","last_unstale":"2024-03-30T13:09:49.939206+0000","last_undegraded":"2024-03-30T13:09:49.939206+0000","last_fullsized":"2024-03-30T13:09:49.939206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:06:37.115753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132459+0000","last_change":"2024-03-30T13:09:43.125963+0000","last_active":"2024-03-30T13:09:49.132459+0000","last_peered":"2024-03-30T13:09:49.132459+0000","last_clean":"2024-03-30T13:09:49.132459+0000","last_became_active":"2024-03-30T13:09:43.125682+0000","last_became_peered":"2024-03-30T13:09:43.125682+0000","last_unstale":"2024-03-30T13:09:49.132459+0000","last_undegraded":"2024-03-30T13:09:49.132459+0000","last_fullsized":"2024-03-30T13:09:49.132459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938090+0000","last_change":"2024-03-30T13:09:42.119717+0000","last_active":"2024-03-30T13:09:49.938090+0000","last_peered":"2024-03-30T13:09:49.938090+0000","last_clean":"2024-03-30T13:09:49.938090+0000","last_became_active":"2024-03-30T13:09:42.119185+0000","last_became_peered":"2024-03-30T13:09:42.119185+0000","last_unstale":"2024-03-30T13:09:49.938090+0000","last_undegraded":"2024-03-30T13:09:49.938090+0000","last_fullsized":"2024-03-30T13:09:49.938090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938410+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:09:49.938410+0000","last_peered":"2024-03-30T13:09:49.938410+0000","last_clean":"2024-03-30T13:09:49.938410+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:09:49.938410+0000","last_undegraded":"2024-03-30T13:09:49.938410+0000","last_fullsized":"2024-03-30T13:09:49.938410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295610+0000","last_change":"2024-03-30T13:09:47.572345+0000","last_active":"2024-03-30T13:09:49.295610+0000","last_peered":"2024-03-30T13:09:49.295610+0000","last_clean":"2024-03-30T13:09:49.295610+0000","last_became_active":"2024-03-30T13:09:47.571956+0000","last_became_peered":"2024-03-30T13:09:47.571956+0000","last_unstale":"2024-03-30T13:09:49.295610+0000","last_undegraded":"2024-03-30T13:09:49.295610+0000","last_fullsized":"2024-03-30T13:09:49.295610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938718+0000","last_change":"2024-03-30T13:09:43.134581+0000","last_active":"2024-03-30T13:09:49.938718+0000","last_peered":"2024-03-30T13:09:49.938718+0000","last_clean":"2024-03-30T13:09:49.938718+0000","last_became_active":"2024-03-30T13:09:43.131462+0000","last_became_peered":"2024-03-30T13:09:43.131462+0000","last_unstale":"2024-03-30T13:09:49.938718+0000","last_undegraded":"2024-03-30T13:09:49.938718+0000","last_fullsized":"2024-03-30T13:09:49.938718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:49:52.247198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821188+0000","last_change":"2024-03-30T13:09:42.119614+0000","last_active":"2024-03-30T13:09:49.821188+0000","last_peered":"2024-03-30T13:09:49.821188+0000","last_clean":"2024-03-30T13:09:49.821188+0000","last_became_active":"2024-03-30T13:09:42.119252+0000","last_became_peered":"2024-03-30T13:09:42.119252+0000","last_unstale":"2024-03-30T13:09:49.821188+0000","last_undegraded":"2024-03-30T13:09:49.821188+0000","last_fullsized":"2024-03-30T13:09:49.821188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295320+0000","last_change":"2024-03-30T13:09:47.571383+0000","last_active":"2024-03-30T13:09:49.295320+0000","last_peered":"2024-03-30T13:09:49.295320+0000","last_clean":"2024-03-30T13:09:49.295320+0000","last_became_active":"2024-03-30T13:09:47.570998+0000","last_became_peered":"2024-03-30T13:09:47.570998+0000","last_unstale":"2024-03-30T13:09:49.295320+0000","last_undegraded":"2024-03-30T13:09:49.295320+0000","last_fullsized":"2024-03-30T13:09:49.295320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938152+0000","last_change":"2024-03-30T13:09:43.127405+0000","last_active":"2024-03-30T13:09:49.938152+0000","last_peered":"2024-03-30T13:09:49.938152+0000","last_clean":"2024-03-30T13:09:49.938152+0000","last_became_active":"2024-03-30T13:09:43.122914+0000","last_became_peered":"2024-03-30T13:09:43.122914+0000","last_unstale":"2024-03-30T13:09:49.938152+0000","last_undegraded":"2024-03-30T13:09:49.938152+0000","last_fullsized":"2024-03-30T13:09:49.938152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:51.506154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938749+0000","last_change":"2024-03-30T13:09:42.113530+0000","last_active":"2024-03-30T13:09:49.938749+0000","last_peered":"2024-03-30T13:09:49.938749+0000","last_clean":"2024-03-30T13:09:49.938749+0000","last_became_active":"2024-03-30T13:09:42.113114+0000","last_became_peered":"2024-03-30T13:09:42.113114+0000","last_unstale":"2024-03-30T13:09:49.938749+0000","last_undegraded":"2024-03-30T13:09:49.938749+0000","last_fullsized":"2024-03-30T13:09:49.938749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:59:09.359720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938814+0000","last_change":"2024-03-30T13:09:47.156241+0000","last_active":"2024-03-30T13:09:49.938814+0000","last_peered":"2024-03-30T13:09:49.938814+0000","last_clean":"2024-03-30T13:09:49.938814+0000","last_became_active":"2024-03-30T13:09:47.155884+0000","last_became_peered":"2024-03-30T13:09:47.155884+0000","last_unstale":"2024-03-30T13:09:49.938814+0000","last_undegraded":"2024-03-30T13:09:49.938814+0000","last_fullsized":"2024-03-30T13:09:49.938814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939042+0000","last_change":"2024-03-30T13:09:43.126029+0000","last_active":"2024-03-30T13:09:49.939042+0000","last_peered":"2024-03-30T13:09:49.939042+0000","last_clean":"2024-03-30T13:09:49.939042+0000","last_became_active":"2024-03-30T13:09:43.125724+0000","last_became_peered":"2024-03-30T13:09:43.125724+0000","last_unstale":"2024-03-30T13:09:49.939042+0000","last_undegraded":"2024-03-30T13:09:49.939042+0000","last_fullsized":"2024-03-30T13:09:49.939042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:44.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821402+0000","last_change":"2024-03-30T13:09:42.125370+0000","last_active":"2024-03-30T13:09:49.821402+0000","last_peered":"2024-03-30T13:09:49.821402+0000","last_clean":"2024-03-30T13:09:49.821402+0000","last_became_active":"2024-03-30T13:09:42.124450+0000","last_became_peered":"2024-03-30T13:09:42.124450+0000","last_unstale":"2024-03-30T13:09:49.821402+0000","last_undegraded":"2024-03-30T13:09:49.821402+0000","last_fullsized":"2024-03-30T13:09:49.821402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937709+0000","last_change":"2024-03-30T13:09:47.571108+0000","last_active":"2024-03-30T13:09:49.937709+0000","last_peered":"2024-03-30T13:09:49.937709+0000","last_clean":"2024-03-30T13:09:49.937709+0000","last_became_active":"2024-03-30T13:09:47.570725+0000","last_became_peered":"2024-03-30T13:09:47.570725+0000","last_unstale":"2024-03-30T13:09:49.937709+0000","last_undegraded":"2024-03-30T13:09:49.937709+0000","last_fullsized":"2024-03-30T13:09:49.937709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938937+0000","last_change":"2024-03-30T13:09:43.125121+0000","last_active":"2024-03-30T13:09:49.938937+0000","last_peered":"2024-03-30T13:09:49.938937+0000","last_clean":"2024-03-30T13:09:49.938937+0000","last_became_active":"2024-03-30T13:09:43.124795+0000","last_became_peered":"2024-03-30T13:09:43.124795+0000","last_unstale":"2024-03-30T13:09:49.938937+0000","last_undegraded":"2024-03-30T13:09:49.938937+0000","last_fullsized":"2024-03-30T13:09:49.938937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:55.220836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132420+0000","last_change":"2024-03-30T13:09:42.118818+0000","last_active":"2024-03-30T13:09:49.132420+0000","last_peered":"2024-03-30T13:09:49.132420+0000","last_clean":"2024-03-30T13:09:49.132420+0000","last_became_active":"2024-03-30T13:09:42.118529+0000","last_became_peered":"2024-03-30T13:09:42.118529+0000","last_unstale":"2024-03-30T13:09:49.132420+0000","last_undegraded":"2024-03-30T13:09:49.132420+0000","last_fullsized":"2024-03-30T13:09:49.132420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:40:03.615310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131582+0000","last_change":"2024-03-30T13:09:47.149385+0000","last_active":"2024-03-30T13:09:49.131582+0000","last_peered":"2024-03-30T13:09:49.131582+0000","last_clean":"2024-03-30T13:09:49.131582+0000","last_became_active":"2024-03-30T13:09:47.149030+0000","last_became_peered":"2024-03-30T13:09:47.149030+0000","last_unstale":"2024-03-30T13:09:49.131582+0000","last_undegraded":"2024-03-30T13:09:49.131582+0000","last_fullsized":"2024-03-30T13:09:49.131582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.118404+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.117853+0000","last_became_peered":"2024-03-30T13:09:43.117853+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820864+0000","last_change":"2024-03-30T13:09:42.117594+0000","last_active":"2024-03-30T13:09:49.820864+0000","last_peered":"2024-03-30T13:09:49.820864+0000","last_clean":"2024-03-30T13:09:49.820864+0000","last_became_active":"2024-03-30T13:09:42.117276+0000","last_became_peered":"2024-03-30T13:09:42.117276+0000","last_unstale":"2024-03-30T13:09:49.820864+0000","last_undegraded":"2024-03-30T13:09:49.820864+0000","last_fullsized":"2024-03-30T13:09:49.820864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939158+0000","last_change":"2024-03-30T13:09:47.155090+0000","last_active":"2024-03-30T13:09:49.939158+0000","last_peered":"2024-03-30T13:09:49.939158+0000","last_clean":"2024-03-30T13:09:49.939158+0000","last_became_active":"2024-03-30T13:09:47.154070+0000","last_became_peered":"2024-03-30T13:09:47.154070+0000","last_unstale":"2024-03-30T13:09:49.939158+0000","last_undegraded":"2024-03-30T13:09:49.939158+0000","last_fullsized":"2024-03-30T13:09:49.939158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295287+0000","last_change":"2024-03-30T13:09:43.127827+0000","last_active":"2024-03-30T13:09:49.295287+0000","last_peered":"2024-03-30T13:09:49.295287+0000","last_clean":"2024-03-30T13:09:49.295287+0000","last_became_active":"2024-03-30T13:09:43.127217+0000","last_became_peered":"2024-03-30T13:09:43.127217+0000","last_unstale":"2024-03-30T13:09:49.295287+0000","last_undegraded":"2024-03-30T13:09:49.295287+0000","last_fullsized":"2024-03-30T13:09:49.295287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295274+0000","last_change":"2024-03-30T13:09:42.112507+0000","last_active":"2024-03-30T13:09:49.295274+0000","last_peered":"2024-03-30T13:09:49.295274+0000","last_clean":"2024-03-30T13:09:49.295274+0000","last_became_active":"2024-03-30T13:09:42.112227+0000","last_became_peered":"2024-03-30T13:09:42.112227+0000","last_unstale":"2024-03-30T13:09:49.295274+0000","last_undegraded":"2024-03-30T13:09:49.295274+0000","last_fullsized":"2024-03-30T13:09:49.295274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133062+0000","last_change":"2024-03-30T13:09:47.572882+0000","last_active":"2024-03-30T13:09:49.133062+0000","last_peered":"2024-03-30T13:09:49.133062+0000","last_clean":"2024-03-30T13:09:49.133062+0000","last_became_active":"2024-03-30T13:09:47.572527+0000","last_became_peered":"2024-03-30T13:09:47.572527+0000","last_unstale":"2024-03-30T13:09:49.133062+0000","last_undegraded":"2024-03-30T13:09:49.133062+0000","last_fullsized":"2024-03-30T13:09:49.133062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:18.356964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938472+0000","last_change":"2024-03-30T13:09:43.125989+0000","last_active":"2024-03-30T13:09:49.938472+0000","last_peered":"2024-03-30T13:09:49.938472+0000","last_clean":"2024-03-30T13:09:49.938472+0000","last_became_active":"2024-03-30T13:09:43.125685+0000","last_became_peered":"2024-03-30T13:09:43.125685+0000","last_unstale":"2024-03-30T13:09:49.938472+0000","last_undegraded":"2024-03-30T13:09:49.938472+0000","last_fullsized":"2024-03-30T13:09:49.938472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:23.535266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938231+0000","last_change":"2024-03-30T13:09:42.122605+0000","last_active":"2024-03-30T13:09:49.938231+0000","last_peered":"2024-03-30T13:09:49.938231+0000","last_clean":"2024-03-30T13:09:49.938231+0000","last_became_active":"2024-03-30T13:09:42.122338+0000","last_became_peered":"2024-03-30T13:09:42.122338+0000","last_unstale":"2024-03-30T13:09:49.938231+0000","last_undegraded":"2024-03-30T13:09:49.938231+0000","last_fullsized":"2024-03-30T13:09:49.938231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133046+0000","last_change":"2024-03-30T13:09:47.571300+0000","last_active":"2024-03-30T13:09:49.133046+0000","last_peered":"2024-03-30T13:09:49.133046+0000","last_clean":"2024-03-30T13:09:49.133046+0000","last_became_active":"2024-03-30T13:09:47.570892+0000","last_became_peered":"2024-03-30T13:09:47.570892+0000","last_unstale":"2024-03-30T13:09:49.133046+0000","last_undegraded":"2024-03-30T13:09:49.133046+0000","last_fullsized":"2024-03-30T13:09:49.133046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:09.284589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938281+0000","last_change":"2024-03-30T13:09:43.125239+0000","last_active":"2024-03-30T13:09:49.938281+0000","last_peered":"2024-03-30T13:09:49.938281+0000","last_clean":"2024-03-30T13:09:49.938281+0000","last_became_active":"2024-03-30T13:09:43.124815+0000","last_became_peered":"2024-03-30T13:09:43.124815+0000","last_unstale":"2024-03-30T13:09:49.938281+0000","last_undegraded":"2024-03-30T13:09:49.938281+0000","last_fullsized":"2024-03-30T13:09:49.938281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939006+0000","last_change":"2024-03-30T13:09:42.113307+0000","last_active":"2024-03-30T13:09:49.939006+0000","last_peered":"2024-03-30T13:09:49.939006+0000","last_clean":"2024-03-30T13:09:49.939006+0000","last_became_active":"2024-03-30T13:09:42.112968+0000","last_became_peered":"2024-03-30T13:09:42.112968+0000","last_unstale":"2024-03-30T13:09:49.939006+0000","last_undegraded":"2024-03-30T13:09:49.939006+0000","last_fullsized":"2024-03-30T13:09:49.939006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938380+0000","last_change":"2024-03-30T13:09:47.572011+0000","last_active":"2024-03-30T13:09:49.938380+0000","last_peered":"2024-03-30T13:09:49.938380+0000","last_clean":"2024-03-30T13:09:49.938380+0000","last_became_active":"2024-03-30T13:09:47.571632+0000","last_became_peered":"2024-03-30T13:09:47.571632+0000","last_unstale":"2024-03-30T13:09:49.938380+0000","last_undegraded":"2024-03-30T13:09:49.938380+0000","last_fullsized":"2024-03-30T13:09:49.938380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938714+0000","last_change":"2024-03-30T13:09:43.129993+0000","last_active":"2024-03-30T13:09:49.938714+0000","last_peered":"2024-03-30T13:09:49.938714+0000","last_clean":"2024-03-30T13:09:49.938714+0000","last_became_active":"2024-03-30T13:09:43.128265+0000","last_became_peered":"2024-03-30T13:09:43.128265+0000","last_unstale":"2024-03-30T13:09:49.938714+0000","last_undegraded":"2024-03-30T13:09:49.938714+0000","last_fullsized":"2024-03-30T13:09:49.938714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:28:32.691360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:52.955617+0000","last_change":"2024-03-30T13:09:42.120865+0000","last_active":"2024-03-30T13:09:52.955617+0000","last_peered":"2024-03-30T13:09:52.955617+0000","last_clean":"2024-03-30T13:09:52.955617+0000","last_became_active":"2024-03-30T13:09:42.120589+0000","last_became_peered":"2024-03-30T13:09:42.120589+0000","last_unstale":"2024-03-30T13:09:52.955617+0000","last_undegraded":"2024-03-30T13:09:52.955617+0000","last_fullsized":"2024-03-30T13:09:52.955617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:03.359370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938224+0000","last_change":"2024-03-30T13:09:47.155432+0000","last_active":"2024-03-30T13:09:49.938224+0000","last_peered":"2024-03-30T13:09:49.938224+0000","last_clean":"2024-03-30T13:09:49.938224+0000","last_became_active":"2024-03-30T13:09:47.155044+0000","last_became_peered":"2024-03-30T13:09:47.155044+0000","last_unstale":"2024-03-30T13:09:49.938224+0000","last_undegraded":"2024-03-30T13:09:49.938224+0000","last_fullsized":"2024-03-30T13:09:49.938224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:16.148467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132299+0000","last_change":"2024-03-30T13:09:43.115819+0000","last_active":"2024-03-30T13:09:49.132299+0000","last_peered":"2024-03-30T13:09:49.132299+0000","last_clean":"2024-03-30T13:09:49.132299+0000","last_became_active":"2024-03-30T13:09:43.115563+0000","last_became_peered":"2024-03-30T13:09:43.115563+0000","last_unstale":"2024-03-30T13:09:49.132299+0000","last_undegraded":"2024-03-30T13:09:49.132299+0000","last_fullsized":"2024-03-30T13:09:49.132299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:59.552427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938506+0000","last_change":"2024-03-30T13:09:42.121667+0000","last_active":"2024-03-30T13:09:49.938506+0000","last_peered":"2024-03-30T13:09:49.938506+0000","last_clean":"2024-03-30T13:09:49.938506+0000","last_became_active":"2024-03-30T13:09:42.121384+0000","last_became_peered":"2024-03-30T13:09:42.121384+0000","last_unstale":"2024-03-30T13:09:49.938506+0000","last_undegraded":"2024-03-30T13:09:49.938506+0000","last_fullsized":"2024-03-30T13:09:49.938506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132454+0000","last_change":"2024-03-30T13:09:47.147513+0000","last_active":"2024-03-30T13:09:49.132454+0000","last_peered":"2024-03-30T13:09:49.132454+0000","last_clean":"2024-03-30T13:09:49.132454+0000","last_became_active":"2024-03-30T13:09:47.147126+0000","last_became_peered":"2024-03-30T13:09:47.147126+0000","last_unstale":"2024-03-30T13:09:49.132454+0000","last_undegraded":"2024-03-30T13:09:49.132454+0000","last_fullsized":"2024-03-30T13:09:49.132454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939064+0000","last_change":"2024-03-30T13:09:43.130550+0000","last_active":"2024-03-30T13:09:49.939064+0000","last_peered":"2024-03-30T13:09:49.939064+0000","last_clean":"2024-03-30T13:09:49.939064+0000","last_became_active":"2024-03-30T13:09:43.130297+0000","last_became_peered":"2024-03-30T13:09:43.130297+0000","last_unstale":"2024-03-30T13:09:49.939064+0000","last_undegraded":"2024-03-30T13:09:49.939064+0000","last_fullsized":"2024-03-30T13:09:49.939064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938180+0000","last_change":"2024-03-30T13:09:42.122699+0000","last_active":"2024-03-30T13:09:49.938180+0000","last_peered":"2024-03-30T13:09:49.938180+0000","last_clean":"2024-03-30T13:09:49.938180+0000","last_became_active":"2024-03-30T13:09:42.122407+0000","last_became_peered":"2024-03-30T13:09:42.122407+0000","last_unstale":"2024-03-30T13:09:49.938180+0000","last_undegraded":"2024-03-30T13:09:49.938180+0000","last_fullsized":"2024-03-30T13:09:49.938180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132389+0000","last_change":"2024-03-30T13:09:47.160317+0000","last_active":"2024-03-30T13:09:49.132389+0000","last_peered":"2024-03-30T13:09:49.132389+0000","last_clean":"2024-03-30T13:09:49.132389+0000","last_became_active":"2024-03-30T13:09:47.159818+0000","last_became_peered":"2024-03-30T13:09:47.159818+0000","last_unstale":"2024-03-30T13:09:49.132389+0000","last_undegraded":"2024-03-30T13:09:49.132389+0000","last_fullsized":"2024-03-30T13:09:49.132389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:22:15.869872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938158+0000","last_change":"2024-03-30T13:09:43.128528+0000","last_active":"2024-03-30T13:09:49.938158+0000","last_peered":"2024-03-30T13:09:49.938158+0000","last_clean":"2024-03-30T13:09:49.938158+0000","last_became_active":"2024-03-30T13:09:43.128239+0000","last_became_peered":"2024-03-30T13:09:43.128239+0000","last_unstale":"2024-03-30T13:09:49.938158+0000","last_undegraded":"2024-03-30T13:09:49.938158+0000","last_fullsized":"2024-03-30T13:09:49.938158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:38:56.141275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295744+0000","last_change":"2024-03-30T13:09:42.118540+0000","last_active":"2024-03-30T13:09:49.295744+0000","last_peered":"2024-03-30T13:09:49.295744+0000","last_clean":"2024-03-30T13:09:49.295744+0000","last_became_active":"2024-03-30T13:09:42.118090+0000","last_became_peered":"2024-03-30T13:09:42.118090+0000","last_unstale":"2024-03-30T13:09:49.295744+0000","last_undegraded":"2024-03-30T13:09:49.295744+0000","last_fullsized":"2024-03-30T13:09:49.295744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132435+0000","last_change":"2024-03-30T13:09:47.571367+0000","last_active":"2024-03-30T13:09:49.132435+0000","last_peered":"2024-03-30T13:09:49.132435+0000","last_clean":"2024-03-30T13:09:49.132435+0000","last_became_active":"2024-03-30T13:09:47.570973+0000","last_became_peered":"2024-03-30T13:09:47.570973+0000","last_unstale":"2024-03-30T13:09:49.132435+0000","last_undegraded":"2024-03-30T13:09:49.132435+0000","last_fullsized":"2024-03-30T13:09:49.132435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:33:15.965915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939033+0000","last_change":"2024-03-30T13:09:47.572196+0000","last_active":"2024-03-30T13:09:49.939033+0000","last_peered":"2024-03-30T13:09:49.939033+0000","last_clean":"2024-03-30T13:09:49.939033+0000","last_became_active":"2024-03-30T13:09:47.571378+0000","last_became_peered":"2024-03-30T13:09:47.571378+0000","last_unstale":"2024-03-30T13:09:49.939033+0000","last_undegraded":"2024-03-30T13:09:49.939033+0000","last_fullsized":"2024-03-30T13:09:49.939033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938039+0000","last_change":"2024-03-30T13:09:43.125653+0000","last_active":"2024-03-30T13:09:49.938039+0000","last_peered":"2024-03-30T13:09:49.938039+0000","last_clean":"2024-03-30T13:09:49.938039+0000","last_became_active":"2024-03-30T13:09:43.125382+0000","last_became_peered":"2024-03-30T13:09:43.125382+0000","last_unstale":"2024-03-30T13:09:49.938039+0000","last_undegraded":"2024-03-30T13:09:49.938039+0000","last_fullsized":"2024-03-30T13:09:49.938039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:25:10.034049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937816+0000","last_change":"2024-03-30T13:09:42.115855+0000","last_active":"2024-03-30T13:09:49.937816+0000","last_peered":"2024-03-30T13:09:49.937816+0000","last_clean":"2024-03-30T13:09:49.937816+0000","last_became_active":"2024-03-30T13:09:42.115369+0000","last_became_peered":"2024-03-30T13:09:42.115369+0000","last_unstale":"2024-03-30T13:09:49.937816+0000","last_undegraded":"2024-03-30T13:09:49.937816+0000","last_fullsized":"2024-03-30T13:09:49.937816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295715+0000","last_change":"2024-03-30T13:09:47.156344+0000","last_active":"2024-03-30T13:09:49.295715+0000","last_peered":"2024-03-30T13:09:49.295715+0000","last_clean":"2024-03-30T13:09:49.295715+0000","last_became_active":"2024-03-30T13:09:47.155917+0000","last_became_peered":"2024-03-30T13:09:47.155917+0000","last_unstale":"2024-03-30T13:09:49.295715+0000","last_undegraded":"2024-03-30T13:09:49.295715+0000","last_fullsized":"2024-03-30T13:09:49.295715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132295+0000","last_change":"2024-03-30T13:09:43.128241+0000","last_active":"2024-03-30T13:09:49.132295+0000","last_peered":"2024-03-30T13:09:49.132295+0000","last_clean":"2024-03-30T13:09:49.132295+0000","last_became_active":"2024-03-30T13:09:43.127971+0000","last_became_peered":"2024-03-30T13:09:43.127971+0000","last_unstale":"2024-03-30T13:09:49.132295+0000","last_undegraded":"2024-03-30T13:09:49.132295+0000","last_fullsized":"2024-03-30T13:09:49.132295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132655+0000","last_change":"2024-03-30T13:09:42.122509+0000","last_active":"2024-03-30T13:09:49.132655+0000","last_peered":"2024-03-30T13:09:49.132655+0000","last_clean":"2024-03-30T13:09:49.132655+0000","last_became_active":"2024-03-30T13:09:42.122130+0000","last_became_peered":"2024-03-30T13:09:42.122130+0000","last_unstale":"2024-03-30T13:09:49.132655+0000","last_undegraded":"2024-03-30T13:09:49.132655+0000","last_fullsized":"2024-03-30T13:09:49.132655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295373+0000","last_change":"2024-03-30T13:09:42.115453+0000","last_active":"2024-03-30T13:09:49.295373+0000","last_peered":"2024-03-30T13:09:49.295373+0000","last_clean":"2024-03-30T13:09:49.295373+0000","last_became_active":"2024-03-30T13:09:42.115182+0000","last_became_peered":"2024-03-30T13:09:42.115182+0000","last_unstale":"2024-03-30T13:09:49.295373+0000","last_undegraded":"2024-03-30T13:09:49.295373+0000","last_fullsized":"2024-03-30T13:09:49.295373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:47:12.895083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938966+0000","last_change":"2024-03-30T13:09:43.127679+0000","last_active":"2024-03-30T13:09:49.938966+0000","last_peered":"2024-03-30T13:09:49.938966+0000","last_clean":"2024-03-30T13:09:49.938966+0000","last_became_active":"2024-03-30T13:09:43.127421+0000","last_became_peered":"2024-03-30T13:09:43.127421+0000","last_unstale":"2024-03-30T13:09:49.938966+0000","last_undegraded":"2024-03-30T13:09:49.938966+0000","last_fullsized":"2024-03-30T13:09:49.938966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821070+0000","last_change":"2024-03-30T13:09:47.154699+0000","last_active":"2024-03-30T13:09:49.821070+0000","last_peered":"2024-03-30T13:09:49.821070+0000","last_clean":"2024-03-30T13:09:49.821070+0000","last_became_active":"2024-03-30T13:09:47.154291+0000","last_became_peered":"2024-03-30T13:09:47.154291+0000","last_unstale":"2024-03-30T13:09:49.821070+0000","last_undegraded":"2024-03-30T13:09:49.821070+0000","last_fullsized":"2024-03-30T13:09:49.821070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:56:47.829620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133085+0000","last_change":"2024-03-30T13:09:42.124644+0000","last_active":"2024-03-30T13:09:49.133085+0000","last_peered":"2024-03-30T13:09:49.133085+0000","last_clean":"2024-03-30T13:09:49.133085+0000","last_became_active":"2024-03-30T13:09:42.124355+0000","last_became_peered":"2024-03-30T13:09:42.124355+0000","last_unstale":"2024-03-30T13:09:49.133085+0000","last_undegraded":"2024-03-30T13:09:49.133085+0000","last_fullsized":"2024-03-30T13:09:49.133085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132040+0000","last_change":"2024-03-30T13:09:43.113299+0000","last_active":"2024-03-30T13:09:49.132040+0000","last_peered":"2024-03-30T13:09:49.132040+0000","last_clean":"2024-03-30T13:09:49.132040+0000","last_became_active":"2024-03-30T13:09:43.113031+0000","last_became_peered":"2024-03-30T13:09:43.113031+0000","last_unstale":"2024-03-30T13:09:49.132040+0000","last_undegraded":"2024-03-30T13:09:49.132040+0000","last_fullsized":"2024-03-30T13:09:49.132040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:32.715713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132575+0000","last_change":"2024-03-30T13:09:47.156912+0000","last_active":"2024-03-30T13:09:49.132575+0000","last_peered":"2024-03-30T13:09:49.132575+0000","last_clean":"2024-03-30T13:09:49.132575+0000","last_became_active":"2024-03-30T13:09:47.156492+0000","last_became_peered":"2024-03-30T13:09:47.156492+0000","last_unstale":"2024-03-30T13:09:49.132575+0000","last_undegraded":"2024-03-30T13:09:49.132575+0000","last_fullsized":"2024-03-30T13:09:49.132575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:55:36.886194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295186+0000","last_change":"2024-03-30T13:09:43.127904+0000","last_active":"2024-03-30T13:09:49.295186+0000","last_peered":"2024-03-30T13:09:49.295186+0000","last_clean":"2024-03-30T13:09:49.295186+0000","last_became_active":"2024-03-30T13:09:43.127563+0000","last_became_peered":"2024-03-30T13:09:43.127563+0000","last_unstale":"2024-03-30T13:09:49.295186+0000","last_undegraded":"2024-03-30T13:09:49.295186+0000","last_fullsized":"2024-03-30T13:09:49.295186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820892+0000","last_change":"2024-03-30T13:09:43.131561+0000","last_active":"2024-03-30T13:09:49.820892+0000","last_peered":"2024-03-30T13:09:49.820892+0000","last_clean":"2024-03-30T13:09:49.820892+0000","last_became_active":"2024-03-30T13:09:43.129703+0000","last_became_peered":"2024-03-30T13:09:43.129703+0000","last_unstale":"2024-03-30T13:09:49.820892+0000","last_undegraded":"2024-03-30T13:09:49.820892+0000","last_fullsized":"2024-03-30T13:09:49.820892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:46.613583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938131+0000","last_change":"2024-03-30T13:09:43.129678+0000","last_active":"2024-03-30T13:09:49.938131+0000","last_peered":"2024-03-30T13:09:49.938131+0000","last_clean":"2024-03-30T13:09:49.938131+0000","last_became_active":"2024-03-30T13:09:43.129405+0000","last_became_peered":"2024-03-30T13:09:43.129405+0000","last_unstale":"2024-03-30T13:09:49.938131+0000","last_undegraded":"2024-03-30T13:09:49.938131+0000","last_fullsized":"2024-03-30T13:09:49.938131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:13:29.545084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938913+0000","last_change":"2024-03-30T13:09:43.135202+0000","last_active":"2024-03-30T13:09:49.938913+0000","last_peered":"2024-03-30T13:09:49.938913+0000","last_clean":"2024-03-30T13:09:49.938913+0000","last_became_active":"2024-03-30T13:09:43.134895+0000","last_became_peered":"2024-03-30T13:09:43.134895+0000","last_unstale":"2024-03-30T13:09:49.938913+0000","last_undegraded":"2024-03-30T13:09:49.938913+0000","last_fullsized":"2024-03-30T13:09:49.938913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133173+0000","last_change":"2024-03-30T13:09:43.127414+0000","last_active":"2024-03-30T13:09:49.133173+0000","last_peered":"2024-03-30T13:09:49.133173+0000","last_clean":"2024-03-30T13:09:49.133173+0000","last_became_active":"2024-03-30T13:09:43.127077+0000","last_became_peered":"2024-03-30T13:09:43.127077+0000","last_unstale":"2024-03-30T13:09:49.133173+0000","last_undegraded":"2024-03-30T13:09:49.133173+0000","last_fullsized":"2024-03-30T13:09:49.133173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938324+0000","last_change":"2024-03-30T13:09:43.119060+0000","last_active":"2024-03-30T13:09:49.938324+0000","last_peered":"2024-03-30T13:09:49.938324+0000","last_clean":"2024-03-30T13:09:49.938324+0000","last_became_active":"2024-03-30T13:09:43.118585+0000","last_became_peered":"2024-03-30T13:09:43.118585+0000","last_unstale":"2024-03-30T13:09:49.938324+0000","last_undegraded":"2024-03-30T13:09:49.938324+0000","last_fullsized":"2024-03-30T13:09:49.938324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295833+0000","last_change":"2024-03-30T13:09:43.122310+0000","last_active":"2024-03-30T13:09:49.295833+0000","last_peered":"2024-03-30T13:09:49.295833+0000","last_clean":"2024-03-30T13:09:49.295833+0000","last_became_active":"2024-03-30T13:09:43.121999+0000","last_became_peered":"2024-03-30T13:09:43.121999+0000","last_unstale":"2024-03-30T13:09:49.295833+0000","last_undegraded":"2024-03-30T13:09:49.295833+0000","last_fullsized":"2024-03-30T13:09:49.295833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133127+0000","last_change":"2024-03-30T13:09:43.122476+0000","last_active":"2024-03-30T13:09:49.133127+0000","last_peered":"2024-03-30T13:09:49.133127+0000","last_clean":"2024-03-30T13:09:49.133127+0000","last_became_active":"2024-03-30T13:09:43.120315+0000","last_became_peered":"2024-03-30T13:09:43.120315+0000","last_unstale":"2024-03-30T13:09:49.133127+0000","last_undegraded":"2024-03-30T13:09:49.133127+0000","last_fullsized":"2024-03-30T13:09:49.133127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937780+0000","last_change":"2024-03-30T13:09:43.128432+0000","last_active":"2024-03-30T13:09:49.937780+0000","last_peered":"2024-03-30T13:09:49.937780+0000","last_clean":"2024-03-30T13:09:49.937780+0000","last_became_active":"2024-03-30T13:09:43.128013+0000","last_became_peered":"2024-03-30T13:09:43.128013+0000","last_unstale":"2024-03-30T13:09:49.937780+0000","last_undegraded":"2024-03-30T13:09:49.937780+0000","last_fullsized":"2024-03-30T13:09:49.937780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132485+0000","last_change":"2024-03-30T13:09:43.120476+0000","last_active":"2024-03-30T13:09:49.132485+0000","last_peered":"2024-03-30T13:09:49.132485+0000","last_clean":"2024-03-30T13:09:49.132485+0000","last_became_active":"2024-03-30T13:09:43.120212+0000","last_became_peered":"2024-03-30T13:09:43.120212+0000","last_unstale":"2024-03-30T13:09:49.132485+0000","last_undegraded":"2024-03-30T13:09:49.132485+0000","last_fullsized":"2024-03-30T13:09:49.132485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:50:23.461880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938025+0000","last_change":"2024-03-30T13:09:43.137167+0000","last_active":"2024-03-30T13:09:49.938025+0000","last_peered":"2024-03-30T13:09:49.938025+0000","last_clean":"2024-03-30T13:09:49.938025+0000","last_became_active":"2024-03-30T13:09:43.136900+0000","last_became_peered":"2024-03-30T13:09:43.136900+0000","last_unstale":"2024-03-30T13:09:49.938025+0000","last_undegraded":"2024-03-30T13:09:49.938025+0000","last_fullsized":"2024-03-30T13:09:49.938025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:15:29.671082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124147+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.123860+0000","last_became_peered":"2024-03-30T13:09:43.123860+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:32:55.194498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295535+0000","last_change":"2024-03-30T13:09:43.127624+0000","last_active":"2024-03-30T13:09:49.295535+0000","last_peered":"2024-03-30T13:09:49.295535+0000","last_clean":"2024-03-30T13:09:49.295535+0000","last_became_active":"2024-03-30T13:09:43.127340+0000","last_became_peered":"2024-03-30T13:09:43.127340+0000","last_unstale":"2024-03-30T13:09:49.295535+0000","last_undegraded":"2024-03-30T13:09:49.295535+0000","last_fullsized":"2024-03-30T13:09:49.295535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938857+0000","last_change":"2024-03-30T13:09:43.122985+0000","last_active":"2024-03-30T13:09:49.938857+0000","last_peered":"2024-03-30T13:09:49.938857+0000","last_clean":"2024-03-30T13:09:49.938857+0000","last_became_active":"2024-03-30T13:09:43.122689+0000","last_became_peered":"2024-03-30T13:09:43.122689+0000","last_unstale":"2024-03-30T13:09:49.938857+0000","last_undegraded":"2024-03-30T13:09:49.938857+0000","last_fullsized":"2024-03-30T13:09:49.938857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:41.817012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821341+0000","last_change":"2024-03-30T13:09:43.131062+0000","last_active":"2024-03-30T13:09:49.821341+0000","last_peered":"2024-03-30T13:09:49.821341+0000","last_clean":"2024-03-30T13:09:49.821341+0000","last_became_active":"2024-03-30T13:09:43.130727+0000","last_became_peered":"2024-03-30T13:09:43.130727+0000","last_unstale":"2024-03-30T13:09:49.821341+0000","last_undegraded":"2024-03-30T13:09:49.821341+0000","last_fullsized":"2024-03-30T13:09:49.821341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:03:31.612660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938468+0000","last_change":"2024-03-30T13:09:43.137027+0000","last_active":"2024-03-30T13:09:49.938468+0000","last_peered":"2024-03-30T13:09:49.938468+0000","last_clean":"2024-03-30T13:09:49.938468+0000","last_became_active":"2024-03-30T13:09:43.136755+0000","last_became_peered":"2024-03-30T13:09:43.136755+0000","last_unstale":"2024-03-30T13:09:49.938468+0000","last_undegraded":"2024-03-30T13:09:49.938468+0000","last_fullsized":"2024-03-30T13:09:49.938468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:38:26.087735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937690+0000","last_change":"2024-03-30T13:09:43.132197+0000","last_active":"2024-03-30T13:09:49.937690+0000","last_peered":"2024-03-30T13:09:49.937690+0000","last_clean":"2024-03-30T13:09:49.937690+0000","last_became_active":"2024-03-30T13:09:43.131927+0000","last_became_peered":"2024-03-30T13:09:43.131927+0000","last_unstale":"2024-03-30T13:09:49.937690+0000","last_undegraded":"2024-03-30T13:09:49.937690+0000","last_fullsized":"2024-03-30T13:09:49.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938139+0000","last_change":"2024-03-30T13:09:43.109275+0000","last_active":"2024-03-30T13:09:49.938139+0000","last_peered":"2024-03-30T13:09:49.938139+0000","last_clean":"2024-03-30T13:09:49.938139+0000","last_became_active":"2024-03-30T13:09:43.108964+0000","last_became_peered":"2024-03-30T13:09:43.108964+0000","last_unstale":"2024-03-30T13:09:49.938139+0000","last_undegraded":"2024-03-30T13:09:49.938139+0000","last_fullsized":"2024-03-30T13:09:49.938139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:48.847357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938228+0000","last_change":"2024-03-30T13:09:43.121915+0000","last_active":"2024-03-30T13:09:49.938228+0000","last_peered":"2024-03-30T13:09:49.938228+0000","last_clean":"2024-03-30T13:09:49.938228+0000","last_became_active":"2024-03-30T13:09:43.121625+0000","last_became_peered":"2024-03-30T13:09:43.121625+0000","last_unstale":"2024-03-30T13:09:49.938228+0000","last_undegraded":"2024-03-30T13:09:49.938228+0000","last_fullsized":"2024-03-30T13:09:49.938228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132890+0000","last_change":"2024-03-30T13:09:43.131187+0000","last_active":"2024-03-30T13:09:49.132890+0000","last_peered":"2024-03-30T13:09:49.132890+0000","last_clean":"2024-03-30T13:09:49.132890+0000","last_became_active":"2024-03-30T13:09:43.130682+0000","last_became_peered":"2024-03-30T13:09:43.130682+0000","last_unstale":"2024-03-30T13:09:49.132890+0000","last_undegraded":"2024-03-30T13:09:49.132890+0000","last_fullsized":"2024-03-30T13:09:49.132890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132240+0000","last_change":"2024-03-30T13:09:43.123005+0000","last_active":"2024-03-30T13:09:49.132240+0000","last_peered":"2024-03-30T13:09:49.132240+0000","last_clean":"2024-03-30T13:09:49.132240+0000","last_became_active":"2024-03-30T13:09:43.122761+0000","last_became_peered":"2024-03-30T13:09:43.122761+0000","last_unstale":"2024-03-30T13:09:49.132240+0000","last_undegraded":"2024-03-30T13:09:49.132240+0000","last_fullsized":"2024-03-30T13:09:49.132240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:48.062694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295578+0000","last_change":"2024-03-30T13:09:43.122715+0000","last_active":"2024-03-30T13:09:49.295578+0000","last_peered":"2024-03-30T13:09:49.295578+0000","last_clean":"2024-03-30T13:09:49.295578+0000","last_became_active":"2024-03-30T13:09:43.122445+0000","last_became_peered":"2024-03-30T13:09:43.122445+0000","last_unstale":"2024-03-30T13:09:49.295578+0000","last_undegraded":"2024-03-30T13:09:49.295578+0000","last_fullsized":"2024-03-30T13:09:49.295578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295783+0000","last_change":"2024-03-30T13:09:43.130345+0000","last_active":"2024-03-30T13:09:49.295783+0000","last_peered":"2024-03-30T13:09:49.295783+0000","last_clean":"2024-03-30T13:09:49.295783+0000","last_became_active":"2024-03-30T13:09:43.130062+0000","last_became_peered":"2024-03-30T13:09:43.130062+0000","last_unstale":"2024-03-30T13:09:49.295783+0000","last_undegraded":"2024-03-30T13:09:49.295783+0000","last_fullsized":"2024-03-30T13:09:49.295783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939037+0000","last_change":"2024-03-30T13:09:43.135018+0000","last_active":"2024-03-30T13:09:49.939037+0000","last_peered":"2024-03-30T13:09:49.939037+0000","last_clean":"2024-03-30T13:09:49.939037+0000","last_became_active":"2024-03-30T13:09:43.134354+0000","last_became_peered":"2024-03-30T13:09:43.134354+0000","last_unstale":"2024-03-30T13:09:49.939037+0000","last_undegraded":"2024-03-30T13:09:49.939037+0000","last_fullsized":"2024-03-30T13:09:49.939037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:04.849718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132188+0000","last_change":"2024-03-30T13:09:43.123158+0000","last_active":"2024-03-30T13:09:49.132188+0000","last_peered":"2024-03-30T13:09:49.132188+0000","last_clean":"2024-03-30T13:09:49.132188+0000","last_became_active":"2024-03-30T13:09:43.122830+0000","last_became_peered":"2024-03-30T13:09:43.122830+0000","last_unstale":"2024-03-30T13:09:49.132188+0000","last_undegraded":"2024-03-30T13:09:49.132188+0000","last_fullsized":"2024-03-30T13:09:49.132188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124628+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.124364+0000","last_became_peered":"2024-03-30T13:09:43.124364+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937937+0000","last_change":"2024-03-30T13:09:43.132387+0000","last_active":"2024-03-30T13:09:49.937937+0000","last_peered":"2024-03-30T13:09:49.937937+0000","last_clean":"2024-03-30T13:09:49.937937+0000","last_became_active":"2024-03-30T13:09:43.132110+0000","last_became_peered":"2024-03-30T13:09:43.132110+0000","last_unstale":"2024-03-30T13:09:49.937937+0000","last_undegraded":"2024-03-30T13:09:49.937937+0000","last_fullsized":"2024-03-30T13:09:49.937937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938308+0000","last_change":"2024-03-30T13:09:43.124528+0000","last_active":"2024-03-30T13:09:49.938308+0000","last_peered":"2024-03-30T13:09:49.938308+0000","last_clean":"2024-03-30T13:09:49.938308+0000","last_became_active":"2024-03-30T13:09:43.124023+0000","last_became_peered":"2024-03-30T13:09:43.124023+0000","last_unstale":"2024-03-30T13:09:49.938308+0000","last_undegraded":"2024-03-30T13:09:49.938308+0000","last_fullsized":"2024-03-30T13:09:49.938308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938061+0000","last_change":"2024-03-30T13:09:43.127663+0000","last_active":"2024-03-30T13:09:49.938061+0000","last_peered":"2024-03-30T13:09:49.938061+0000","last_clean":"2024-03-30T13:09:49.938061+0000","last_became_active":"2024-03-30T13:09:43.127180+0000","last_became_peered":"2024-03-30T13:09:43.127180+0000","last_unstale":"2024-03-30T13:09:49.938061+0000","last_undegraded":"2024-03-30T13:09:49.938061+0000","last_fullsized":"2024-03-30T13:09:49.938061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:36:33.418013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.121807+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.121498+0000","last_became_peered":"2024-03-30T13:09:43.121498+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938554+0000","last_change":"2024-03-30T13:09:43.124024+0000","last_active":"2024-03-30T13:09:49.938554+0000","last_peered":"2024-03-30T13:09:49.938554+0000","last_clean":"2024-03-30T13:09:49.938554+0000","last_became_active":"2024-03-30T13:09:43.123748+0000","last_became_peered":"2024-03-30T13:09:43.123748+0000","last_unstale":"2024-03-30T13:09:49.938554+0000","last_undegraded":"2024-03-30T13:09:49.938554+0000","last_fullsized":"2024-03-30T13:09:49.938554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938980+0000","last_change":"2024-03-30T13:09:43.122707+0000","last_active":"2024-03-30T13:09:49.938980+0000","last_peered":"2024-03-30T13:09:49.938980+0000","last_clean":"2024-03-30T13:09:49.938980+0000","last_became_active":"2024-03-30T13:09:43.122418+0000","last_became_peered":"2024-03-30T13:09:43.122418+0000","last_unstale":"2024-03-30T13:09:49.938980+0000","last_undegraded":"2024-03-30T13:09:49.938980+0000","last_fullsized":"2024-03-30T13:09:49.938980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:49.970892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":700230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":109406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":109280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":109440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":109370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:10:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T13:10:06.563 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-30T13:10:06.563 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-30T13:10:06.563 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-30T13:10:06.563 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-30T13:10:06.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.702+0000 7fcf4e93d640 1 Processor -- start 2024-03-30T13:10:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.702+0000 7fcf4e93d640 1 -- start start 2024-03-30T13:10:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4e93d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf48147410 0x7fcf48071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4e93d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcf48145220 0x7fcf4806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4e93d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4815e9a0 con 0x7fcf4806dde0 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4e93d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4806c5b0 con 0x7fcf48147410 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4e93d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4806c800 con 0x7fcf48145220 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf47fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf48147410 0x7fcf48071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4ceda640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcf4806dde0 0x7fcf480ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53826/0 (socket says 172.21.15.66:53826) 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf4ceda640 1 -- 172.21.15.66:0/1317475088 learned_addr learned my addr 172.21.15.66:0/1317475088 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf477fe640 1 --2- 172.21.15.66:0/1317475088 >> v2:172.21.15.117:3300/0 conn(0x7fcf48145220 0x7fcf4806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf47fff640 1 -- 172.21.15.66:0/1317475088 >> v2:172.21.15.117:3300/0 conn(0x7fcf48145220 msgr2=0x7fcf4806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf47fff640 1 --2- 172.21.15.66:0/1317475088 >> v2:172.21.15.117:3300/0 conn(0x7fcf48145220 0x7fcf4806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf47fff640 1 -- 172.21.15.66:0/1317475088 >> v1:172.21.15.66:6789/0 conn(0x7fcf4806dde0 legacy=0x7fcf480ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf47fff640 1 -- 172.21.15.66:0/1317475088 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4806cae0 con 0x7fcf48147410 2024-03-30T13:10:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf46ffd640 1 -- 172.21.15.66:0/1317475088 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2722575436 0 0) 0x7fcf4815e9a0 con 0x7fcf4806dde0 2024-03-30T13:10:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.703+0000 7fcf477fe640 1 --2- 172.21.15.66:0/1317475088 >> v2:172.21.15.117:3300/0 conn(0x7fcf48145220 0x7fcf4806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.704+0000 7fcf47fff640 1 --2- 172.21.15.66:0/1317475088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48147410 0x7fcf48071cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf480b4de0 tx=0x7fcf3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910d3941b22171c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.704+0000 7fcf46ffd640 1 -- 172.21.15.66:0/1317475088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38035020 con 0x7fcf48147410 2024-03-30T13:10:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.704+0000 7fcf46ffd640 1 -- 172.21.15.66:0/1317475088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38031830 con 0x7fcf48147410 2024-03-30T13:10:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.704+0000 7fcf46ffd640 1 -- 172.21.15.66:0/1317475088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38031c00 con 0x7fcf48147410 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 -- 172.21.15.66:0/1317475088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48147410 msgr2=0x7fcf48071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/1317475088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48147410 0x7fcf48071cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf480b4de0 tx=0x7fcf3802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 -- 172.21.15.66:0/1317475088 shutdown_connections 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/1317475088 >> v2:172.21.15.117:3300/0 conn(0x7fcf48145220 0x7fcf4806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/1317475088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48147410 0x7fcf48071cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 -- 172.21.15.66:0/1317475088 >> 172.21.15.66:0/1317475088 conn(0x7fcf4806f820 msgr2=0x7fcf4806fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:06.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 -- 172.21.15.66:0/1317475088 shutdown_connections 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.705+0000 7fcf4e93d640 1 -- 172.21.15.66:0/1317475088 wait complete. 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.706+0000 7fcf4e93d640 1 Processor -- start 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.706+0000 7fcf4e93d640 1 -- start start 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4e93d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4806dde0 0x7fcf480cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4e93d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 0x7fcf481f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4e93d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf48147410 0x7fcf4815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4e93d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4806d380 con 0x7fcf48145220 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4e93d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4806d0a0 con 0x7fcf4806dde0 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4e93d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4806e270 con 0x7fcf48147410 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 0x7fcf481f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 0x7fcf481f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34174/0 (socket says 172.21.15.66:34174) 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 -- 172.21.15.66:0/2282630166 learned_addr learned my addr 172.21.15.66:0/2282630166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf47fff640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4806dde0 0x7fcf480cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf4ceda640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf48147410 0x7fcf4815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 -- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf48147410 msgr2=0x7fcf4815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf48147410 0x7fcf4815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 -- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4806dde0 msgr2=0x7fcf480cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4806dde0 0x7fcf480cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 -- 172.21.15.66:0/2282630166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf481f8700 con 0x7fcf48145220 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf477fe640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 0x7fcf481f7f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00ec10 tx=0x7fcf3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.707+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c017070 con 0x7fcf48145220 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.708+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00cc30 con 0x7fcf48145220 2024-03-30T13:10:06.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.708+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf481f89e0 con 0x7fcf48145220 2024-03-30T13:10:06.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.708+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c013470 con 0x7fcf48145220 2024-03-30T13:10:06.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.708+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4815db80 con 0x7fcf48145220 2024-03-30T13:10:06.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.709+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf48075a10 con 0x7fcf48145220 2024-03-30T13:10:06.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.709+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf3c01a070 con 0x7fcf48145220 2024-03-30T13:10:06.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.709+0000 7fcf44ff9640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcf3406c530 0x7fcf3406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:06.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.710+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcf3c08b8e0 con 0x7fcf48145220 2024-03-30T13:10:06.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.710+0000 7fcf47fff640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcf3406c530 0x7fcf3406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:06.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.710+0000 7fcf47fff640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcf3406c530 0x7fcf3406e9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf38036000 tx=0x7fcf3803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.714+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcf3c059580 con 0x7fcf48145220 2024-03-30T13:10:06.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.949+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcf48071aa0 con 0x7fcf48145220 2024-03-30T13:10:06.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.952+0000 7fcf44ff9640 1 -- 172.21.15.66:0/2282630166 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173733 (secure 0 0 0) 0x7fcf3c05d880 con 0x7fcf48145220 2024-03-30T13:10:06.959 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:06.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcf3406c530 msgr2=0x7fcf3406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcf3406c530 0x7fcf3406e9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf38036000 tx=0x7fcf3803d040 comp rx=0 tx=0).stop 2024-03-30T13:10:06.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 msgr2=0x7fcf481f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:06.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 0x7fcf481f7f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00ec10 tx=0x7fcf3c00c520 comp rx=0 tx=0).stop 2024-03-30T13:10:06.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 shutdown_connections 2024-03-30T13:10:06.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcf3406c530 0x7fcf3406e9f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf48147410 0x7fcf4815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48145220 0x7fcf481f7f90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 --2- 172.21.15.66:0/2282630166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4806dde0 0x7fcf480cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:06.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 >> 172.21.15.66:0/2282630166 conn(0x7fcf4806f820 msgr2=0x7fcf480cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:06.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 shutdown_connections 2024-03-30T13:10:06.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:06.956+0000 7fcf4e93d640 1 -- 172.21.15.66:0/2282630166 wait complete. 2024-03-30T13:10:06.971 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1829857110},{"type":"v1","addr":"172.21.15.66:6833","nonce":1829857110}]},"active_addr":"172.21.15.66:6833/1829857110","active_change":"2024-03-30T13:09:22.785960+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":663588464}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":997189497}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3092632695}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3773506129}]}]} 2024-03-30T13:10:06.975 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-30T13:10:06.975 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-30T13:10:06.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:07.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.078+0000 7f2580c3c640 1 Processor -- start 2024-03-30T13:10:07.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.078+0000 7f2580c3c640 1 -- start start 2024-03-30T13:10:07.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.078+0000 7f2580c3c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c06dfb0 0x7f257c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2580c3c640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257c15e9a0 con 0x7f257c071cc0 2024-03-30T13:10:07.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2580c3c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257c06c800 con 0x7f257c147410 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2580c3c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f257c06e440 con 0x7f257c06dfb0 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2571d9b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f257c071cc0 0x7f257c06d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53830/0 (socket says 172.21.15.66:53830) 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2571d9b640 1 -- 172.21.15.66:0/4153440225 learned_addr learned my addr 172.21.15.66:0/4153440225 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f257ad9d640 1 --2- 172.21.15.66:0/4153440225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c06dfb0 0x7f257c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2579d9b640 1 -- 172.21.15.66:0/4153440225 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2441691368 0 0) 0x7f257c15e9a0 con 0x7f257c071cc0 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2579d9b640 1 -- 172.21.15.66:0/4153440225 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2568003520 con 0x7f257c071cc0 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f257ad9d640 1 -- 172.21.15.66:0/4153440225 >> v1:172.21.15.117:6789/0 conn(0x7f257c147410 legacy=0x7f257c145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f2579d9b640 1 -- 172.21.15.66:0/4153440225 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567237027 0 0) 0x7f257c06c800 con 0x7f257c147410 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f257ad9d640 1 -- 172.21.15.66:0/4153440225 >> v1:172.21.15.66:6789/0 conn(0x7f257c071cc0 legacy=0x7f257c06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.079+0000 7f257ad9d640 1 -- 172.21.15.66:0/4153440225 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c06cae0 con 0x7f257c06dfb0 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f257ad9d640 1 --2- 172.21.15.66:0/4153440225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c06dfb0 0x7f257c0ce8c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2574009dc0 tx=0x7f257402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e59a31c847e89b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f2579d9b640 1 -- 172.21.15.66:0/4153440225 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574039070 con 0x7f257c06dfb0 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f2579d9b640 1 -- 172.21.15.66:0/4153440225 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257402ccd0 con 0x7f257c06dfb0 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f2579d9b640 1 -- 172.21.15.66:0/4153440225 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574034640 con 0x7f257c06dfb0 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f2580c3c640 1 -- 172.21.15.66:0/4153440225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c06dfb0 msgr2=0x7f257c0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f2580c3c640 1 --2- 172.21.15.66:0/4153440225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c06dfb0 0x7f257c0ce8c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2574009dc0 tx=0x7f257402bf20 comp rx=0 tx=0).stop 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.080+0000 7f2580c3c640 1 -- 172.21.15.66:0/4153440225 shutdown_connections 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.081+0000 7f2580c3c640 1 --2- 172.21.15.66:0/4153440225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c06dfb0 0x7f257c0ce8c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.081+0000 7f2580c3c640 1 -- 172.21.15.66:0/4153440225 >> 172.21.15.66:0/4153440225 conn(0x7f257c06f820 msgr2=0x7f257c06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.081+0000 7f2580c3c640 1 -- 172.21.15.66:0/4153440225 shutdown_connections 2024-03-30T13:10:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.081+0000 7f2580c3c640 1 -- 172.21.15.66:0/4153440225 wait complete. 2024-03-30T13:10:07.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.082+0000 7f2580c3c640 1 Processor -- start 2024-03-30T13:10:07.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.082+0000 7f2580c3c640 1 -- start start 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2580c3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 0x7f257c0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2580c3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f257c071cc0 0x7f257c0c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2580c3c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c147410 0x7f257c0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2580c3c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f257c06d0a0 con 0x7f257c06dfb0 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2580c3c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f257c06c5b0 con 0x7f257c071cc0 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2580c3c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f257c06d380 con 0x7f257c147410 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 0x7f257c0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 0x7f257c0b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34184/0 (socket says 172.21.15.66:34184) 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 -- 172.21.15.66:0/300001423 learned_addr learned my addr 172.21.15.66:0/300001423 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f2571d9b640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f257c071cc0 0x7f257c0c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257ad9d640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c147410 0x7f257c0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 -- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c147410 msgr2=0x7f257c0c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c147410 0x7f257c0c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 -- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f257c071cc0 msgr2=0x7f257c0c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f257c071cc0 0x7f257c0c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 -- 172.21.15.66:0/300001423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c0af360 con 0x7f257c06dfb0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.083+0000 7f257a59c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 0x7f257c0b1360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f256c00c440 tx=0x7f256c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.084+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c017070 con 0x7f257c06dfb0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.084+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c0035f0 con 0x7f257c06dfb0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.084+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c0c7250 con 0x7f257c06dfb0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.084+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c0039e0 con 0x7f257c06dfb0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.084+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c0c7810 con 0x7f257c06dfb0 2024-03-30T13:10:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.085+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c075a10 con 0x7f257c06dfb0 2024-03-30T13:10:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.085+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f256c01a070 con 0x7f257c06dfb0 2024-03-30T13:10:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.085+0000 7f25737fe640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f255c06c530 0x7f255c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.086+0000 7f2571d9b640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f255c06c530 0x7f255c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.086+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f256c08adc0 con 0x7f257c06dfb0 2024-03-30T13:10:07.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.086+0000 7f2571d9b640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f255c06c530 0x7f255c06e9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f257c0cf330 tx=0x7f2564009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.091+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f256c0589d0 con 0x7f257c06dfb0 2024-03-30T13:10:07.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.278+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f257c071aa0 con 0x7f257c06dfb0 2024-03-30T13:10:07.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.279+0000 7f25737fe640 1 -- 172.21.15.66:0/300001423 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f256c05ccd0 con 0x7f257c06dfb0 2024-03-30T13:10:07.284 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:07.284 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f255c06c530 msgr2=0x7f255c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f255c06c530 0x7f255c06e9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f257c0cf330 tx=0x7f2564009040 comp rx=0 tx=0).stop 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 msgr2=0x7f257c0b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 0x7f257c0b1360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f256c00c440 tx=0x7f256c00c900 comp rx=0 tx=0).stop 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 shutdown_connections 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f255c06c530 0x7f255c06e9f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f257c147410 0x7f257c0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f257c071cc0 0x7f257c0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 --2- 172.21.15.66:0/300001423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f257c06dfb0 0x7f257c0b1360 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 >> 172.21.15.66:0/300001423 conn(0x7f257c06f820 msgr2=0x7f257c0b6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 shutdown_connections 2024-03-30T13:10:07.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.281+0000 7f2580c3c640 1 -- 172.21.15.66:0/300001423 wait complete. 2024-03-30T13:10:07.297 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-30T13:10:07.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:07.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.401+0000 7f2a9cece640 1 Processor -- start 2024-03-30T13:10:07.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.401+0000 7f2a9cece640 1 -- start start 2024-03-30T13:10:07.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9cece640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a9806c690 0x7f2a980cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9cece640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2a980d0300 0x7f2a9815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9cece640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a981624f0 con 0x7f2a98155990 2024-03-30T13:10:07.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9cece640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f2a980d0770 con 0x7f2a980d0300 2024-03-30T13:10:07.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9cece640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2a981622a0 con 0x7f2a9806c690 2024-03-30T13:10:07.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9659c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2a98155990 0x7f2a980b4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53840/0 (socket says 172.21.15.66:53840) 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a9659c640 1 -- 172.21.15.66:0/125543412 learned_addr learned my addr 172.21.15.66:0/125543412 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a8ffff640 1 --2- 172.21.15.66:0/125543412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a9806c690 0x7f2a980cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a96d9d640 1 --2- 172.21.15.66:0/125543412 >> v2:172.21.15.117:3300/0 conn(0x7f2a980d0300 0x7f2a9815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a95d9b640 1 -- 172.21.15.66:0/125543412 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2200852503 0 0) 0x7f2a981624f0 con 0x7f2a98155990 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a95d9b640 1 -- 172.21.15.66:0/125543412 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a84003520 con 0x7f2a98155990 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a8ffff640 1 -- 172.21.15.66:0/125543412 >> v2:172.21.15.117:3300/0 conn(0x7f2a980d0300 msgr2=0x7f2a9815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a8ffff640 1 --2- 172.21.15.66:0/125543412 >> v2:172.21.15.117:3300/0 conn(0x7f2a980d0300 0x7f2a9815eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a8ffff640 1 -- 172.21.15.66:0/125543412 >> v1:172.21.15.66:6789/0 conn(0x7f2a98155990 legacy=0x7f2a980b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.402+0000 7f2a8ffff640 1 -- 172.21.15.66:0/125543412 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981627d0 con 0x7f2a9806c690 2024-03-30T13:10:07.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.403+0000 7f2a8ffff640 1 --2- 172.21.15.66:0/125543412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a9806c690 0x7f2a980cfdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a800097a0 tx=0x7f2a8002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea869910ab459a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.403+0000 7f2a95d9b640 1 -- 172.21.15.66:0/125543412 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a800315a0 con 0x7f2a9806c690 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.403+0000 7f2a95d9b640 1 -- 172.21.15.66:0/125543412 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a80037070 con 0x7f2a9806c690 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.403+0000 7f2a95d9b640 1 -- 172.21.15.66:0/125543412 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80031960 con 0x7f2a9806c690 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 -- 172.21.15.66:0/125543412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a9806c690 msgr2=0x7f2a980cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 --2- 172.21.15.66:0/125543412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a9806c690 0x7f2a980cfdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a800097a0 tx=0x7f2a8002b840 comp rx=0 tx=0).stop 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 -- 172.21.15.66:0/125543412 shutdown_connections 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 --2- 172.21.15.66:0/125543412 >> v2:172.21.15.117:3300/0 conn(0x7f2a980d0300 0x7f2a9815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 --2- 172.21.15.66:0/125543412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a9806c690 0x7f2a980cfdc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 -- 172.21.15.66:0/125543412 >> 172.21.15.66:0/125543412 conn(0x7f2a980672c0 msgr2=0x7f2a980676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 -- 172.21.15.66:0/125543412 shutdown_connections 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.404+0000 7f2a9cece640 1 -- 172.21.15.66:0/125543412 wait complete. 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.405+0000 7f2a9cece640 1 Processor -- start 2024-03-30T13:10:07.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.405+0000 7f2a9cece640 1 -- start start 2024-03-30T13:10:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9cece640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a9806c690 0x7f2a9814e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9cece640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a980d0300 0x7f2a981570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9cece640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 0x7f2a98157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9cece640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a980cf9a0 con 0x7f2a98155990 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9cece640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98162d90 con 0x7f2a9806c690 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9cece640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2a980cfb20 con 0x7f2a980d0300 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 0x7f2a98157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 0x7f2a98157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34190/0 (socket says 172.21.15.66:34190) 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 -- 172.21.15.66:0/3163835909 learned_addr learned my addr 172.21.15.66:0/3163835909 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a8ffff640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a980d0300 0x7f2a981570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a9659c640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a9806c690 0x7f2a9814e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 -- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a980d0300 msgr2=0x7f2a981570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a980d0300 0x7f2a981570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 -- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a9806c690 msgr2=0x7f2a9814e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a9806c690 0x7f2a9814e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a96d9d640 1 -- 172.21.15.66:0/3163835909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a980c7b40 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.406+0000 7f2a8ffff640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a980d0300 0x7f2a981570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.407+0000 7f2a96d9d640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 0x7f2a98157620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009dd0 tx=0x7f2a9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.407+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90018020 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.407+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a90014480 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.407+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a980c6aa0 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.407+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90014870 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.407+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a980c7060 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.408+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98070790 con 0x7f2a98155990 2024-03-30T13:10:07.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.408+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2a90005d10 con 0x7f2a98155990 2024-03-30T13:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.409+0000 7f2a8effd640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2a7806c530 0x7f2a7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.409+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2a9008c320 con 0x7f2a98155990 2024-03-30T13:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.409+0000 7f2a9659c640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2a7806c530 0x7f2a7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.409+0000 7f2a9659c640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2a7806c530 0x7f2a7806e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005880 tx=0x7f2a88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.413+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2a98070790 con 0x7f2a98155990 2024-03-30T13:10:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.600+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a980cf5c0 con 0x7f2a98155990 2024-03-30T13:10:07.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.601+0000 7f2a8effd640 1 -- 172.21.15.66:0/3163835909 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f2a90022070 con 0x7f2a98155990 2024-03-30T13:10:07.605 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:07.605 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:07.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2a7806c530 msgr2=0x7f2a7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2a7806c530 0x7f2a7806e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005880 tx=0x7f2a88009040 comp rx=0 tx=0).stop 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 msgr2=0x7f2a98157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 0x7f2a98157620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009dd0 tx=0x7f2a9000c490 comp rx=0 tx=0).stop 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 shutdown_connections 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2a7806c530 0x7f2a7806e9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98155990 0x7f2a98157620 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a980d0300 0x7f2a981570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 --2- 172.21.15.66:0/3163835909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a9806c690 0x7f2a9814e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.603+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 >> 172.21.15.66:0/3163835909 conn(0x7f2a980672c0 msgr2=0x7f2a980676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.604+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 shutdown_connections 2024-03-30T13:10:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.604+0000 7f2a9cece640 1 -- 172.21.15.66:0/3163835909 wait complete. 2024-03-30T13:10:07.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-30T13:10:07.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-30T13:10:07.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-30T13:10:07.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-30T13:10:07.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-30T13:10:07.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-30T13:10:07.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-30T13:10:07.622 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-30T13:10:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.801+0000 7f1b9089a640 1 Processor -- start 2024-03-30T13:10:07.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.801+0000 7f1b9089a640 1 -- start start 2024-03-30T13:10:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.802+0000 7f1b9089a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b8c1605a0 0x7f1b8c0d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.802+0000 7f1b9089a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c070f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.802+0000 7f1b9089a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8c163df0 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.802+0000 7f1b9089a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f1b8c146060 con 0x7f1b8c1605a0 2024-03-30T13:10:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.802+0000 7f1b9089a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c163bb0 con 0x7f1b8c163760 2024-03-30T13:10:07.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b8c1605a0 0x7f1b8c0d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b8c1605a0 0x7f1b8c0d2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34826/0 (socket says 172.21.15.66:34826) 2024-03-30T13:10:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 -- 172.21.15.66:0/2835419466 learned_addr learned my addr 172.21.15.66:0/2835419466 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8ad9d640 1 --2- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 -- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 msgr2=0x7f1b8c070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 --2- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 -- 172.21.15.66:0/2835419466 >> v1:172.21.15.66:6789/0 conn(0x7f1b8c0d26c0 legacy=0x7f1b8c163050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f1b8a59c640 1 -- 172.21.15.66:0/2835419466 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c167330 con 0x7f1b8c1605a0 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.804+0000 7f1b8a59c640 1 --2- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c0d2180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0681c0 tx=0x7f1b8402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=675beffeec648efa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f1b8959a640 1 -- 172.21.15.66:0/2835419466 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8402c9c0 con 0x7f1b8c1605a0 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f1b8959a640 1 -- 172.21.15.66:0/2835419466 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8402cb60 con 0x7f1b8c1605a0 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f11dd7f2640 1 Processor -- start 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7f11dd7f2640 1 -- start start 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11dd7f2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11d80735e0 0x7f11d8169260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11dd7f2640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d80c9760 con 0x7f11d80cf140 2024-03-30T13:10:07.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11dd7f2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d80c9a40 con 0x7f11d80ced60 2024-03-30T13:10:07.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11dd7f2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f11d81480c0 con 0x7f11d80735e0 2024-03-30T13:10:07.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7fa51bffb640 1 Processor -- start 2024-03-30T13:10:07.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.803+0000 7fa51bffb640 1 -- start start 2024-03-30T13:10:07.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.804+0000 7f14e89d5640 1 Processor -- start 2024-03-30T13:10:07.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.804+0000 7f14e89d5640 1 -- start start 2024-03-30T13:10:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f1b8959a640 1 -- 172.21.15.66:0/2835419466 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b840343f0 con 0x7f1b8c1605a0 2024-03-30T13:10:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11d77fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11d80735e0 0x7f11d8169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11d67fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f11d80cf140 0x7f11d8072ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53844/0 (socket says 172.21.15.66:53844) 2024-03-30T13:10:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11d77fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11d80735e0 0x7f11d8169260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34834/0 (socket says 172.21.15.66:34834) 2024-03-30T13:10:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11d67fc640 1 -- 172.21.15.66:0/2854999820 learned_addr learned my addr 172.21.15.66:0/2854999820 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7fa51bffb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 0x7fa51414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f14e89d5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14e4162f20 0x7f14e40b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e89d5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f14e41675a0 0x7f14e4166990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e89d5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e89d5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f14e4167340 con 0x7f14e41675a0 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e89d5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f14e4163300 con 0x7f14e4162f20 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e89d5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f14e40b77c0 con 0x7f14e4166ed0 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14e4162f20 0x7f14e40b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e259c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14e4162f20 0x7f14e40b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34846/0 (socket says 172.21.15.66:34846) 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e259c640 1 -- 172.21.15.66:0/1948777336 learned_addr learned my addr 172.21.15.66:0/1948777336 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f1b9089a640 1 -- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c1605a0 msgr2=0x7f1b8c0d2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f1b9089a640 1 --2- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c0d2180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0681c0 tx=0x7f1b8402be20 comp rx=0 tx=0).stop 2024-03-30T13:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa519d97640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 0x7fa51414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa519d97640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 0x7fa51414bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39114/0 (socket says 172.21.15.66:39114) 2024-03-30T13:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f1b9089a640 1 -- 172.21.15.66:0/2835419466 shutdown_connections 2024-03-30T13:10:07.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f14e1d9b640 1 --2- 172.21.15.66:0/1948777336 >> v2:172.21.15.66:3300/0 conn(0x7f14e41675a0 0x7f14e4166990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa51bffb640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa51406db60 0x7fa514145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa519d97640 1 -- 172.21.15.66:0/1156952006 learned_addr learned my addr 172.21.15.66:0/1156952006 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa514148070 con 0x7fa51414c050 2024-03-30T13:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa514148310 con 0x7fa51406db60 2024-03-30T13:10:07.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa5141485f0 con 0x7fa514162630 2024-03-30T13:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f1b9089a640 1 --2- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f1b9089a640 1 --2- 172.21.15.66:0/2835419466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c0d2180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f1b9089a640 1 -- 172.21.15.66:0/2835419466 >> 172.21.15.66:0/2835419466 conn(0x7f1b8c06f8b0 msgr2=0x7f1b8c06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f5c80d40640 1 Processor -- start 2024-03-30T13:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057904224 0 0) 0x7f11d80c9760 con 0x7f11d80cf140 2024-03-30T13:10:07.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.805+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11c0003520 con 0x7f11d80cf140 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 598664771 0 0) 0x7f11c0003520 con 0x7f11d80cf140 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 >> v2:172.21.15.117:3300/0 conn(0x7f11d80735e0 msgr2=0x7f11d8169260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f11d5ffb640 1 --2- 172.21.15.66:0/2854999820 >> v2:172.21.15.117:3300/0 conn(0x7f11d80735e0 0x7f11d8169260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.806+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 >> v1:172.21.15.117:6789/0 conn(0x7f11d80ced60 legacy=0x7f11d8162af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80c9d20 con 0x7f11d80cf140 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f11cc001670 con 0x7f11d80cf140 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 >> v1:172.21.15.66:6789/0 conn(0x7f11d80cf140 legacy=0x7f11d8072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 0x7f11c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11c00141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11c00148f0 con 0x7f11c0004800 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11c0005990 con 0x7f11c00051c0 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f11c0005b90 con 0x7f11c00039b0 2024-03-30T13:10:07.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d6ffd640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 0x7f11c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f11d67fc640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7fa51a598640 1 --2- 172.21.15.66:0/1156952006 >> v2:172.21.15.117:3300/0 conn(0x7fa51406db60 0x7fa514145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7fa51a598640 1 -- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 msgr2=0x7fa51414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7fa51a598640 1 --2- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 0x7fa51414bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7fa51a598640 1 -- 172.21.15.66:0/1156952006 >> v1:172.21.15.66:6789/0 conn(0x7fa51414c050 legacy=0x7fa51406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7fa51a598640 1 -- 172.21.15.66:0/1156952006 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5141488d0 con 0x7fa51406db60 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7fa519d97640 1 --2- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 0x7fa51414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:07.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e259c640 1 -- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 msgr2=0x7f14e40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e259c640 1 --2- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e40d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e259c640 1 -- 172.21.15.66:0/1948777336 >> v2:172.21.15.66:3300/0 conn(0x7f14e41675a0 msgr2=0x7f14e4166990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e259c640 1 --2- 172.21.15.66:0/1948777336 >> v2:172.21.15.66:3300/0 conn(0x7f14e41675a0 0x7f14e4166990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e259c640 1 -- 172.21.15.66:0/1948777336 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e40d3d20 con 0x7f14e4162f20 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e2d9d640 1 --2- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e40d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f14e1d9b640 1 --2- 172.21.15.66:0/1948777336 >> v2:172.21.15.66:3300/0 conn(0x7f14e41675a0 0x7f14e4166990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f0b6f59e640 1 Processor -- start 2024-03-30T13:10:07.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f5c80d40640 1 -- start start 2024-03-30T13:10:07.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f6b7e9b5640 1 Processor -- start 2024-03-30T13:10:07.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f6b7e9b5640 1 -- start start 2024-03-30T13:10:07.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d77fe640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11c00141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d67fc640 1 -- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 msgr2=0x7f11c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d67fc640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11c00039b0 0x7f11c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d67fc640 1 -- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 msgr2=0x7f11c00141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d67fc640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11c00051c0 0x7f11c00141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d67fc640 1 -- 172.21.15.66:0/2854999820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80c9d20 con 0x7f11c0004800 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f11d77fe640 1 -- 172.21.15.66:0/2854999820 reap_dead start 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.807+0000 7f0b6f59e640 1 -- start start 2024-03-30T13:10:07.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f14e259c640 1 --2- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e40b7280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14e40b7020 tx=0x7f14d402b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5491595352bae9fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f5c80d40640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5c7c0d0b40 0x7f5c7c16d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f0b6f59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b68006b50 0x7f0b68007f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f0b6f59e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b680342f0 0x7f0b68006f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f0b6f59e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b68055d10 con 0x7f0b680dfa60 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f0b6f59e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0b680069d0 con 0x7f0b68006b50 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f0b6f59e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f0b68055ff0 con 0x7f0b680342f0 2024-03-30T13:10:07.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.808+0000 7f1b9089a640 1 -- 172.21.15.66:0/2835419466 shutdown_connections 2024-03-30T13:10:07.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6dd9b640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0b680dfa60 0x7f0b680062c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:48414/0 (socket says 172.21.15.66:48414) 2024-03-30T13:10:07.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6dd9b640 1 -- 172.21.15.66:0/1910211354 learned_addr learned my addr 172.21.15.66:0/1910211354 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f14e159a640 1 -- 172.21.15.66:0/1948777336 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4035020 con 0x7f14e4162f20 2024-03-30T13:10:07.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/1910211354 >> v2:172.21.15.117:3300/0 conn(0x7f0b680342f0 0x7f0b68006f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f14e159a640 1 -- 172.21.15.66:0/1948777336 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d4031880 con 0x7f14e4162f20 2024-03-30T13:10:07.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f14e159a640 1 -- 172.21.15.66:0/1948777336 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4031c50 con 0x7f14e4162f20 2024-03-30T13:10:07.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7fa51a598640 1 --2- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51406db60 0x7fa514145f40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009dc0 tx=0x7fa50c02be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c03c64d67935611 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6e59c640 1 --2- 172.21.15.66:0/1910211354 >> v2:172.21.15.66:3300/0 conn(0x7f0b68006b50 0x7f0b68007f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f6b7e9b5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6b78145090 0x7f6b780b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f11d67fc640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c210 tx=0x7f11c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cbf116975483a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6d59a640 1 -- 172.21.15.66:0/1910211354 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382389132 0 0) 0x7f0b68055d10 con 0x7f0b680dfa60 2024-03-30T13:10:07.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6d59a640 1 -- 172.21.15.66:0/1910211354 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b58003520 con 0x7f0b680dfa60 2024-03-30T13:10:07.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7fa518d95640 1 -- 172.21.15.66:0/1156952006 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c039070 con 0x7fa51406db60 2024-03-30T13:10:07.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7fa518d95640 1 -- 172.21.15.66:0/1156952006 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c02cc40 con 0x7fa51406db60 2024-03-30T13:10:07.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f6b7e9b5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6b78160540 0x7f6b780d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7fa518d95640 1 -- 172.21.15.66:0/1156952006 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c039070 con 0x7fa51406db60 2024-03-30T13:10:07.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f6b777fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6b780b5aa0 0x7f6b7815fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:48434/0 (socket says 172.21.15.66:48434) 2024-03-30T13:10:07.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc007d00 con 0x7f11c0004800 2024-03-30T13:10:07.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11cc030070 con 0x7f11c0004800 2024-03-30T13:10:07.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f11d5ffb640 1 -- 172.21.15.66:0/2854999820 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc02cdc0 con 0x7f11c0004800 2024-03-30T13:10:07.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f6b777fe640 1 -- 172.21.15.66:0/572280315 learned_addr learned my addr 172.21.15.66:0/572280315 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f6b7cf52640 1 --2- 172.21.15.66:0/572280315 >> v2:172.21.15.117:3300/0 conn(0x7f6b78160540 0x7f6b780d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 -- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 msgr2=0x7f14e40b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e40b7280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14e40b7020 tx=0x7f14d402b920 comp rx=0 tx=0).stop 2024-03-30T13:10:07.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f5c80d40640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7c0722b0 con 0x7f5c7c145b80 2024-03-30T13:10:07.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f5c80d40640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7c072590 con 0x7f5c7c0cc7e0 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f5c80d40640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7c072870 con 0x7f5c7c0d0b40 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f5c7a59c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5c7c145b80 0x7f5c7c071940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53860/0 (socket says 172.21.15.66:53860) 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f5c7a59c640 1 -- 172.21.15.66:0/1594156039 learned_addr learned my addr 172.21.15.66:0/1594156039 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f5c7ad9d640 1 --2- 172.21.15.66:0/1594156039 >> v2:172.21.15.66:3300/0 conn(0x7f5c7c0d0b40 0x7f5c7c16d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/1594156039 >> v1:172.21.15.117:6789/0 conn(0x7f5c7c0cc7e0 legacy=0x7f5c7c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/1594156039 >> v1:172.21.15.66:6789/0 conn(0x7f5c7c145b80 legacy=0x7f5c7c071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/1594156039 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c06da40 con 0x7f5c7c0d0b40 2024-03-30T13:10:07.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 -- 172.21.15.66:0/1948777336 shutdown_connections 2024-03-30T13:10:07.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e40d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1948777336 >> v2:172.21.15.66:3300/0 conn(0x7f14e41675a0 0x7f14e4166990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1948777336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e40b7280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/1910211354 >> v2:172.21.15.66:3300/0 conn(0x7f0b68006b50 msgr2=0x7f0b68007f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/1910211354 >> v2:172.21.15.66:3300/0 conn(0x7f0b68006b50 0x7f0b68007f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f14e89d5640 1 -- 172.21.15.66:0/1948777336 >> 172.21.15.66:0/1948777336 conn(0x7f14e40b5f40 msgr2=0x7f14e40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/1910211354 >> v1:172.21.15.117:6789/0 conn(0x7f0b680dfa60 legacy=0x7f0b680062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/1910211354 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b68036a80 con 0x7f0b680342f0 2024-03-30T13:10:07.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f1b9089a640 1 -- 172.21.15.66:0/2835419466 wait complete. 2024-03-30T13:10:07.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b780b5f80 con 0x7f6b780b5aa0 2024-03-30T13:10:07.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f11dd7f2640 1 -- 172.21.15.66:0/2854999820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 msgr2=0x7f11c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f14e89d5640 1 -- 172.21.15.66:0/1948777336 shutdown_connections 2024-03-30T13:10:07.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f6b77fff640 1 --2- 172.21.15.66:0/572280315 >> v2:172.21.15.66:3300/0 conn(0x7f6b78145090 0x7f6b780b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6b780b6260 con 0x7f6b78145090 2024-03-30T13:10:07.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/1910211354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680342f0 0x7f0b68006f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b680553a0 tx=0x7f0b6402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f5b396ffec84c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f88a3288640 1 Processor -- start 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.809+0000 7f88a3288640 1 -- start start 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.810+0000 7f88a3288640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f889c15a720 0x7f889c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a3288640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f889c0ca820 0x7f889c0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a3288640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889c0ce860 con 0x7f889c0ccec0 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a3288640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f889c0cf470 con 0x7f889c15a720 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a3288640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f889c0ceb40 con 0x7f889c0ca820 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a1825640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f889c0ccec0 0x7f889c06d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53872/0 (socket says 172.21.15.66:53872) 2024-03-30T13:10:07.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a1825640 1 -- 172.21.15.66:0/218795803 learned_addr learned my addr 172.21.15.66:0/218795803 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f14e89d5640 1 -- 172.21.15.66:0/1948777336 wait complete. 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f6b76ffd640 1 -- 172.21.15.66:0/572280315 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713481411 0 0) 0x7f6b780b5f80 con 0x7f6b780b5aa0 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f6b780b6540 con 0x7f6b78160540 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a0823640 1 --2- 172.21.15.66:0/218795803 >> v2:172.21.15.117:3300/0 conn(0x7f889c0ca820 0x7f889c0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f8893fff640 1 -- 172.21.15.66:0/218795803 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153101858 0 0) 0x7f889c0ce860 con 0x7f889c0ccec0 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f88a1024640 1 --2- 172.21.15.66:0/218795803 >> v2:172.21.15.66:3300/0 conn(0x7f889c15a720 0x7f889c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f8893fff640 1 -- 172.21.15.66:0/218795803 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8888003500 con 0x7f889c0ccec0 2024-03-30T13:10:07.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f1b9089a640 1 Processor -- start 2024-03-30T13:10:07.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c210 tx=0x7f11c0005830 comp rx=0 tx=0).stop 2024-03-30T13:10:07.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f11dd7f2640 1 -- 172.21.15.66:0/2854999820 shutdown_connections 2024-03-30T13:10:07.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/2854999820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11c0004c80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f11dd7f2640 1 -- 172.21.15.66:0/2854999820 >> 172.21.15.66:0/2854999820 conn(0x7f11d80b1c90 msgr2=0x7f11d80b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51406db60 msgr2=0x7fa514145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 --2- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51406db60 0x7fa514145f40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009dc0 tx=0x7fa50c02be90 comp rx=0 tx=0).stop 2024-03-30T13:10:07.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f0b6d59a640 1 -- 172.21.15.66:0/1910211354 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64035020 con 0x7f0b680342f0 2024-03-30T13:10:07.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f0b6d59a640 1 -- 172.21.15.66:0/1910211354 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64031640 con 0x7f0b680342f0 2024-03-30T13:10:07.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f5c7ad9d640 1 --2- 172.21.15.66:0/1594156039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c0d0b40 0x7f5c7c16d4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400b050 tx=0x7f5c7402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c984b4012b4f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f5c7959a640 1 -- 172.21.15.66:0/1594156039 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74039070 con 0x7f5c7c0d0b40 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f5c7959a640 1 -- 172.21.15.66:0/1594156039 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7402cb90 con 0x7f5c7c0d0b40 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f5c7959a640 1 -- 172.21.15.66:0/1594156039 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74034440 con 0x7f5c7c0d0b40 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f6b76ffd640 1 -- 172.21.15.66:0/572280315 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b60003520 con 0x7f6b780b5aa0 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 shutdown_connections 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 --2- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51406db60 0x7fa514145f40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 --2- 172.21.15.66:0/1156952006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa514162630 0x7fa51414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 >> 172.21.15.66:0/1156952006 conn(0x7fa5140b1d90 msgr2=0x7fa5140b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f6b7cf52640 1 -- 172.21.15.66:0/572280315 >> v2:172.21.15.66:3300/0 conn(0x7f6b78145090 msgr2=0x7f6b780b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f11dd7f2640 1 -- 172.21.15.66:0/2854999820 shutdown_connections 2024-03-30T13:10:07.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 shutdown_connections 2024-03-30T13:10:07.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f0b6f59e640 1 -- 172.21.15.66:0/1910211354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680342f0 msgr2=0x7f0b68006f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f0b6f59e640 1 --2- 172.21.15.66:0/1910211354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680342f0 0x7f0b68006f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b680553a0 tx=0x7f0b6402b8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f6b76ffd640 1 -- 172.21.15.66:0/572280315 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2448534580 0 0) 0x7f6b60003520 con 0x7f6b780b5aa0 2024-03-30T13:10:07.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f0b6f59e640 1 -- 172.21.15.66:0/1910211354 shutdown_connections 2024-03-30T13:10:07.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f0b6f59e640 1 --2- 172.21.15.66:0/1910211354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680342f0 0x7f0b68006f30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f0b6f59e640 1 --2- 172.21.15.66:0/1910211354 >> v2:172.21.15.66:3300/0 conn(0x7f0b68006b50 0x7f0b68007f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f0b6f59e640 1 -- 172.21.15.66:0/1910211354 >> 172.21.15.66:0/1910211354 conn(0x7f0b6801e6c0 msgr2=0x7f0b680553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f6b7cf52640 1 --2- 172.21.15.66:0/572280315 >> v2:172.21.15.66:3300/0 conn(0x7f6b78145090 0x7f6b780b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.811+0000 7f1b9089a640 1 -- start start 2024-03-30T13:10:07.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f1b9089a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c0d26c0 0x7f1b8c13e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f1b9089a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f6b7cf52640 1 -- 172.21.15.66:0/572280315 >> v1:172.21.15.117:6789/0 conn(0x7f6b780b5aa0 legacy=0x7f6b7815fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f1b9089a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f1b9089a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c166b30 con 0x7f1b8c1605a0 2024-03-30T13:10:07.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f1b9089a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c166850 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f1b9089a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c166e10 con 0x7f1b8c163760 2024-03-30T13:10:07.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f5c80d40640 1 -- 172.21.15.66:0/1594156039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c0d0b40 msgr2=0x7f5c7c16d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c0d26c0 0x7f1b8c13e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c0d26c0 0x7f1b8c13e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34910/0 (socket says 172.21.15.66:34910) 2024-03-30T13:10:07.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f5c80d40640 1 --2- 172.21.15.66:0/1594156039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c0d0b40 0x7f5c7c16d4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400b050 tx=0x7f5c7402bff0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 -- 172.21.15.66:0/4177002717 learned_addr learned my addr 172.21.15.66:0/4177002717 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7fa51bffb640 1 -- 172.21.15.66:0/1156952006 wait complete. 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8ad9d640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b89d9b640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 -- 172.21.15.66:0/4177002717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 msgr2=0x7f1b8c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8c163760 0x7f1b8c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 -- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b8c1605a0 msgr2=0x7f1b8c140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f6b7cf52640 1 -- 172.21.15.66:0/572280315 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b780b6820 con 0x7f6b78160540 2024-03-30T13:10:07.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 -- 172.21.15.66:0/4177002717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c1416e0 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f6b77fff640 1 --2- 172.21.15.66:0/572280315 >> v2:172.21.15.66:3300/0 conn(0x7f6b78145090 0x7f6b780b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:07.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b8a59c640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c0d26c0 0x7f1b8c13e520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b84036040 tx=0x7f1b84004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b89d9b640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b8c1605a0 0x7f1b8c140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:07.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84043020 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f14e89d5640 1 Processor -- start 2024-03-30T13:10:07.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.813+0000 7f1b9089a640 1 -- 172.21.15.66:0/4177002717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c1419c0 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f1b9089a640 1 -- 172.21.15.66:0/4177002717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c0b3640 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8403e450 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f88a0823640 1 -- 172.21.15.66:0/218795803 >> v2:172.21.15.66:3300/0 conn(0x7f889c15a720 msgr2=0x7f889c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:07.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8403e820 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f88a0823640 1 --2- 172.21.15.66:0/218795803 >> v2:172.21.15.66:3300/0 conn(0x7f889c15a720 0x7f889c0ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7fa51bffb640 1 Processor -- start 2024-03-30T13:10:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f5c80d40640 1 -- 172.21.15.66:0/1594156039 shutdown_connections 2024-03-30T13:10:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f5c80d40640 1 --2- 172.21.15.66:0/1594156039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c0d0b40 0x7f5c7c16d4c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f11dd7f2640 1 -- 172.21.15.66:0/2854999820 wait complete. 2024-03-30T13:10:07.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f5c80d40640 1 -- 172.21.15.66:0/1594156039 >> 172.21.15.66:0/1594156039 conn(0x7f5c7c06b580 msgr2=0x7f5c7c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f11dd7f2640 1 Processor -- start 2024-03-30T13:10:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f88a0823640 1 -- 172.21.15.66:0/218795803 >> v1:172.21.15.66:6789/0 conn(0x7f889c0ccec0 legacy=0x7f889c06d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f88a0823640 1 -- 172.21.15.66:0/218795803 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c0cee20 con 0x7f889c0ca820 2024-03-30T13:10:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.812+0000 7f88a1024640 1 --2- 172.21.15.66:0/218795803 >> v2:172.21.15.66:3300/0 conn(0x7f889c15a720 0x7f889c0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7fa51bffb640 1 -- start start 2024-03-30T13:10:07.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f88a0823640 1 --2- 172.21.15.66:0/218795803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ca820 0x7f889c0cc980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f889c0b28d0 tx=0x7f889802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381c89cb52c3b42e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f0b6f59e640 1 -- 172.21.15.66:0/1910211354 shutdown_connections 2024-03-30T13:10:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f14e89d5640 1 -- start start 2024-03-30T13:10:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f8893fff640 1 -- 172.21.15.66:0/218795803 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898035020 con 0x7f889c0ca820 2024-03-30T13:10:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f8893fff640 1 -- 172.21.15.66:0/218795803 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8898031830 con 0x7f889c0ca820 2024-03-30T13:10:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f8893fff640 1 -- 172.21.15.66:0/218795803 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898031c00 con 0x7f889c0ca820 2024-03-30T13:10:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f0b6f59e640 1 -- 172.21.15.66:0/1910211354 wait complete. 2024-03-30T13:10:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.815+0000 7f1b9089a640 1 -- 172.21.15.66:0/4177002717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b70001030 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.814+0000 7f11dd7f2640 1 -- start start 2024-03-30T13:10:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.815+0000 7f11dd7f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11d8161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.815+0000 7f5c80d40640 1 -- 172.21.15.66:0/1594156039 shutdown_connections 2024-03-30T13:10:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.815+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b84046020 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa51bffb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51406db60 0x7fa5141659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11d6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11d8161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e89d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e89d5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e41fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa51bffb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51414c050 0x7fa5140b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11d6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11d8161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34254/0 (socket says 172.21.15.66:34254) 2024-03-30T13:10:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f88a3288640 1 -- 172.21.15.66:0/218795803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ca820 msgr2=0x7f889c0cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f88a3288640 1 --2- 172.21.15.66:0/218795803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ca820 0x7f889c0cc980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f889c0b28d0 tx=0x7f889802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa519d97640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51406db60 0x7fa5141659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa519d97640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51406db60 0x7fa5141659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39152/0 (socket says 172.21.15.66:39152) 2024-03-30T13:10:07.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa519596640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51414c050 0x7fa5140b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa519596640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51414c050 0x7fa5140b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34930/0 (socket says 172.21.15.66:34930) 2024-03-30T13:10:07.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e41fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e41fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39168/0 (socket says 172.21.15.66:39168) 2024-03-30T13:10:07.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa51bffb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa514162630 0x7fa51413fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa51bffb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa514162a10 con 0x7fa514162630 2024-03-30T13:10:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa519d97640 1 -- 172.21.15.66:0/3225981499 learned_addr learned my addr 172.21.15.66:0/3225981499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa51bffb640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140afa60 con 0x7fa51414c050 2024-03-30T13:10:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7fa51bffb640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa514162b90 con 0x7fa51406db60 2024-03-30T13:10:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e41526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34924/0 (socket says 172.21.15.66:34924) 2024-03-30T13:10:07.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f6b7cf52640 1 --2- 172.21.15.66:0/572280315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 0x7f6b780d11c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b780d01e0 tx=0x7f6b7002bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc03a0137404b93 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11d6ffd640 1 -- 172.21.15.66:0/1172962721 learned_addr learned my addr 172.21.15.66:0/1172962721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e1d9b640 1 -- 172.21.15.66:0/1216833375 learned_addr learned my addr 172.21.15.66:0/1216833375 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d815fd30 0x7f11d8130750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f0b6f59e640 1 Processor -- start 2024-03-30T13:10:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f88a3288640 1 -- 172.21.15.66:0/218795803 shutdown_connections 2024-03-30T13:10:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f88a3288640 1 --2- 172.21.15.66:0/218795803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ca820 0x7f889c0cc980 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f889c0b28d0 tx=0x7f889802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f88a3288640 1 --2- 172.21.15.66:0/218795803 >> v2:172.21.15.66:3300/0 conn(0x7f889c15a720 0x7f889c0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f88a3288640 1 -- 172.21.15.66:0/218795803 >> 172.21.15.66:0/218795803 conn(0x7f889c0b1c90 msgr2=0x7f889c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f1b7affd640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1b6c06c530 0x7f1b6c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14e41675a0 0x7f14e40d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d813e7b0 0x7f11d80cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40b9c00 con 0x7f14e41675a0 2024-03-30T13:10:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40b9920 con 0x7f14e4162f20 2024-03-30T13:10:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f14e40d2870 con 0x7f14e4166ed0 2024-03-30T13:10:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f6b76ffd640 1 -- 172.21.15.66:0/572280315 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70039070 con 0x7f6b78160540 2024-03-30T13:10:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11d67fc640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d815fd30 0x7f11d8130750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8072ed0 con 0x7f11c0004800 2024-03-30T13:10:07.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f11d77fe640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d813e7b0 0x7f11d80cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f6b76ffd640 1 -- 172.21.15.66:0/572280315 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7002c830 con 0x7f6b78160540 2024-03-30T13:10:07.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80ca2e0 con 0x7f11d815fd30 2024-03-30T13:10:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa519596640 1 -- 172.21.15.66:0/3225981499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51406db60 msgr2=0x7fa5141659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa519596640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa51406db60 0x7fa5141659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa519596640 1 -- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa514162630 msgr2=0x7fa51413fa60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:10:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f11d8073050 con 0x7f11d813e7b0 2024-03-30T13:10:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f6b76ffd640 1 -- 172.21.15.66:0/572280315 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68001670 con 0x7f6b78160540 2024-03-30T13:10:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1b840ae2e0 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f1b7affd640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f1b6c075c30 0x7f1b6c078070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e1d9b640 1 -- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 msgr2=0x7f14e41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e1d9b640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4162f20 0x7f14e41526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e1d9b640 1 -- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14e41675a0 msgr2=0x7f14e40d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 --> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b8403ae50 con 0x7f1b6c075c30 2024-03-30T13:10:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e2d9d640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14e41675a0 0x7f14e40d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b840b5050 con 0x7f1b8c0d26c0 2024-03-30T13:10:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f1b8ad9d640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f1b6c075c30 0x7f1b6c078070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e1d9b640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14e41675a0 0x7f14e40d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e1d9b640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e41fc820 con 0x7f14e4166ed0 2024-03-30T13:10:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa51a598640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa514162630 0x7fa51413fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.815+0000 7f5c80d40640 1 -- 172.21.15.66:0/1594156039 wait complete. 2024-03-30T13:10:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f5c80d40640 1 Processor -- start 2024-03-30T13:10:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f5c80d40640 1 -- start start 2024-03-30T13:10:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa519596640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa514162630 0x7fa51413fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa519596640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5140b1790 con 0x7fa51414c050 2024-03-30T13:10:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f88a3288640 1 -- 172.21.15.66:0/218795803 shutdown_connections 2024-03-30T13:10:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7fa51a598640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa514162630 0x7fa51413fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.817+0000 7f14e1d9b640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e41fbf20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f14d800e990 tx=0x7f14d800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f88a3288640 1 -- 172.21.15.66:0/218795803 wait complete. 2024-03-30T13:10:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f1b89d9b640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1b6c06c530 0x7f1b6c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7fa519596640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51414c050 0x7fa5140b1250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa514148240 tx=0x7fa51000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 msgr2=0x7f6b780d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f6b7e9b5640 1 --2- 172.21.15.66:0/572280315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 0x7f6b780d11c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b780d01e0 tx=0x7f6b7002bd00 comp rx=0 tx=0).stop 2024-03-30T13:10:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.816+0000 7f0b6f59e640 1 -- start start 2024-03-30T13:10:07.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f1b8ad9d640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f1b6c075c30 0x7f1b6c078070 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b68006b50 0x7f0b680dd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6f59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b680342f0 0x7f0b680dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f11d6ffd640 1 -- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d813e7b0 msgr2=0x7f11d80cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f11d6ffd640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11d813e7b0 0x7f11d80cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f11d6ffd640 1 -- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d815fd30 msgr2=0x7f11d8130750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f11d6ffd640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d815fd30 0x7f11d8130750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f11d6ffd640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d8160280 con 0x7f11c0004800 2024-03-30T13:10:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8017070 con 0x7f14e4166ed0 2024-03-30T13:10:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f11d67fc640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11d815fd30 0x7f11d8130750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6f59e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680dfa60 0x7f0b680db520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6f59e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b680366a0 con 0x7f0b680342f0 2024-03-30T13:10:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6f59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68035000 con 0x7f0b68006b50 2024-03-30T13:10:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6f59e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0b68036820 con 0x7f0b680dfa60 2024-03-30T13:10:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d800c9e0 con 0x7f14e4166ed0 2024-03-30T13:10:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b680342f0 0x7f0b680dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b680342f0 0x7f0b680dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34276/0 (socket says 172.21.15.66:34276) 2024-03-30T13:10:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6dd9b640 1 -- 172.21.15.66:0/2608266125 learned_addr learned my addr 172.21.15.66:0/2608266125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d800cd90 con 0x7f14e4166ed0 2024-03-30T13:10:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680dfa60 0x7f0b680db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f1b89d9b640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1b6c06c530 0x7f1b6c06e9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b80006fd0 tx=0x7f1b80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c80d40640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c7c0d0b40 0x7f5c7c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c80d40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c145b80 0x7f5c7c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c80d40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7c0cec80 0x7f5c7c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c80d40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c06f250 con 0x7f5c7c145b80 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c80d40640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c072050 con 0x7f5c7c0cec80 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c80d40640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c06f3d0 con 0x7f5c7c0d0b40 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c7a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c7c0d0b40 0x7f5c7c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c7ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7c0cec80 0x7f5c7c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c7ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7c0cec80 0x7f5c7c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34946/0 (socket says 172.21.15.66:34946) 2024-03-30T13:10:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/3553835832 learned_addr learned my addr 172.21.15.66:0/3553835832 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b68006b50 msgr2=0x7f0b680dd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b68006b50 0x7f0b680dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b680342f0 msgr2=0x7f0b680dafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b680342f0 0x7f0b680dafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510017070 con 0x7fa51414c050 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5100052d0 con 0x7fa51414c050 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.818+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510003830 con 0x7fa51414c050 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7fa51bffb640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51413ffa0 con 0x7fa51414c050 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c7c0d0b40 msgr2=0x7f5c7c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f5c7ad9d640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c7c0d0b40 0x7f5c7c0cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c145b80 msgr2=0x7f5c7c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f5c7ad9d640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c7c145b80 0x7f5c7c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/3553835832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c0cde10 con 0x7f5c7c0cec80 2024-03-30T13:10:07.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/2608266125 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b680dc880 con 0x7f0b680dfa60 2024-03-30T13:10:07.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== osd.1 v2:172.21.15.66:6808/1885904429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b8403ae50 con 0x7f1b6c075c30 2024-03-30T13:10:07.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7fa51bffb640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa514164f10 con 0x7fa51414c050 2024-03-30T13:10:07.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f5c7ad9d640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7c0cec80 0x7f5c7c15aae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c7402c530 tx=0x7f5c74004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e41fcb00 con 0x7f14e4166ed0 2024-03-30T13:10:07.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e4151ed0 con 0x7f14e4166ed0 2024-03-30T13:10:07.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 shutdown_connections 2024-03-30T13:10:07.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f6b7e9b5640 1 --2- 172.21.15.66:0/572280315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 0x7f6b780d11c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f6b7e9b5640 1 --2- 172.21.15.66:0/572280315 >> v2:172.21.15.66:3300/0 conn(0x7f6b78145090 0x7f6b780b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 >> 172.21.15.66:0/572280315 conn(0x7f6b7806f8b0 msgr2=0x7f6b7806fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f0b6ed9d640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680dfa60 0x7f0b680db520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0b64033000 tx=0x7f0b640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f11d6ffd640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11d8161320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11c800f850 tx=0x7f11c800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64040070 con 0x7f0b680dfa60 2024-03-30T13:10:07.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f0b6f59e640 1 -- 172.21.15.66:0/2608266125 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b680dcb60 con 0x7f0b680dfa60 2024-03-30T13:10:07.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f0b6f59e640 1 -- 172.21.15.66:0/2608266125 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b680dd120 con 0x7f0b680dfa60 2024-03-30T13:10:07.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.819+0000 7f88a3288640 1 Processor -- start 2024-03-30T13:10:07.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f88a3288640 1 -- start start 2024-03-30T13:10:07.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8001ce0 con 0x7f11c0004800 2024-03-30T13:10:07.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d8160560 con 0x7f11c0004800 2024-03-30T13:10:07.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c8018070 con 0x7f11c0004800 2024-03-30T13:10:07.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d8160b20 con 0x7f11c0004800 2024-03-30T13:10:07.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800d6d0 con 0x7f11c0004800 2024-03-30T13:10:07.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7fa51bffb640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5141655d0 con 0x7fa51414c050 2024-03-30T13:10:07.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74055020 con 0x7f5c7c0cec80 2024-03-30T13:10:07.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c740506a0 con 0x7f5c7c0cec80 2024-03-30T13:10:07.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74039040 con 0x7f5c7c0cec80 2024-03-30T13:10:07.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14e4075a10 con 0x7f14e4166ed0 2024-03-30T13:10:07.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c0cd570 con 0x7f5c7c0cec80 2024-03-30T13:10:07.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c0cdb30 con 0x7f5c7c0cec80 2024-03-30T13:10:07.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.820+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64039b30 con 0x7f0b680dfa60 2024-03-30T13:10:07.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64035040 con 0x7f0b680dfa60 2024-03-30T13:10:07.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b680dd500 con 0x7f0b680dfa60 2024-03-30T13:10:07.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 shutdown_connections 2024-03-30T13:10:07.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f14d801a070 con 0x7f14e4166ed0 2024-03-30T13:10:07.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa51001a070 con 0x7fa51414c050 2024-03-30T13:10:07.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7fa50a7fc640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa4f806c630 0x7fa4f806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f88a3288640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ccec0 0x7f889c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa51008ad20 con 0x7fa51414c050 2024-03-30T13:10:07.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f88a3288640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f889c15a720 0x7f889c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f88a3288640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c13f590 0x7f889c1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f88a3288640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f889c06f610 con 0x7f889c13f590 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f88a3288640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f889c073120 con 0x7f889c0ccec0 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.821+0000 7f88a3288640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f889c06f790 con 0x7f889c15a720 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c13f590 0x7f889c1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c13f590 0x7f889c1512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34284/0 (socket says 172.21.15.66:34284) 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 -- 172.21.15.66:0/3001896171 learned_addr learned my addr 172.21.15.66:0/3001896171 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1024640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ccec0 0x7f889c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 -- 172.21.15.66:0/3001896171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f889c15a720 msgr2=0x7f889c15ab90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f889c15a720 0x7f889c15ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 -- 172.21.15.66:0/3001896171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ccec0 msgr2=0x7f889c154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c0ccec0 0x7f889c154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.822+0000 7f88a1825640 1 -- 172.21.15.66:0/3001896171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c15b2c0 con 0x7f889c13f590 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f88a1825640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c13f590 0x7f889c1512a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f889400cb40 tx=0x7f889400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894017070 con 0x7f889c13f590 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889400aca0 con 0x7f889c13f590 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894012620 con 0x7f889c13f590 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c15b5a0 con 0x7f889c13f590 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c15bb60 con 0x7f889c13f590 2024-03-30T13:10:07.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f11c800d870 con 0x7f11c0004800 2024-03-30T13:10:07.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7fa50a7fc640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7fa4f8075d30 0x7fa4f8078170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa510009e50 con 0x7fa4f8075d30 2024-03-30T13:10:07.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f14d2ffd640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f14c406c630 0x7f14c406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f14d808ba70 con 0x7f14e4166ed0 2024-03-30T13:10:07.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f14d2ffd640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f14c4075db0 0x7f14c40781f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14d8016ec0 con 0x7f14c4075db0 2024-03-30T13:10:07.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14d80902a0 con 0x7f14e4166ed0 2024-03-30T13:10:07.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f14e2d9d640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f14c4075db0 0x7f14c40781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b64038020 con 0x7f0b680dfa60 2024-03-30T13:10:07.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c7c13e130 con 0x7f5c7c0cec80 2024-03-30T13:10:07.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/572280315 wait complete. 2024-03-30T13:10:07.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f11bf7fe640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f11c406c580 0x7f11c406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f0b5effd640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b5006c600 0x7f0b5006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7fa51a598640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7fa4f8075d30 0x7fa4f8078170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0b640ada70 con 0x7f0b680dfa60 2024-03-30T13:10:07.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f0b5effd640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7f0b50075d80 0x7f0b500781c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f14e2d9d640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f14c4075db0 0x7f14c40781f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 --> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b50078890 con 0x7f0b50075d80 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b640ade60 con 0x7f0b680dfa60 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.823+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5c74058020 con 0x7f5c7c0cec80 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f5c72ffd640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5c6406c530 0x7f5c6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5c740bf5d0 con 0x7f5c7c0cec80 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f5c72ffd640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5c64075cb0 0x7f5c640780f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 --> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c740506a0 con 0x7f5c64075cb0 2024-03-30T13:10:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5c740c6050 con 0x7f5c7c0cec80 2024-03-30T13:10:07.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f0b6dd9b640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7f0b50075d80 0x7f0b500781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.824+0000 7f5c79d9b640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5c64075cb0 0x7f5c640780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f5c7a59c640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5c6406c530 0x7f5c6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f14e259c640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f14c406c630 0x7f14c406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f6b7e9b5640 1 Processor -- start 2024-03-30T13:10:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7fa519d97640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa4f806c630 0x7fa4f806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa510054ba0 con 0x7fa51414c050 2024-03-30T13:10:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f0b6e59c640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b5006c600 0x7f0b5006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7fa51a598640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7fa4f8075d30 0x7fa4f8078170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f6b7e9b5640 1 -- start start 2024-03-30T13:10:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f5c79d9b640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5c64075cb0 0x7f5c640780f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f5c7a59c640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5c6406c530 0x7f5c6406e9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c007fd0 tx=0x7f5c6c007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.825+0000 7f0b6dd9b640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7f0b50075d80 0x7f0b500781c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== osd.5 v2:172.21.15.117:6802/1603026034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b50078890 con 0x7f0b50075d80 2024-03-30T13:10:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== osd.4 v2:172.21.15.117:6824/2408712778 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14d8016ec0 con 0x7f14c4075db0 2024-03-30T13:10:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== osd.6 v2:172.21.15.117:6800/698765583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c740506a0 con 0x7f5c64075cb0 2024-03-30T13:10:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f6b7e9b5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b780b5aa0 0x7f6b781f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f0b6e59c640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b5006c600 0x7f0b5006eac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b7002d940 tx=0x7f0b7006a0f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== osd.3 v2:172.21.15.66:6824/2124064569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa510009e50 con 0x7fa4f8075d30 2024-03-30T13:10:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f14e259c640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f14c406c630 0x7f14c406eaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14d4036000 tx=0x7f14d403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f11c808b6e0 con 0x7f11c0004800 2024-03-30T13:10:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f11d67fc640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f11c406c580 0x7f11c406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.826+0000 7f6b7e9b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b78145090 0x7f6b78140d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f6b7e9b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 0x7f6b781f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f11d67fc640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f11c406c580 0x7f11c406ea40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11cc034000 tx=0x7f11cc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b78145090 0x7f6b78140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b78145090 0x7f6b78140d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39214/0 (socket says 172.21.15.66:39214) 2024-03-30T13:10:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f6b777fe640 1 -- 172.21.15.66:0/2770768162 learned_addr learned my addr 172.21.15.66:0/2770768162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7fa519d97640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa4f806c630 0x7fa4f806eaf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa504009630 tx=0x7fa504009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7f11b8001650 0x7f11b8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11b8006bb0 con 0x7f11b8001650 2024-03-30T13:10:07.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f11d77fe640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7f11b8001650 0x7f11b8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f11d77fe640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7f11b8001650 0x7f11b8003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f887c001030 con 0x7f889c13f590 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f889401a070 con 0x7f889c13f590 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f8891ffb640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f888006c630 0x7f888006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f88a1024640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f888006c630 0x7f888006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f889408b840 con 0x7f889c13f590 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f8891ffb640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7f8880075db0 0x7f88800781f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 --> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f889400ae50 con 0x7f8880075db0 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8894092050 con 0x7f889c13f590 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f88a0823640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7f8880075db0 0x7f88800781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f88a1024640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f888006c630 0x7f888006eaf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78145470 con 0x7f6b780b5aa0 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.828+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78146000 con 0x7f6b78160540 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.827+0000 7f6b77fff640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b780b5aa0 0x7f6b781f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f6b7cf52640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 0x7f6b781f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.829+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6b781455f0 con 0x7f6b78145090 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f88a0823640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7f8880075db0 0x7f88800781f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== osd.2 v2:172.21.15.66:6800/948131126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11b8006bb0 con 0x7f11b8001650 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f6b777fe640 1 -- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 msgr2=0x7f6b781f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f6b777fe640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b78160540 0x7f6b781f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f6b777fe640 1 -- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b780b5aa0 msgr2=0x7f6b781f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f6b777fe640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b780b5aa0 0x7f6b781f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.830+0000 7f6b777fe640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b781f8350 con 0x7f6b78145090 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f6b777fe640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b78145090 0x7f6b78140d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800c6d0 tx=0x7f6b6800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== osd.0 v2:172.21.15.66:6816/3495913091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f889400ae50 con 0x7f8880075db0 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68005ad0 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b68005c70 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b780aed90 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68007270 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.831+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b78141270 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.834+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6b68007410 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.834+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b781f41c0 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.834+0000 7f6b74ff9640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b6406c560 0x7f6b6406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.834+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6b6808b700 con 0x7f6b78145090 2024-03-30T13:10:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.835+0000 7f6b74ff9640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7f6b64075c60 0x7f6b640780a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.835+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b64078770 con 0x7f6b64075c60 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.835+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b68092050 con 0x7f6b78145090 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.835+0000 7f6b77fff640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b6406c560 0x7f6b6406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.836+0000 7f6b77fff640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b6406c560 0x7f6b6406ea20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009560 tx=0x7f6b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.837+0000 7f6b7cf52640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7f6b64075c60 0x7f6b640780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.837+0000 7f6b7cf52640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7f6b64075c60 0x7f6b640780a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.838+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== osd.7 v2:172.21.15.117:6816/3946720435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b64078770 con 0x7f6b64075c60 2024-03-30T13:10:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.850+0000 7f1b9089a640 1 -- 172.21.15.66:0/4177002717 --> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b70002dd0 con 0x7f1b6c075c30 2024-03-30T13:10:07.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.851+0000 7f1b7affd640 1 -- 172.21.15.66:0/4177002717 <== osd.1 v2:172.21.15.66:6808/1885904429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b70002dd0 con 0x7f1b6c075c30 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.851+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f1b6c075c30 msgr2=0x7f1b6c078070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.851+0000 7f1b78ff9640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f1b6c075c30 0x7f1b6c078070 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1b6c06c530 msgr2=0x7f1b6c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b78ff9640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1b6c06c530 0x7f1b6c06e9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b80006fd0 tx=0x7f1b80008040 comp rx=0 tx=0).stop 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c0d26c0 msgr2=0x7f1b8c13e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b78ff9640 1 --2- 172.21.15.66:0/4177002717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b8c0d26c0 0x7f1b8c13e520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b84036040 tx=0x7f1b84004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b8ad9d640 1 -- 172.21.15.66:0/4177002717 reap_dead start 2024-03-30T13:10:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 shutdown_connections 2024-03-30T13:10:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.852+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 >> 172.21.15.66:0/4177002717 conn(0x7f1b8c06f8b0 msgr2=0x7f1b8c1613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.853+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 shutdown_connections 2024-03-30T13:10:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.853+0000 7f1b78ff9640 1 -- 172.21.15.66:0/4177002717 wait complete. 2024-03-30T13:10:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.858+0000 7fa51bffb640 1 -- 172.21.15.66:0/3225981499 --> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa514075890 con 0x7fa4f8075d30 2024-03-30T13:10:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.859+0000 7fa50a7fc640 1 -- 172.21.15.66:0/3225981499 <== osd.3 v2:172.21.15.66:6824/2124064569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa514075890 con 0x7fa4f8075d30 2024-03-30T13:10:07.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 --> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c7c075a10 con 0x7f5c64075cb0 2024-03-30T13:10:07.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7fa4f8075d30 msgr2=0x7fa4f8078170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa4f7fff640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7fa4f8075d30 0x7fa4f8078170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 --> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14e40b6710 con 0x7f14c4075db0 2024-03-30T13:10:07.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa4f806c630 msgr2=0x7fa4f806eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa4f7fff640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa4f806c630 0x7fa4f806eaf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa504009630 tx=0x7fa504009240 comp rx=0 tx=0).stop 2024-03-30T13:10:07.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51414c050 msgr2=0x7fa5140b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa4f7fff640 1 --2- 172.21.15.66:0/3225981499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51414c050 0x7fa5140b1250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa514148240 tx=0x7fa51000c820 comp rx=0 tx=0).stop 2024-03-30T13:10:07.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7fa51a598640 1 -- 172.21.15.66:0/3225981499 reap_dead start 2024-03-30T13:10:07.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.860+0000 7f5c72ffd640 1 -- 172.21.15.66:0/3553835832 <== osd.6 v2:172.21.15.117:6800/698765583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5c7c075a10 con 0x7f5c64075cb0 2024-03-30T13:10:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5c64075cb0 msgr2=0x7f5c640780f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f14d2ffd640 1 -- 172.21.15.66:0/1216833375 <== osd.4 v2:172.21.15.117:6824/2408712778 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f14e40b6710 con 0x7f14c4075db0 2024-03-30T13:10:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 shutdown_connections 2024-03-30T13:10:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 >> 172.21.15.66:0/3225981499 conn(0x7fa5140b1d90 msgr2=0x7fa51406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 --> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b68005390 con 0x7f0b50075d80 2024-03-30T13:10:07.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f5c80d40640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5c64075cb0 0x7f5c640780f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 shutdown_connections 2024-03-30T13:10:07.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f0b5effd640 1 -- 172.21.15.66:0/2608266125 <== osd.5 v2:172.21.15.117:6802/1603026034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b68005390 con 0x7f0b50075d80 2024-03-30T13:10:07.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7fa4f7fff640 1 -- 172.21.15.66:0/3225981499 wait complete. 2024-03-30T13:10:07.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5c6406c530 msgr2=0x7f5c6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.861+0000 7f5c80d40640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5c6406c530 0x7f5c6406e9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c007fd0 tx=0x7f5c6c007e80 comp rx=0 tx=0).stop 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f14c4075db0 msgr2=0x7f14c40781f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f14c4075db0 0x7f14c40781f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f14c406c630 msgr2=0x7f14c406eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f14c406c630 0x7f14c406eaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14d4036000 tx=0x7f14d403c040 comp rx=0 tx=0).stop 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 msgr2=0x7f14e41fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 --2- 172.21.15.66:0/1216833375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14e4166ed0 0x7f14e41fbf20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f14d800e990 tx=0x7f14d800ee50 comp rx=0 tx=0).stop 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e2d9d640 1 -- 172.21.15.66:0/1216833375 reap_dead start 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 shutdown_connections 2024-03-30T13:10:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 >> 172.21.15.66:0/1216833375 conn(0x7f14e40b5f40 msgr2=0x7f14e4164620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7c0cec80 msgr2=0x7f5c7c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f5c80d40640 1 --2- 172.21.15.66:0/3553835832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7c0cec80 0x7f5c7c15aae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c7402c530 tx=0x7f5c74004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f5c7ad9d640 1 -- 172.21.15.66:0/3553835832 reap_dead start 2024-03-30T13:10:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 shutdown_connections 2024-03-30T13:10:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f14e89d5640 1 -- 172.21.15.66:0/1216833375 wait complete. 2024-03-30T13:10:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 shutdown_connections 2024-03-30T13:10:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.862+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 >> 172.21.15.66:0/3553835832 conn(0x7f5c7c06b580 msgr2=0x7f5c7c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7f0b50075d80 msgr2=0x7f0b500781c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 shutdown_connections 2024-03-30T13:10:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7f0b50075d80 0x7f0b500781c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b5006c600 msgr2=0x7f0b5006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f5c80d40640 1 -- 172.21.15.66:0/3553835832 wait complete. 2024-03-30T13:10:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b5006c600 0x7f0b5006eac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b7002d940 tx=0x7f0b7006a0f0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680dfa60 msgr2=0x7f0b680db520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 --2- 172.21.15.66:0/2608266125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680dfa60 0x7f0b680db520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0b64033000 tx=0x7f0b640046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b6ed9d640 1 -- 172.21.15.66:0/2608266125 reap_dead start 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.863+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 shutdown_connections 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.864+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 >> 172.21.15.66:0/2608266125 conn(0x7f0b6801e6c0 msgr2=0x7f0b6800bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.864+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 shutdown_connections 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.864+0000 7f0b5cff9640 1 -- 172.21.15.66:0/2608266125 wait complete. 2024-03-30T13:10:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.864+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 --> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11b8005c40 con 0x7f11b8001650 2024-03-30T13:10:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.865+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 --> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f887c002dd0 con 0x7f8880075db0 2024-03-30T13:10:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.865+0000 7f11bf7fe640 1 -- 172.21.15.66:0/1172962721 <== osd.2 v2:172.21.15.66:6800/948131126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f11b8005c40 con 0x7f11b8001650 2024-03-30T13:10:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.865+0000 7f8891ffb640 1 -- 172.21.15.66:0/3001896171 <== osd.0 v2:172.21.15.66:6816/3495913091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f887c002dd0 con 0x7f8880075db0 2024-03-30T13:10:07.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7f11b8001650 msgr2=0x7f11b8003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7f11b8001650 0x7f11b8003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7f8880075db0 msgr2=0x7f88800781f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f88a3288640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7f8880075db0 0x7f88800781f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f888006c630 msgr2=0x7f888006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f88a3288640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f888006c630 0x7f888006eaf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).stop 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c13f590 msgr2=0x7f889c1512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f88a3288640 1 --2- 172.21.15.66:0/3001896171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f889c13f590 0x7f889c1512a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f889400cb40 tx=0x7f889400a060 comp rx=0 tx=0).stop 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f11c406c580 msgr2=0x7f11c406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.866+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f11c406c580 0x7f11c406ea40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11cc034000 tx=0x7f11cc009030 comp rx=0 tx=0).stop 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 msgr2=0x7f11d8161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11dd7f2640 1 --2- 172.21.15.66:0/1172962721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c0004800 0x7f11d8161320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11c800f850 tx=0x7f11c800fd10 comp rx=0 tx=0).stop 2024-03-30T13:10:07.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11d77fe640 1 -- 172.21.15.66:0/1172962721 reap_dead start 2024-03-30T13:10:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f88a1825640 1 -- 172.21.15.66:0/3001896171 reap_dead start 2024-03-30T13:10:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 shutdown_connections 2024-03-30T13:10:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 >> 172.21.15.66:0/1172962721 conn(0x7f11d80b1c90 msgr2=0x7f11d80b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 shutdown_connections 2024-03-30T13:10:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 >> 172.21.15.66:0/3001896171 conn(0x7f889c0b1c90 msgr2=0x7f889c06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 shutdown_connections 2024-03-30T13:10:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f11dd7f2640 1 -- 172.21.15.66:0/1172962721 wait complete. 2024-03-30T13:10:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 shutdown_connections 2024-03-30T13:10:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.867+0000 7f88a3288640 1 -- 172.21.15.66:0/3001896171 wait complete. 2024-03-30T13:10:07.934 INFO:teuthology.orchestra.run.smithi066.stdout:60129542153 2024-03-30T13:10:07.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T13:10:07.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.869+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 --> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b78075a10 con 0x7f6b64075c60 2024-03-30T13:10:07.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.869+0000 7f6b74ff9640 1 -- 172.21.15.66:0/2770768162 <== osd.7 v2:172.21.15.117:6816/3946720435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6b78075a10 con 0x7f6b64075c60 2024-03-30T13:10:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.870+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7f6b64075c60 msgr2=0x7f6b640780a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.870+0000 7f6b7e9b5640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7f6b64075c60 0x7f6b640780a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.870+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b6406c560 msgr2=0x7f6b6406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.870+0000 7f6b7e9b5640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b6406c560 0x7f6b6406ea20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009560 tx=0x7f6b6c008040 comp rx=0 tx=0).stop 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.870+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b78145090 msgr2=0x7f6b78140d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.871+0000 7f6b7e9b5640 1 --2- 172.21.15.66:0/2770768162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b78145090 0x7f6b78140d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800c6d0 tx=0x7f6b6800cb90 comp rx=0 tx=0).stop 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.871+0000 7f6b7cf52640 1 -- 172.21.15.66:0/2770768162 reap_dead start 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.871+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 shutdown_connections 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.871+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 >> 172.21.15.66:0/2770768162 conn(0x7f6b7806f8b0 msgr2=0x7f6b7806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.871+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 shutdown_connections 2024-03-30T13:10:07.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:07.871+0000 7f6b7e9b5640 1 -- 172.21.15.66:0/2770768162 wait complete. 2024-03-30T13:10:07.938 INFO:teuthology.orchestra.run.smithi066.stdout:64424509452 2024-03-30T13:10:07.938 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T13:10:07.939 INFO:teuthology.orchestra.run.smithi066.stdout:64424509449 2024-03-30T13:10:07.939 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T13:10:07.940 INFO:teuthology.orchestra.run.smithi066.stdout:64424509449 2024-03-30T13:10:07.940 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T13:10:07.941 INFO:teuthology.orchestra.run.smithi066.stdout:55834574859 2024-03-30T13:10:07.942 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T13:10:07.942 INFO:teuthology.orchestra.run.smithi066.stdout:60129542154 2024-03-30T13:10:07.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T13:10:07.943 INFO:teuthology.orchestra.run.smithi066.stdout:64424509452 2024-03-30T13:10:07.944 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T13:10:07.945 INFO:teuthology.orchestra.run.smithi066.stdout:64424509450 2024-03-30T13:10:07.945 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T13:10:08.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.162+0000 7fc76c1c8640 1 Processor -- start 2024-03-30T13:10:08.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.162+0000 7fc76c1c8640 1 -- start start 2024-03-30T13:10:08.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.162+0000 7faa6adcd640 1 Processor -- start 2024-03-30T13:10:08.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.162+0000 7faa6adcd640 1 -- start start 2024-03-30T13:10:08.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7fe86b0ac640 1 Processor -- start 2024-03-30T13:10:08.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.162+0000 7f0830aba640 1 Processor -- start 2024-03-30T13:10:08.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.162+0000 7f0830aba640 1 -- start start 2024-03-30T13:10:08.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7f9b81095640 1 Processor -- start 2024-03-30T13:10:08.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7fe86b0ac640 1 -- start start 2024-03-30T13:10:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7f9b81095640 1 -- start start 2024-03-30T13:10:08.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7f0830aba640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f082c06f2c0 0x7f082c15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7f0830aba640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082c15b990 con 0x7f082c151430 2024-03-30T13:10:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7f0830aba640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082c13e1e0 con 0x7f082c1405a0 2024-03-30T13:10:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7f0830aba640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f082c140a50 con 0x7f082c06f2c0 2024-03-30T13:10:08.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f3ba1bab640 1 Processor -- start 2024-03-30T13:10:08.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f3ba1bab640 1 -- start start 2024-03-30T13:10:08.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7fa069fd6640 1 Processor -- start 2024-03-30T13:10:08.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fe86b0ac640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe864160610 0x7fe8641639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fe86b0ac640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8640afb30 0x7fe86406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fe86b0ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe86406ecb0 0x7fe86406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fe86b0ac640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe86406dc70 con 0x7fe864160610 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fe86b0ac640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe864162f10 con 0x7fe8640afb30 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fe86b0ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe8641631f0 con 0x7fe86406ecb0 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a8ab640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe86406ecb0 0x7fe86406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a8ab640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe86406ecb0 0x7fe86406cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39238/0 (socket says 172.21.15.66:39238) 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a8ab640 1 -- 172.21.15.66:0/952965355 learned_addr learned my addr 172.21.15.66:0/952965355 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/952965355 >> v2:172.21.15.66:3300/0 conn(0x7fe864160610 0x7fe8641639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.163+0000 7faa6adcd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c038620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa6adcd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa5c04f530 con 0x7faa5c0e1080 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa6adcd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa5c04f810 con 0x7faa5c03a5c0 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa6adcd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c04faf0 con 0x7faa5c050dc0 2024-03-30T13:10:08.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa68b69640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7faa5c0e1080 0x7faa5c039eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53880/0 (socket says 172.21.15.66:53880) 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa68b69640 1 -- 172.21.15.66:0/2866127375 learned_addr learned my addr 172.21.15.66:0/2866127375 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa63fff640 1 --2- 172.21.15.66:0/2866127375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c038620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7faa637fe640 1 -- 172.21.15.66:0/2866127375 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760504079 0 0) 0x7faa5c04f530 con 0x7faa5c0e1080 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7faa637fe640 1 -- 172.21.15.66:0/2866127375 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa54003520 con 0x7faa5c0e1080 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fc76c1c8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc76415a480 0x7fc7640b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fc76a765640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc7640d0780 0x7fc764071c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53882/0 (socket says 172.21.15.66:53882) 2024-03-30T13:10:08.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fc76a765640 1 -- 172.21.15.66:0/2049183181 learned_addr learned my addr 172.21.15.66:0/2049183181 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:08.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fb733cc5640 1 Processor -- start 2024-03-30T13:10:08.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fb733cc5640 1 -- start start 2024-03-30T13:10:08.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f082affd640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f082c1405a0 0x7f082c13dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:48436/0 (socket says 172.21.15.66:48436) 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f082affd640 1 -- 172.21.15.66:0/289365636 learned_addr learned my addr 172.21.15.66:0/289365636 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722989615 0 0) 0x7f082c13e1e0 con 0x7f082c1405a0 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0814003520 con 0x7f082c1405a0 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4164848334 0 0) 0x7f0814003520 con 0x7f082c1405a0 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 >> v2:172.21.15.117:3300/0 conn(0x7f082c06f2c0 msgr2=0x7f082c15a9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 --2- 172.21.15.66:0/289365636 >> v2:172.21.15.117:3300/0 conn(0x7f082c06f2c0 0x7f082c15a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 >> v1:172.21.15.66:6789/0 conn(0x7f082c151430 legacy=0x7f082c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c15b3d0 con 0x7f082c1405a0 2024-03-30T13:10:08.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f08240019f0 con 0x7f082c1405a0 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 >> v1:172.21.15.117:6789/0 conn(0x7f082c1405a0 legacy=0x7f082c13dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814003a00 0x7f0814003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0814004800 0x7f0814004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08140052e0 0x7f081400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f082c140a50 con 0x7f08140052e0 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f081400c220 con 0x7f0814004800 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f081400c420 con 0x7f0814003a00 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082affd640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0814004800 0x7f0814004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7faa63fff640 1 -- 172.21.15.66:0/2866127375 >> v1:172.21.15.117:6789/0 conn(0x7faa5c03a5c0 legacy=0x7faa5c005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7faa63fff640 1 -- 172.21.15.66:0/2866127375 >> v1:172.21.15.66:6789/0 conn(0x7faa5c0e1080 legacy=0x7faa5c039eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7faa63fff640 1 -- 172.21.15.66:0/2866127375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c04fdd0 con 0x7faa5c050dc0 2024-03-30T13:10:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7fa069fd6640 1 -- start start 2024-03-30T13:10:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fa069fd6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa05c0ead00 0x7fa05c04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fa069fd6640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa05c04ffd0 0x7fa05c037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb733cc5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb72c15ba80 0x7fb72c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fc769f64640 1 --2- 172.21.15.66:0/2049183181 >> v2:172.21.15.117:3300/0 conn(0x7fc76415a480 0x7fc7640b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fa069fd6640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05c053720 con 0x7fa05c038110 2024-03-30T13:10:08.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb733cc5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c0d0520 0x7fb72c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb733cc5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72c0cc790 con 0x7fb72c154d30 2024-03-30T13:10:08.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb733cc5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb72c0cca50 con 0x7fb72c15ba80 2024-03-30T13:10:08.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb733cc5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0ccd30 con 0x7fb72c0d0520 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76415e090 con 0x7fc7640d0780 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76415e370 con 0x7fc7640b4090 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a0aa640 1 -- 172.21.15.66:0/952965355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe86406ecb0 msgr2=0x7fe86406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/952965355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe86406ecb0 0x7fe86406cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a0aa640 1 -- 172.21.15.66:0/952965355 >> v2:172.21.15.117:3300/0 conn(0x7fe8640afb30 msgr2=0x7fe86406c590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/952965355 >> v2:172.21.15.117:3300/0 conn(0x7fe8640afb30 0x7fe86406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7fe86a0aa640 1 -- 172.21.15.66:0/952965355 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8641634d0 con 0x7fe864160610 2024-03-30T13:10:08.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc76415a860 con 0x7fc76415a480 2024-03-30T13:10:08.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fa0697d5640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa05c038110 0x7fa05c038580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:48462/0 (socket says 172.21.15.66:48462) 2024-03-30T13:10:08.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fa0697d5640 1 -- 172.21.15.66:0/1707128333 learned_addr learned my addr 172.21.15.66:0/1707128333 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:08.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 680847143 0 0) 0x7fc76415e090 con 0x7fc7640d0780 2024-03-30T13:10:08.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fa063fff640 1 --2- 172.21.15.66:0/1707128333 >> v2:172.21.15.66:3300/0 conn(0x7fa05c04ffd0 0x7fa05c037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc750003520 con 0x7fc7640d0780 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.164+0000 7f9b81095640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74051060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b81095640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b74006a60 0x7f9b74006eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74051060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74051060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39232/0 (socket says 172.21.15.66:39232) 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 -- 172.21.15.66:0/4069761835 learned_addr learned my addr 172.21.15.66:0/4069761835 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b740090f0 con 0x7f9b740e3810 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9b740515a0 con 0x7f9b74006a60 2024-03-30T13:10:08.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9b740093d0 con 0x7f9b74050be0 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7b59e640 1 --2- 172.21.15.66:0/4069761835 >> v2:172.21.15.66:3300/0 conn(0x7f9b74006a60 0x7f9b74006eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 -- 172.21.15.66:0/4069761835 >> v2:172.21.15.66:3300/0 conn(0x7f9b74006a60 msgr2=0x7f9b74006eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 --2- 172.21.15.66:0/4069761835 >> v2:172.21.15.66:3300/0 conn(0x7f9b74006a60 0x7f9b74006eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 -- 172.21.15.66:0/4069761835 >> v1:172.21.15.66:6789/0 conn(0x7f9b740e3810 legacy=0x7f9b74008580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f9b7a59c640 1 -- 172.21.15.66:0/4069761835 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b74039400 con 0x7f9b74050be0 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.165+0000 7f082affd640 1 -- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814003a00 msgr2=0x7f0814003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f082affd640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814003a00 0x7f0814003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f082affd640 1 -- 172.21.15.66:0/289365636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08140052e0 msgr2=0x7f081400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:10:08.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f082affd640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08140052e0 0x7f081400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f082affd640 1 -- 172.21.15.66:0/289365636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0814003520 con 0x7f0814004800 2024-03-30T13:10:08.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f082bfff640 1 -- 172.21.15.66:0/289365636 reap_dead start 2024-03-30T13:10:08.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba1bab640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b9c0cf540 0x7f3b9c163440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba1bab640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b9c166420 0x7f3b9c163980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba1bab640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c1669e0 con 0x7f3b9c0ca890 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba1bab640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c166cc0 con 0x7f3b9c0cf540 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba1bab640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c166fa0 con 0x7f3b9c166420 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba13aa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b9c166420 0x7f3b9c163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba13aa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b9c166420 0x7f3b9c163980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:34992/0 (socket says 172.21.15.66:34992) 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba13aa640 1 -- 172.21.15.66:0/93259586 learned_addr learned my addr 172.21.15.66:0/93259586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3b93fff640 1 --2- 172.21.15.66:0/93259586 >> v2:172.21.15.66:3300/0 conn(0x7f3b9c0cf540 0x7f3b9c163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba13aa640 1 -- 172.21.15.66:0/93259586 >> v2:172.21.15.66:3300/0 conn(0x7f3b9c0cf540 msgr2=0x7f3b9c163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba13aa640 1 --2- 172.21.15.66:0/93259586 >> v2:172.21.15.66:3300/0 conn(0x7f3b9c0cf540 0x7f3b9c163440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7f3ba13aa640 1 -- 172.21.15.66:0/93259586 >> v1:172.21.15.66:6789/0 conn(0x7f3b9c0ca890 legacy=0x7f3b9c0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7f3ba13aa640 1 -- 172.21.15.66:0/93259586 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c167280 con 0x7f3b9c166420 2024-03-30T13:10:08.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa05c054330 con 0x7fa05c04ffd0 2024-03-30T13:10:08.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 798240352 0 0) 0x7fc750003520 con 0x7fc7640d0780 2024-03-30T13:10:08.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 >> v2:172.21.15.117:3300/0 conn(0x7fc76415a480 msgr2=0x7fc7640b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa05c053a00 con 0x7fa05c0ead00 2024-03-30T13:10:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa0637fe640 1 -- 172.21.15.66:0/1707128333 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2693949578 0 0) 0x7fa05c053720 con 0x7fa05c038110 2024-03-30T13:10:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa0637fe640 1 -- 172.21.15.66:0/1707128333 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa054003520 con 0x7fa05c038110 2024-03-30T13:10:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa068fd4640 1 --2- 172.21.15.66:0/1707128333 >> v2:172.21.15.117:3300/0 conn(0x7fa05c0ead00 0x7fa05c04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/952965355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe864160610 0x7fe8641639f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe86406d150 tx=0x7fe85402bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76fe7f48a0e97a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7faa63fff640 1 --2- 172.21.15.66:0/2866127375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c038620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa500097d0 tx=0x7faa5002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49fd0151dad0aa32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb732262640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c0d0520 0x7fb72c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb732262640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c0d0520 0x7fb72c0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39252/0 (socket says 172.21.15.66:39252) 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb732262640 1 -- 172.21.15.66:0/2029080173 learned_addr learned my addr 172.21.15.66:0/2029080173 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.166+0000 7fb731260640 1 --2- 172.21.15.66:0/2029080173 >> v2:172.21.15.66:3300/0 conn(0x7fb72c15ba80 0x7fb72c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fb731260640 1 -- 172.21.15.66:0/2029080173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c0d0520 msgr2=0x7fb72c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fb731260640 1 --2- 172.21.15.66:0/2029080173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c0d0520 0x7fb72c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fb731260640 1 -- 172.21.15.66:0/2029080173 >> v1:172.21.15.117:6789/0 conn(0x7fb72c154d30 legacy=0x7fb72c0cb230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:10:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fb731260640 1 -- 172.21.15.66:0/2029080173 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c0cd010 con 0x7fb72c15ba80 2024-03-30T13:10:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fe8690a8640 1 -- 172.21.15.66:0/952965355 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85402d660 con 0x7fe864160610 2024-03-30T13:10:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fe8690a8640 1 -- 172.21.15.66:0/952965355 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85402d800 con 0x7fe864160610 2024-03-30T13:10:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fe8690a8640 1 -- 172.21.15.66:0/952965355 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85402dbd0 con 0x7fe864160610 2024-03-30T13:10:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7faa637fe640 1 -- 172.21.15.66:0/2866127375 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50035020 con 0x7faa5c050dc0 2024-03-30T13:10:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7faa637fe640 1 -- 172.21.15.66:0/2866127375 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa50031800 con 0x7faa5c050dc0 2024-03-30T13:10:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7faa637fe640 1 -- 172.21.15.66:0/2866127375 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50035020 con 0x7faa5c050dc0 2024-03-30T13:10:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 --2- 172.21.15.66:0/2049183181 >> v2:172.21.15.117:3300/0 conn(0x7fc76415a480 0x7fc7640b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 >> v1:172.21.15.117:6789/0 conn(0x7fc7640b4090 legacy=0x7fc7640d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76415e650 con 0x7fc7640d0780 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7fc75c001a20 con 0x7fc7640d0780 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc769f64640 1 --2- 172.21.15.66:0/2049183181 >> v2:172.21.15.117:3300/0 conn(0x7fc76415a480 0x7fc7640b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 >> v1:172.21.15.66:6789/0 conn(0x7fc7640d0780 legacy=0x7fc764071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc750003a20 0x7fc750003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750004800 0x7fc750004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc750005230 0x7fc75000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc76415a860 con 0x7fc750004800 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc75000c190 con 0x7fc750003a20 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc75000c390 con 0x7fc750005230 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc769f64640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc750003a20 0x7fc750003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fc769763640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750004800 0x7fc750004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa0637fe640 1 -- 172.21.15.66:0/1707128333 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1331074953 0 0) 0x7fa054003520 con 0x7fa05c038110 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa063fff640 1 -- 172.21.15.66:0/1707128333 >> v2:172.21.15.117:3300/0 conn(0x7fa05c0ead00 msgr2=0x7fa05c04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa063fff640 1 --2- 172.21.15.66:0/1707128333 >> v2:172.21.15.117:3300/0 conn(0x7fa05c0ead00 0x7fa05c04fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa063fff640 1 -- 172.21.15.66:0/1707128333 >> v1:172.21.15.117:6789/0 conn(0x7fa05c038110 legacy=0x7fa05c038580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.167+0000 7fa063fff640 1 -- 172.21.15.66:0/1707128333 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c053ce0 con 0x7fa05c04ffd0 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f082affd640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0814004800 0x7f0814004c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0824002230 tx=0x7f0814005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922ac340224891f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc769f64640 1 -- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc750005230 msgr2=0x7fc75000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:10:08.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f9b7a59c640 1 --2- 172.21.15.66:0/4069761835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74051060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0098d0 tx=0x7f9b6c03c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea22ed5a0d916c51 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc769f64640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc750005230 0x7fc75000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc769f64640 1 -- 172.21.15.66:0/2049183181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750004800 msgr2=0x7fc750004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:08.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc769f64640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750004800 0x7fc750004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc769f64640 1 -- 172.21.15.66:0/2049183181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76415e650 con 0x7fc750003a20 2024-03-30T13:10:08.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc76a765640 1 -- 172.21.15.66:0/2049183181 reap_dead start 2024-03-30T13:10:08.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f3ba13aa640 1 --2- 172.21.15.66:0/93259586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c163980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c15d3f0 tx=0x7f3b9402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e39c9fcaf12f5eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fc769763640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750004800 0x7fc750004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:08.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f3b937fe640 1 -- 172.21.15.66:0/93259586 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9402ca80 con 0x7f3b9c166420 2024-03-30T13:10:08.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f3b937fe640 1 -- 172.21.15.66:0/93259586 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9402cc20 con 0x7f3b9c166420 2024-03-30T13:10:08.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f3b937fe640 1 -- 172.21.15.66:0/93259586 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b940343f0 con 0x7f3b9c166420 2024-03-30T13:10:08.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fe86b0ac640 1 -- 172.21.15.66:0/952965355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe864160610 msgr2=0x7fe8641639f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/952965355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe864160610 0x7fe8641639f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe86406d150 tx=0x7fe85402bab0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f9b79d9b640 1 -- 172.21.15.66:0/4069761835 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c045070 con 0x7f9b74050be0 2024-03-30T13:10:08.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f9b79d9b640 1 -- 172.21.15.66:0/4069761835 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c041800 con 0x7f9b74050be0 2024-03-30T13:10:08.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7f9b79d9b640 1 -- 172.21.15.66:0/4069761835 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c041c10 con 0x7f9b74050be0 2024-03-30T13:10:08.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.168+0000 7faa6adcd640 1 -- 172.21.15.66:0/2866127375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 msgr2=0x7faa5c038620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7faa6adcd640 1 --2- 172.21.15.66:0/2866127375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c038620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa500097d0 tx=0x7faa5002b8a0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fa063fff640 1 --2- 172.21.15.66:0/1707128333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c04ffd0 0x7fa05c037bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa05c007a60 tx=0x7fa05002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff333888fafe9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7faa6adcd640 1 -- 172.21.15.66:0/2866127375 shutdown_connections 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7faa6adcd640 1 --2- 172.21.15.66:0/2866127375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c038620 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7faa6adcd640 1 -- 172.21.15.66:0/2866127375 >> 172.21.15.66:0/2866127375 conn(0x7faa5c009fa0 msgr2=0x7faa5c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 -- 172.21.15.66:0/952965355 shutdown_connections 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/952965355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe86406ecb0 0x7fe86406cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/952965355 >> v2:172.21.15.117:3300/0 conn(0x7fe8640afb30 0x7fe86406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/952965355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe864160610 0x7fe8641639f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 -- 172.21.15.66:0/952965355 >> 172.21.15.66:0/952965355 conn(0x7fe864071b10 msgr2=0x7fe864071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fa0637fe640 1 -- 172.21.15.66:0/1707128333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050038070 con 0x7fa05c04ffd0 2024-03-30T13:10:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fa0637fe640 1 -- 172.21.15.66:0/1707128333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050033440 con 0x7fa05c04ffd0 2024-03-30T13:10:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fa0637fe640 1 -- 172.21.15.66:0/1707128333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064050140 con 0x7fa05c04ffd0 2024-03-30T13:10:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 msgr2=0x7f9b74051060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f9b81095640 1 --2- 172.21.15.66:0/4069761835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74051060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0098d0 tx=0x7f9b6c03c940 comp rx=0 tx=0).stop 2024-03-30T13:10:08.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fb731260640 1 --2- 172.21.15.66:0/2029080173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c15ba80 0x7fb72c0cffe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb72800dbc0 tx=0x7fb72802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0e58393d4d59c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 shutdown_connections 2024-03-30T13:10:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f9b81095640 1 --2- 172.21.15.66:0/4069761835 >> v2:172.21.15.66:3300/0 conn(0x7f9b74006a60 0x7f9b74006eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f9b81095640 1 --2- 172.21.15.66:0/4069761835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74051060 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 >> 172.21.15.66:0/4069761835 conn(0x7f9b74055bc0 msgr2=0x7f9b74055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7faa6adcd640 1 -- 172.21.15.66:0/2866127375 shutdown_connections 2024-03-30T13:10:08.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7faa6adcd640 1 -- 172.21.15.66:0/2866127375 wait complete. 2024-03-30T13:10:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082400a410 con 0x7f0814004800 2024-03-30T13:10:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824034630 con 0x7f0814004800 2024-03-30T13:10:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 -- 172.21.15.66:0/952965355 shutdown_connections 2024-03-30T13:10:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f3ba1bab640 1 -- 172.21.15.66:0/93259586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 msgr2=0x7f3b9c163980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/93259586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c163980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c15d3f0 tx=0x7f3b9402bee0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fe86b0ac640 1 -- 172.21.15.66:0/952965355 wait complete. 2024-03-30T13:10:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f082a7fc640 1 -- 172.21.15.66:0/289365636 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824034aa0 con 0x7f0814004800 2024-03-30T13:10:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f3ba1bab640 1 -- 172.21.15.66:0/93259586 shutdown_connections 2024-03-30T13:10:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/93259586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c163980 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/93259586 >> v2:172.21.15.66:3300/0 conn(0x7f3b9c0cf540 0x7f3b9c163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f3ba1bab640 1 -- 172.21.15.66:0/93259586 >> 172.21.15.66:0/93259586 conn(0x7f3b9c0cc7b0 msgr2=0x7f3b9c0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.169+0000 7fb730a5f640 1 -- 172.21.15.66:0/2029080173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72800b450 con 0x7fb72c15ba80 2024-03-30T13:10:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fb730a5f640 1 -- 172.21.15.66:0/2029080173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72800b5f0 con 0x7fb72c15ba80 2024-03-30T13:10:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fb730a5f640 1 -- 172.21.15.66:0/2029080173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72800b9c0 con 0x7fb72c15ba80 2024-03-30T13:10:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f3ba1bab640 1 -- 172.21.15.66:0/93259586 shutdown_connections 2024-03-30T13:10:08.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f3ba1bab640 1 -- 172.21.15.66:0/93259586 wait complete. 2024-03-30T13:10:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 -- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0814004800 msgr2=0x7f0814004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0814004800 0x7f0814004c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0824002230 tx=0x7f0814005830 comp rx=0 tx=0).stop 2024-03-30T13:10:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c04ffd0 msgr2=0x7fa05c037bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fc769f64640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc750003a20 0x7fc750003e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7600095c0 tx=0x7fc750005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e748e523a8572623 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fa069fd6640 1 --2- 172.21.15.66:0/1707128333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c04ffd0 0x7fa05c037bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa05c007a60 tx=0x7fa05002b7a0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 -- 172.21.15.66:0/289365636 shutdown_connections 2024-03-30T13:10:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 --2- 172.21.15.66:0/289365636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0814004800 0x7f0814004c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 -- 172.21.15.66:0/289365636 >> 172.21.15.66:0/289365636 conn(0x7f082c0cc7b0 msgr2=0x7f082c0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7600091c0 con 0x7fc750003a20 2024-03-30T13:10:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760004030 con 0x7fc750003a20 2024-03-30T13:10:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fc768f62640 1 -- 172.21.15.66:0/2049183181 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760004420 con 0x7fc750003a20 2024-03-30T13:10:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7fb733cc5640 1 -- 172.21.15.66:0/2029080173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c15ba80 msgr2=0x7fb72c0cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 shutdown_connections 2024-03-30T13:10:08.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fa069fd6640 1 --2- 172.21.15.66:0/1707128333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c04ffd0 0x7fa05c037bd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fa069fd6640 1 --2- 172.21.15.66:0/1707128333 >> v2:172.21.15.117:3300/0 conn(0x7fa05c0ead00 0x7fa05c04fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 -- 172.21.15.66:0/289365636 shutdown_connections 2024-03-30T13:10:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.170+0000 7f0830aba640 1 -- 172.21.15.66:0/289365636 wait complete. 2024-03-30T13:10:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 >> 172.21.15.66:0/1707128333 conn(0x7fa05c009fb0 msgr2=0x7fa05c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7faa6adcd640 1 Processor -- start 2024-03-30T13:10:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fb733cc5640 1 --2- 172.21.15.66:0/2029080173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c15ba80 0x7fb72c0cffe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb72800dbc0 tx=0x7fb72802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fb733cc5640 1 -- 172.21.15.66:0/2029080173 shutdown_connections 2024-03-30T13:10:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fb733cc5640 1 --2- 172.21.15.66:0/2029080173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c0d0520 0x7fb72c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 shutdown_connections 2024-03-30T13:10:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fa069fd6640 1 -- 172.21.15.66:0/1707128333 wait complete. 2024-03-30T13:10:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc750003a20 msgr2=0x7fc750003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc750003a20 0x7fc750003e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7600095c0 tx=0x7fc750005830 comp rx=0 tx=0).stop 2024-03-30T13:10:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fe86b0ac640 1 Processor -- start 2024-03-30T13:10:08.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 shutdown_connections 2024-03-30T13:10:08.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2049183181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc750003a20 0x7fc750003e00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 >> 172.21.15.66:0/2049183181 conn(0x7fc76406ede0 msgr2=0x7fc76406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7faa6adcd640 1 -- start start 2024-03-30T13:10:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 shutdown_connections 2024-03-30T13:10:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7f9b81095640 1 -- 172.21.15.66:0/4069761835 wait complete. 2024-03-30T13:10:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fb733cc5640 1 --2- 172.21.15.66:0/2029080173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c15ba80 0x7fb72c0cffe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fb733cc5640 1 -- 172.21.15.66:0/2029080173 >> 172.21.15.66:0/2029080173 conn(0x7fb72c06d610 msgr2=0x7fb72c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 shutdown_connections 2024-03-30T13:10:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.171+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2049183181 wait complete. 2024-03-30T13:10:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7fb733cc5640 1 -- 172.21.15.66:0/2029080173 shutdown_connections 2024-03-30T13:10:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7fb733cc5640 1 -- 172.21.15.66:0/2029080173 wait complete. 2024-03-30T13:10:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7f3ba1bab640 1 Processor -- start 2024-03-30T13:10:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7f3ba1bab640 1 -- start start 2024-03-30T13:10:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7faa6adcd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 0x7faa5c0d55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7faa6adcd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c0d2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7f0830aba640 1 Processor -- start 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7faa6adcd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 0x7faa5c0d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7faa68b69640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 0x7faa5c0d55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7faa68b69640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 0x7faa5c0d55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35018/0 (socket says 172.21.15.66:35018) 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c0d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c0d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39276/0 (socket says 172.21.15.66:39276) 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa6936a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 0x7faa5c0d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa6936a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 0x7faa5c0d2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34338/0 (socket says 172.21.15.66:34338) 2024-03-30T13:10:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.172+0000 7f0830aba640 1 -- start start 2024-03-30T13:10:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7fa069fd6640 1 Processor -- start 2024-03-30T13:10:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7fa069fd6640 1 -- start start 2024-03-30T13:10:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7fc76c1c8640 1 Processor -- start 2024-03-30T13:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa6adcd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c006690 con 0x7faa5c0e1080 2024-03-30T13:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7fc76c1c8640 1 -- start start 2024-03-30T13:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa6adcd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c050390 con 0x7faa5c03a5c0 2024-03-30T13:10:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa6adcd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c006810 con 0x7faa5c050dc0 2024-03-30T13:10:08.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa68b69640 1 -- 172.21.15.66:0/1275685281 learned_addr learned my addr 172.21.15.66:0/1275685281 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7f9b81095640 1 Processor -- start 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 -- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 msgr2=0x7faa5c0d55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 0x7faa5c0d55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 -- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 msgr2=0x7faa5c0d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 0x7faa5c0d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa63fff640 1 -- 172.21.15.66:0/1275685281 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c04e660 con 0x7faa5c050dc0 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.173+0000 7faa6936a640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 0x7faa5c0d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7faa68b69640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 0x7faa5c0d55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-30T13:10:08.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7faa63fff640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c0d2640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00b320 tx=0x7faa500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f9b81095640 1 -- start start 2024-03-30T13:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa069fd6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 0x7fa05c0db980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa069fd6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa05c04ffd0 0x7fa05c0d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc76c1c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750003a20 0x7fc764149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f0830aba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814004800 0x7f082c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f0830aba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 0x7f082c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f0830aba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c072e20 0x7f082c071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f0830aba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f082c1518a0 con 0x7f082c159480 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f0830aba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f082c0cdd40 con 0x7f082c072e20 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f0830aba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f082c15a9c0 con 0x7f0814004800 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 0x7f082c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 0x7f082c0710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34350/0 (socket says 172.21.15.66:34350) 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 -- 172.21.15.66:0/2288997431 learned_addr learned my addr 172.21.15.66:0/2288997431 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082bfff640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c072e20 0x7f082c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 -- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814004800 msgr2=0x7f082c0b2af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814004800 0x7f082c0b2af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 -- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c072e20 msgr2=0x7f082c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c072e20 0x7f082c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7f082affd640 1 -- 172.21.15.66:0/2288997431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c073300 con 0x7f082c159480 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa068fd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 0x7fa05c0db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa068fd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 0x7fa05c0db980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35032/0 (socket says 172.21.15.66:35032) 2024-03-30T13:10:08.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fb733cc5640 1 Processor -- start 2024-03-30T13:10:08.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b81095640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b74006a60 0x7f9b74175030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc76c1c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7640b0660 0x7fc76415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f3ba1bab640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 0x7f3b9c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc76c1c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7640b0ae0 0x7fc76415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fb733cc5640 1 -- start start 2024-03-30T13:10:08.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa063fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa05c04ffd0 0x7fa05c0d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b81095640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74177e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc76c1c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc76406f780 con 0x7fc750003a20 2024-03-30T13:10:08.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc76c1c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc76406f600 con 0x7fc7640b0ae0 2024-03-30T13:10:08.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa063fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa05c04ffd0 0x7fa05c0d85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39284/0 (socket says 172.21.15.66:39284) 2024-03-30T13:10:08.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc76c1c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc7640b3b50 con 0x7fc7640b0660 2024-03-30T13:10:08.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa068fd4640 1 -- 172.21.15.66:0/3759503290 learned_addr learned my addr 172.21.15.66:0/3759503290 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.174+0000 7fe86b0ac640 1 -- start start 2024-03-30T13:10:08.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86b0ac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 0x7fe86415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86b0ac640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8640afb30 0x7fe864200bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86b0ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe864160610 0x7fe8640b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86b0ac640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe86406da10 con 0x7fe86406ecb0 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c0ead00 0x7fa05c043680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b81095640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b740e3810 0x7f9b740d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b7ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b74006a60 0x7f9b74175030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b7ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b74006a60 0x7f9b74175030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34364/0 (socket says 172.21.15.66:34364) 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b7a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74177e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b7a59c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74177e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39316/0 (socket says 172.21.15.66:39316) 2024-03-30T13:10:08.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86b0ac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe86406d890 con 0x7fe864160610 2024-03-30T13:10:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86b0ac640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe8640b3640 con 0x7fe8640afb30 2024-03-30T13:10:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86a0aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 0x7fe86415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86a0aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 0x7fe86415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34376/0 (socket says 172.21.15.66:34376) 2024-03-30T13:10:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fe86a0aa640 1 -- 172.21.15.66:0/1656567576 learned_addr learned my addr 172.21.15.66:0/1656567576 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50048070 con 0x7faa5c050dc0 2024-03-30T13:10:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b7ad9d640 1 -- 172.21.15.66:0/3494732235 learned_addr learned my addr 172.21.15.66:0/3494732235 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc769f64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750003a20 0x7fc764149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc769f64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750003a20 0x7fc764149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34358/0 (socket says 172.21.15.66:34358) 2024-03-30T13:10:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc769f64640 1 -- 172.21.15.66:0/2539434114 learned_addr learned my addr 172.21.15.66:0/2539434114 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5003bd70 con 0x7faa5c050dc0 2024-03-30T13:10:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fc769763640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7640b0660 0x7fc76415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c0547c0 con 0x7fa05c0ead00 2024-03-30T13:10:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5c04e940 con 0x7faa5c050dc0 2024-03-30T13:10:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74039040 con 0x7f9b74006a60 2024-03-30T13:10:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c038c90 con 0x7fa05c038110 2024-03-30T13:10:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74050960 con 0x7f9b740e3810 2024-03-30T13:10:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9b740391c0 con 0x7f9b74050be0 2024-03-30T13:10:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.175+0000 7f9b7b59e640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b740e3810 0x7f9b740d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa05c054940 con 0x7fa05c04ffd0 2024-03-30T13:10:08.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe8698a9640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8640afb30 0x7fe864200bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5c049840 con 0x7faa5c050dc0 2024-03-30T13:10:08.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a8ab640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe864160610 0x7fe8640b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50035020 con 0x7faa5c050dc0 2024-03-30T13:10:08.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a0aa640 1 -- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8640afb30 msgr2=0x7fe864200bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8640afb30 0x7fe864200bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a0aa640 1 -- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe864160610 msgr2=0x7fe8640b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe864160610 0x7fe8640b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a0aa640 1 -- 172.21.15.66:0/1656567576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8640cd160 con 0x7fe86406ecb0 2024-03-30T13:10:08.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f9b7a59c640 1 -- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b740e3810 msgr2=0x7f9b740d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f9b7a59c640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b740e3810 0x7f9b740d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f9b7a59c640 1 -- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b74006a60 msgr2=0x7f9b74175030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f9b7a59c640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b74006a60 0x7f9b74175030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f9b7a59c640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7404e010 con 0x7f9b74050be0 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f082bfff640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c072e20 0x7f082c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba1bab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b9c0cf540 0x7f3b9c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba1bab640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba1bab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0cac70 con 0x7f3b9c0cf540 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba1bab640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0cc4d0 con 0x7f3b9c166420 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba1bab640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c0718e0 con 0x7f3b9c0ca890 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba0ba9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 0x7f3b9c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba0ba9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 0x7f3b9c15c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39314/0 (socket says 172.21.15.66:39314) 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f3ba0ba9640 1 -- 172.21.15.66:0/2085748206 learned_addr learned my addr 172.21.15.66:0/2085748206 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7f9b7a59c640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74177e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009550 tx=0x7f9b6c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fe86a0aa640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 0x7fe86415aa70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe854009ac0 tx=0x7fe8540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.176+0000 7fa068fd4640 1 -- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa05c04ffd0 msgr2=0x7fa05c0d85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fa068fd4640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa05c04ffd0 0x7fa05c0d85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fa068fd4640 1 -- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c0ead00 msgr2=0x7fa05c043680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fa068fd4640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c0ead00 0x7fa05c043680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f3ba13aa640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fa068fd4640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c0dbec0 con 0x7fa05c038110 2024-03-30T13:10:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f3ba0ba9640 1 -- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 msgr2=0x7f3b9c0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb733cc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c0d0520 0x7fb72c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb733cc5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c154d30 0x7fb72c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f3ba0ba9640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c0afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fa068fd4640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 0x7fa05c0db980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d620 tx=0x7fa05800dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f3ba0ba9640 1 -- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b9c0cf540 msgr2=0x7f3b9c0b1530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:10:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb733cc5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c15ba80 0x7fb72c0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f3ba0ba9640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b9c0cf540 0x7f3b9c0b1530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fc76a765640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7640b0ae0 0x7fc76415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f3ba0ba9640 1 -- 172.21.15.66:0/2085748206 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c0b0490 con 0x7f3b9c0ca890 2024-03-30T13:10:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb733cc5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c155110 con 0x7fb72c0d0520 2024-03-30T13:10:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c059070 con 0x7f9b74050be0 2024-03-30T13:10:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c041e30 con 0x7f9b74050be0 2024-03-30T13:10:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb732262640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c15ba80 0x7fb72c0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb732262640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c15ba80 0x7fb72c0b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39322/0 (socket says 172.21.15.66:39322) 2024-03-30T13:10:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb732262640 1 -- 172.21.15.66:0/3366309941 learned_addr learned my addr 172.21.15.66:0/3366309941 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7f082affd640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 0x7f082c0710d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f082400a8b0 tx=0x7f0824001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fc76a765640 1 -- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7640b0660 msgr2=0x7fc76415ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fc76a765640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7640b0660 0x7fc76415ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fc76a765640 1 -- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750003a20 msgr2=0x7fc764149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fc76a765640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750003a20 0x7fc764149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fc76a765640 1 -- 172.21.15.66:0/2539434114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc764150d50 con 0x7fc7640b0ae0 2024-03-30T13:10:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c06f530 con 0x7fb72c154d30 2024-03-30T13:10:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.177+0000 7fb731a61640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c0d0520 0x7fb72c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7f3ba0ba9640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 0x7f3b9c15c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00b3f0 tx=0x7f3b8c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c045040 con 0x7f9b74050be0 2024-03-30T13:10:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b741783b0 con 0x7f9b74050be0 2024-03-30T13:10:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa44005290 con 0x7faa5c050dc0 2024-03-30T13:10:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0b0970 con 0x7fb72c15ba80 2024-03-30T13:10:08.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7404a370 con 0x7f9b74050be0 2024-03-30T13:10:08.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c154d30 0x7fb72c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fc76a765640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7640b0ae0 0x7fc76415bf70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc76406e750 tx=0x7fc75c00cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faa5004b070 con 0x7faa5c050dc0 2024-03-30T13:10:08.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 -- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c15ba80 msgr2=0x7fb72c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c15ba80 0x7fb72c0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7faa617fa640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faa4806c600 0x7faa4806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faa500ad5a0 con 0x7faa5c050dc0 2024-03-30T13:10:08.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 -- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c0d0520 msgr2=0x7fb72c14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c0d0520 0x7fb72c14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c0b0c50 con 0x7fb72c154d30 2024-03-30T13:10:08.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731a61640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c0d0520 0x7fb72c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:08.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7faa68b69640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faa4806c600 0x7faa4806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.178+0000 7fb731260640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c154d30 0x7fb72c0b1e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728032000 tx=0x7fb7280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7404ac60 con 0x7f9b74050be0 2024-03-30T13:10:08.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7faa68b69640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faa4806c600 0x7faa4806eac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa6404fbc0 tx=0x7faa6406f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05800b620 con 0x7fa05c038110 2024-03-30T13:10:08.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728041070 con 0x7fb72c154d30 2024-03-30T13:10:08.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c004d90 con 0x7fc7640b0ae0 2024-03-30T13:10:08.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c0d9330 con 0x7fa05c038110 2024-03-30T13:10:08.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa058019070 con 0x7fa05c038110 2024-03-30T13:10:08.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb728046460 con 0x7fb72c154d30 2024-03-30T13:10:08.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76414a500 con 0x7fc7640b0ae0 2024-03-30T13:10:08.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7641f0b00 con 0x7fc7640b0ae0 2024-03-30T13:10:08.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c00a5b0 con 0x7fc7640b0ae0 2024-03-30T13:10:08.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00a9a0 con 0x7fc7640b0ae0 2024-03-30T13:10:08.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.179+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c0d98f0 con 0x7fa05c038110 2024-03-30T13:10:08.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.180+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72800b450 con 0x7fb72c154d30 2024-03-30T13:10:08.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.180+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05800ba00 con 0x7fa05c038110 2024-03-30T13:10:08.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.180+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c0b0f30 con 0x7fb72c154d30 2024-03-30T13:10:08.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.180+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c015020 con 0x7f3b9c0ca890 2024-03-30T13:10:08.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.180+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c0b0770 con 0x7f3b9c0ca890 2024-03-30T13:10:08.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.180+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c0b0d30 con 0x7f3b9c0ca890 2024-03-30T13:10:08.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c0b14f0 con 0x7fb72c154d30 2024-03-30T13:10:08.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c005390 con 0x7fa05c038110 2024-03-30T13:10:08.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c005290 con 0x7fc7640b0ae0 2024-03-30T13:10:08.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854003810 con 0x7fe86406ecb0 2024-03-30T13:10:08.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824039040 con 0x7f082c159480 2024-03-30T13:10:08.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7f0830aba640 1 -- 172.21.15.66:0/2288997431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c0b35c0 con 0x7f082c159480 2024-03-30T13:10:08.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7f0830aba640 1 -- 172.21.15.66:0/2288997431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c0ce960 con 0x7f082c159480 2024-03-30T13:10:08.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c075890 con 0x7f3b9c0ca890 2024-03-30T13:10:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b6c05c070 con 0x7f9b74050be0 2024-03-30T13:10:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8640cd440 con 0x7fe86406ecb0 2024-03-30T13:10:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.181+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8640b1fe0 con 0x7fe86406ecb0 2024-03-30T13:10:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe854045070 con 0x7fe86406ecb0 2024-03-30T13:10:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854003bb0 con 0x7fe86406ecb0 2024-03-30T13:10:08.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7f9b637fe640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b6806c580 0x7f9b6806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c075900 con 0x7fb72c154d30 2024-03-30T13:10:08.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc75c011070 con 0x7fc7640b0ae0 2024-03-30T13:10:08.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7f9b7ad9d640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b6806c580 0x7f9b6806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.182+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c075890 con 0x7f082c159480 2024-03-30T13:10:08.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7fc75a7fc640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc74806c530 0x7fc74806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb72803a040 con 0x7fb72c154d30 2024-03-30T13:10:08.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc75c08a850 con 0x7fc7640b0ae0 2024-03-30T13:10:08.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7fc769f64640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc74806c530 0x7fc74806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7f9b7ad9d640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b6806c580 0x7f9b6806ea40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c064bb0 tx=0x7f9b7c06b4b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7fb7227fc640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb71006c530 0x7fb71006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8640b5b40 con 0x7fe86406ecb0 2024-03-30T13:10:08.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fc769f64640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc74806c530 0x7fc74806e9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc76003a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb7280ade90 con 0x7fb72c154d30 2024-03-30T13:10:08.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa05800bba0 con 0x7fa05c038110 2024-03-30T13:10:08.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fa0617fa640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa04806c730 0x7fa04806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824043070 con 0x7f082c159480 2024-03-30T13:10:08.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.183+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824034a20 con 0x7f082c159480 2024-03-30T13:10:08.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa05800b7c0 con 0x7fa05c038110 2024-03-30T13:10:08.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fa063fff640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa04806c730 0x7fa04806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fb731a61640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb71006c530 0x7fb71006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.184+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe85403f070 con 0x7fe86406ecb0 2024-03-30T13:10:08.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7fa063fff640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa04806c730 0x7fa04806ebf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa050035000 tx=0x7fa05002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7fb731a61640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb71006c530 0x7fb71006e9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb72c150810 tx=0x7fb71c009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7fe85affd640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe84806c530 0x7fe84806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9b6c08ab10 con 0x7f9b74050be0 2024-03-30T13:10:08.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faa50040070 con 0x7faa5c050dc0 2024-03-30T13:10:08.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe8540ad880 con 0x7fe86406ecb0 2024-03-30T13:10:08.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0824034bc0 con 0x7f082c159480 2024-03-30T13:10:08.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7fe8698a9640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe84806c530 0x7fe84806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c004020 con 0x7f3b9c0ca890 2024-03-30T13:10:08.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.185+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c009200 con 0x7f3b9c0ca890 2024-03-30T13:10:08.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.186+0000 7f0813fff640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f081806c530 0x7f081806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.186+0000 7fe8698a9640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe84806c530 0x7fe84806e9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe8640b44d0 tx=0x7fe8600090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.186+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f08240aea00 con 0x7f082c159480 2024-03-30T13:10:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.186+0000 7f082b7fe640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f081806c530 0x7f081806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.186+0000 7f082b7fe640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f081806c530 0x7f081806e9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f081c0095f0 tx=0x7f081c006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.186+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3b8c018020 con 0x7f3b9c0ca890 2024-03-30T13:10:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.187+0000 7f3b917fa640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b8806c530 0x7f3b8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.187+0000 7f3b93fff640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b8806c530 0x7f3b8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.187+0000 7f3b93fff640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b8806c530 0x7f3b8806e9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b5c80 tx=0x7f3b98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.190+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f082407cde0 con 0x7f082c159480 2024-03-30T13:10:08.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.190+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3b8c024de0 con 0x7f3b9c0ca890 2024-03-30T13:10:08.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.190+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b6c063020 con 0x7f9b74050be0 2024-03-30T13:10:08.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.191+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa05c0dbec0 con 0x7fa05c038110 2024-03-30T13:10:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.194+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3b8c01f030 con 0x7f3b9c0ca890 2024-03-30T13:10:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.199+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe85403d040 con 0x7fe86406ecb0 2024-03-30T13:10:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.199+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc75c01a060 con 0x7fc7640b0ae0 2024-03-30T13:10:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.208+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb72807bba0 con 0x7fb72c154d30 2024-03-30T13:10:08.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.606+0000 7f9b81095640 1 -- 172.21.15.66:0/3494732235 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9b740057d0 con 0x7f9b74050be0 2024-03-30T13:10:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.607+0000 7f9b637fe640 1 -- 172.21.15.66:0/3494732235 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b741783b0 con 0x7f9b74050be0 2024-03-30T13:10:08.614 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-03-30T13:10:08.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.610+0000 7f9b617fa640 1 -- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b6806c580 msgr2=0x7f9b6806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.610+0000 7f9b617fa640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b6806c580 0x7f9b6806ea40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c064bb0 tx=0x7f9b7c06b4b0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.610+0000 7f9b617fa640 1 -- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 msgr2=0x7f9b74177e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.610+0000 7f9b617fa640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74177e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009550 tx=0x7f9b6c004730 comp rx=0 tx=0).stop 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.610+0000 7f9b617fa640 1 -- 172.21.15.66:0/3494732235 shutdown_connections 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.610+0000 7f9b617fa640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b6806c580 0x7f9b6806ea40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.611+0000 7f9b617fa640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b740e3810 0x7f9b740d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.611+0000 7f9b617fa640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b74050be0 0x7f9b74177e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.611+0000 7f9b617fa640 1 --2- 172.21.15.66:0/3494732235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b74006a60 0x7f9b74175030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.611+0000 7f9b617fa640 1 -- 172.21.15.66:0/3494732235 >> 172.21.15.66:0/3494732235 conn(0x7f9b74055bc0 msgr2=0x7f9b740dc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.611+0000 7f9b617fa640 1 -- 172.21.15.66:0/3494732235 shutdown_connections 2024-03-30T13:10:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.611+0000 7f9b617fa640 1 -- 172.21.15.66:0/3494732235 wait complete. 2024-03-30T13:10:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.618+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faa44005580 con 0x7faa5c050dc0 2024-03-30T13:10:08.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.618+0000 7faa617fa640 1 -- 172.21.15.66:0/1275685281 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faa5007b2a0 con 0x7faa5c050dc0 2024-03-30T13:10:08.622 INFO:teuthology.orchestra.run.smithi066.stdout:55834574857 2024-03-30T13:10:08.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.621+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faa4806c600 msgr2=0x7faa4806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.621+0000 7faa6adcd640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faa4806c600 0x7faa4806eac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa6404fbc0 tx=0x7faa6406f040 comp rx=0 tx=0).stop 2024-03-30T13:10:08.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.622+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 msgr2=0x7faa5c0d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.622+0000 7faa6adcd640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c0d2640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00b320 tx=0x7faa500046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.622+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 shutdown_connections 2024-03-30T13:10:08.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.622+0000 7faa6adcd640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faa4806c600 0x7faa4806eac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.622+0000 7faa6adcd640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa5c0e1080 0x7faa5c0d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.622+0000 7faa6adcd640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa5c050dc0 0x7faa5c0d2640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.623+0000 7faa6adcd640 1 --2- 172.21.15.66:0/1275685281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa5c03a5c0 0x7faa5c0d55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.623+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 >> 172.21.15.66:0/1275685281 conn(0x7faa5c009fa0 msgr2=0x7faa5c006bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.623+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 shutdown_connections 2024-03-30T13:10:08.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.623+0000 7faa6adcd640 1 -- 172.21.15.66:0/1275685281 wait complete. 2024-03-30T13:10:08.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.625+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3b9c0cf320 con 0x7f3b9c0ca890 2024-03-30T13:10:08.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.626+0000 7f3b917fa640 1 -- 172.21.15.66:0/2085748206 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b8c059d60 con 0x7f3b9c0ca890 2024-03-30T13:10:08.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.626+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa05c0d9ff0 con 0x7fa05c038110 2024-03-30T13:10:08.630 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-03-30T13:10:08.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.627+0000 7fa0617fa640 1 -- 172.21.15.66:0/3759503290 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa05805a9b0 con 0x7fa05c038110 2024-03-30T13:10:08.630 INFO:teuthology.orchestra.run.smithi066.stdout:64424509451 2024-03-30T13:10:08.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.628+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f082c159ff0 con 0x7f082c159480 2024-03-30T13:10:08.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.629+0000 7f0813fff640 1 -- 172.21.15.66:0/2288997431 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f082407c6e0 con 0x7f082c159480 2024-03-30T13:10:08.632 INFO:teuthology.orchestra.run.smithi066.stdout:64424509449 2024-03-30T13:10:08.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b8806c530 msgr2=0x7f3b8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b8806c530 0x7f3b8806e9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b5c80 tx=0x7f3b98008040 comp rx=0 tx=0).stop 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 msgr2=0x7f3b9c15c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 0x7f3b9c15c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00b3f0 tx=0x7f3b8c00b8b0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 shutdown_connections 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b8806c530 0x7f3b8806e9f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b9c166420 0x7f3b9c0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b9c0cf540 0x7f3b9c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 --2- 172.21.15.66:0/2085748206 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9c0ca890 0x7f3b9c15c2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 >> 172.21.15.66:0/2085748206 conn(0x7f3b9c0cc7b0 msgr2=0x7f3b9c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa04806c730 msgr2=0x7fa04806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 shutdown_connections 2024-03-30T13:10:08.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7f3ba1bab640 1 -- 172.21.15.66:0/2085748206 wait complete. 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa04806c730 0x7fa04806ebf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa050035000 tx=0x7fa05002f000 comp rx=0 tx=0).stop 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 msgr2=0x7fa05c0db980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.630+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 0x7fa05c0db980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d620 tx=0x7fa05800dae0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 shutdown_connections 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa04806c730 0x7fa04806ebf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa05c0ead00 0x7fa05c043680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa05c04ffd0 0x7fa05c0d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 --2- 172.21.15.66:0/3759503290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa05c038110 0x7fa05c0db980 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d620 tx=0x7fa05800dae0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 >> 172.21.15.66:0/3759503290 conn(0x7fa05c009fb0 msgr2=0x7fa05c0497a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 shutdown_connections 2024-03-30T13:10:08.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.631+0000 7fa069fd6640 1 -- 172.21.15.66:0/3759503290 wait complete. 2024-03-30T13:10:08.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f081806c530 msgr2=0x7f081806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f081806c530 0x7f081806e9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f081c0095f0 tx=0x7f081c006bd0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 msgr2=0x7f082c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 0x7f082c0710d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f082400a8b0 tx=0x7f0824001cf0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc74c005580 con 0x7fc7640b0ae0 2024-03-30T13:10:08.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 shutdown_connections 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f081806c530 0x7f081806e9f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c072e20 0x7f082c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f082c159480 0x7f082c0710d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 --2- 172.21.15.66:0/2288997431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0814004800 0x7f082c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 >> 172.21.15.66:0/2288997431 conn(0x7f082c0cc7b0 msgr2=0x7f082c158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe86415bca0 con 0x7fe86406ecb0 2024-03-30T13:10:08.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.633+0000 7fc75a7fc640 1 -- 172.21.15.66:0/2539434114 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc764150d50 con 0x7fc7640b0ae0 2024-03-30T13:10:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.632+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 shutdown_connections 2024-03-30T13:10:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.633+0000 7f0811ffb640 1 -- 172.21.15.66:0/2288997431 wait complete. 2024-03-30T13:10:08.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.633+0000 7fe85affd640 1 -- 172.21.15.66:0/1656567576 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe854033030 con 0x7fe86406ecb0 2024-03-30T13:10:08.642 INFO:teuthology.orchestra.run.smithi066.stdout:60129542153 2024-03-30T13:10:08.642 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-03-30T13:10:08.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe84806c530 msgr2=0x7fe84806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc74806c530 msgr2=0x7fc74806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc74806c530 0x7fc74806e9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc76003a040 comp rx=0 tx=0).stop 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7640b0ae0 msgr2=0x7fc76415bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7640b0ae0 0x7fc76415bf70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc76406e750 tx=0x7fc75c00cc10 comp rx=0 tx=0).stop 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 shutdown_connections 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc74806c530 0x7fc74806e9f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7640b0ae0 0x7fc76415bf70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7640b0660 0x7fc76415ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 --2- 172.21.15.66:0/2539434114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc750003a20 0x7fc764149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 >> 172.21.15.66:0/2539434114 conn(0x7fc76406ede0 msgr2=0x7fc764153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe84806c530 0x7fe84806e9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe8640b44d0 tx=0x7fe8600090e0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 msgr2=0x7fe86415aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 0x7fe86415aa70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe854009ac0 tx=0x7fe8540046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 shutdown_connections 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe84806c530 0x7fe84806e9f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe864160610 0x7fe8640b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 shutdown_connections 2024-03-30T13:10:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fc76c1c8640 1 -- 172.21.15.66:0/2539434114 wait complete. 2024-03-30T13:10:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8640afb30 0x7fe864200bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 --2- 172.21.15.66:0/1656567576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe86406ecb0 0x7fe86415aa70 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe854009ac0 tx=0x7fe8540046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.636+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 >> 172.21.15.66:0/1656567576 conn(0x7fe864071b10 msgr2=0x7fe8640b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 shutdown_connections 2024-03-30T13:10:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.637+0000 7fe86b0ac640 1 -- 172.21.15.66:0/1656567576 wait complete. 2024-03-30T13:10:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.639+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb72c0cdcf0 con 0x7fb72c154d30 2024-03-30T13:10:08.646 INFO:teuthology.orchestra.run.smithi066.stdout:64424509451 2024-03-30T13:10:08.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.639+0000 7fb7227fc640 1 -- 172.21.15.66:0/3366309941 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb72807fea0 con 0x7fb72c154d30 2024-03-30T13:10:08.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.5 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb71006c530 msgr2=0x7fb71006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb71006c530 0x7fb71006e9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb72c150810 tx=0x7fb71c009060 comp rx=0 tx=0).stop 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c154d30 msgr2=0x7fb72c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c154d30 0x7fb72c0b1e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb728032000 tx=0x7fb7280046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 shutdown_connections 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb71006c530 0x7fb71006e9f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c15ba80 0x7fb72c0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c154d30 0x7fb72c0b1e80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 --2- 172.21.15.66:0/3366309941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb72c0d0520 0x7fb72c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 >> 172.21.15.66:0/3366309941 conn(0x7fb72c06d610 msgr2=0x7fb72c0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 shutdown_connections 2024-03-30T13:10:08.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:08.643+0000 7fb733cc5640 1 -- 172.21.15.66:0/3366309941 wait complete. 2024-03-30T13:10:08.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.3 2024-03-30T13:10:08.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-03-30T13:10:08.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.1 2024-03-30T13:10:08.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.4 2024-03-30T13:10:08.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.2 2024-03-30T13:10:08.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.0 2024-03-30T13:10:09.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T13:10:09.649 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T13:10:09.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T13:10:09.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T13:10:09.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T13:10:09.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T13:10:09.663 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T13:10:09.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.819+0000 7f403fae0640 1 Processor -- start 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.819+0000 7f403fae0640 1 -- start start 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.820+0000 7f403fae0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f403815f000 0x7f40380cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.820+0000 7f403fae0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40380ccd30 0x7f40380ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.820+0000 7f403fae0640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40380cb850 con 0x7f403815a720 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.821+0000 7f403fae0640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f40380cd1e0 con 0x7f403815f000 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.821+0000 7f403fae0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f40380cb570 con 0x7f40380ccd30 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.821+0000 7f403d87c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f403815a720 0x7f40380caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53924/0 (socket says 172.21.15.66:53924) 2024-03-30T13:10:09.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.821+0000 7f403d87c640 1 -- 172.21.15.66:0/2802410945 learned_addr learned my addr 172.21.15.66:0/2802410945 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:09.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.822+0000 7f403d07b640 1 --2- 172.21.15.66:0/2802410945 >> v2:172.21.15.66:3300/0 conn(0x7f403815f000 0x7f40380cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.822+0000 7f403d07b640 1 -- 172.21.15.66:0/2802410945 >> v2:172.21.15.117:3300/0 conn(0x7f40380ccd30 msgr2=0x7f40380ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.822+0000 7f403d07b640 1 --2- 172.21.15.66:0/2802410945 >> v2:172.21.15.117:3300/0 conn(0x7f40380ccd30 0x7f40380ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.822+0000 7f403d07b640 1 -- 172.21.15.66:0/2802410945 >> v1:172.21.15.66:6789/0 conn(0x7f403815a720 legacy=0x7f40380caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.822+0000 7f403d07b640 1 -- 172.21.15.66:0/2802410945 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403815e520 con 0x7f403815f000 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.824+0000 7f403d07b640 1 --2- 172.21.15.66:0/2802410945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f403815f000 0x7f40380cc7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4038070f20 tx=0x7f403402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a4759d3dbc6197 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.824+0000 7f403c87a640 1 -- 172.21.15.66:0/2802410945 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034035020 con 0x7f403815f000 2024-03-30T13:10:09.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.824+0000 7f403c87a640 1 -- 172.21.15.66:0/2802410945 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4034031870 con 0x7f403815f000 2024-03-30T13:10:09.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 -- 172.21.15.66:0/2802410945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f403815f000 msgr2=0x7f40380cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 --2- 172.21.15.66:0/2802410945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f403815f000 0x7f40380cc7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4038070f20 tx=0x7f403402b910 comp rx=0 tx=0).stop 2024-03-30T13:10:09.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 -- 172.21.15.66:0/2802410945 shutdown_connections 2024-03-30T13:10:09.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 --2- 172.21.15.66:0/2802410945 >> v2:172.21.15.117:3300/0 conn(0x7f40380ccd30 0x7f40380ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 --2- 172.21.15.66:0/2802410945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f403815f000 0x7f40380cc7f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 -- 172.21.15.66:0/2802410945 >> 172.21.15.66:0/2802410945 conn(0x7f40380b1c20 msgr2=0x7f40380b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 -- 172.21.15.66:0/2802410945 shutdown_connections 2024-03-30T13:10:09.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.826+0000 7f403fae0640 1 -- 172.21.15.66:0/2802410945 wait complete. 2024-03-30T13:10:09.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.829+0000 7f403fae0640 1 Processor -- start 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.829+0000 7f403fae0640 1 -- start start 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403fae0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 0x7f4038158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403fae0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f403815a720 0x7f40380c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403fae0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40380c68b0 0x7f40380b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403fae0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f403815bc60 con 0x7f40380c68b0 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403fae0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f403815eae0 con 0x7f40380ccd30 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403fae0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f40380cb1e0 con 0x7f403815a720 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403d87c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 0x7f4038158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403d87c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 0x7f4038158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35096/0 (socket says 172.21.15.66:35096) 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.830+0000 7f403d87c640 1 -- 172.21.15.66:0/3702639252 learned_addr learned my addr 172.21.15.66:0/3702639252 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.831+0000 7f403d87c640 1 -- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f403815a720 msgr2=0x7f40380c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.831+0000 7f403d87c640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f403815a720 0x7f40380c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.831+0000 7f403d87c640 1 -- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40380c68b0 msgr2=0x7f40380b2ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:10:09.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.831+0000 7f403e07d640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40380c68b0 0x7f40380b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.832+0000 7f403d87c640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40380c68b0 0x7f40380b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.832+0000 7f403d87c640 1 -- 172.21.15.66:0/3702639252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40380b4020 con 0x7f40380ccd30 2024-03-30T13:10:09.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.832+0000 7f403e07d640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40380c68b0 0x7f40380b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:09.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.832+0000 7f403d87c640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 0x7f4038158fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4030001a50 tx=0x7f4030001f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.833+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030018070 con 0x7f40380ccd30 2024-03-30T13:10:09.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.833+0000 7f403fae0640 1 -- 172.21.15.66:0/3702639252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40380b4300 con 0x7f40380ccd30 2024-03-30T13:10:09.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.833+0000 7f403fae0640 1 -- 172.21.15.66:0/3702639252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40380b48c0 con 0x7f40380ccd30 2024-03-30T13:10:09.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.835+0000 7f403fae0640 1 -- 172.21.15.66:0/3702639252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40380c6cc0 con 0x7f40380ccd30 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.835+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403000d6c0 con 0x7f40380ccd30 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.836+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000dab0 con 0x7f40380ccd30 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.836+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f403001b070 con 0x7f40380ccd30 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.836+0000 7f402e7fc640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f401c070a40 0x7f401c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.837+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f403008b1b0 con 0x7f40380ccd30 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.837+0000 7f403d07b640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f401c070a40 0x7f401c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.838+0000 7f403d07b640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f401c070a40 0x7f401c072f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4034036000 tx=0x7f4034030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.841+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f40380cb850 con 0x7f40380ccd30 2024-03-30T13:10:09.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.871+0000 7f73fa862640 1 Processor -- start 2024-03-30T13:10:09.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.871+0000 7f73fa862640 1 -- start start 2024-03-30T13:10:09.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f084d022640 1 Processor -- start 2024-03-30T13:10:09.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f385359e640 1 Processor -- start 2024-03-30T13:10:09.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f385359e640 1 -- start start 2024-03-30T13:10:09.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.872+0000 7f73e82c5640 1 Processor -- start 2024-03-30T13:10:09.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.872+0000 7f73e82c5640 1 -- start start 2024-03-30T13:10:09.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.872+0000 7f225999d640 1 Processor -- start 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f73fa862640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f4154d30 0x7f73f40cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f73fa862640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f40cc790 con 0x7f73f415ba80 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f73fa862640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f40cca50 con 0x7f73f40d06f0 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f73fa862640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f73f40ccd30 con 0x7f73f4154d30 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7fa09e3d3640 1 Processor -- start 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7fa09e3d3640 1 -- start start 2024-03-30T13:10:09.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e82c5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73d8054970 0x7f73d8050290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e82c5640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d8050d40 con 0x7f73d8008de0 2024-03-30T13:10:09.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e82c5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d8050fe0 con 0x7f73d8009270 2024-03-30T13:10:09.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e82c5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f73d80e58e0 con 0x7f73d8054970 2024-03-30T13:10:09.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f225999d640 1 -- start start 2024-03-30T13:10:09.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f225999d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 0x7f224c03a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f73f37fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f73f415ba80 0x7f73f40cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53930/0 (socket says 172.21.15.66:53930) 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.873+0000 7f73f37fe640 1 -- 172.21.15.66:0/1645872723 learned_addr learned my addr 172.21.15.66:0/1645872723 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e6061640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73d8054970 0x7f73d8050290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e6061640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73d8054970 0x7f73d8050290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35108/0 (socket says 172.21.15.66:35108) 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e6061640 1 -- 172.21.15.66:0/2058412954 learned_addr learned my addr 172.21.15.66:0/2058412954 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f225999d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f224c00ab30 0x7f224c009d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f225899b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 0x7f224c03a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f225899b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 0x7f224c03a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39336/0 (socket says 172.21.15.66:39336) 2024-03-30T13:10:09.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f225899b640 1 -- 172.21.15.66:0/1987824652 learned_addr learned my addr 172.21.15.66:0/1987824652 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f2253fff640 1 --2- 172.21.15.66:0/1987824652 >> v2:172.21.15.117:3300/0 conn(0x7f224c00ab30 0x7f224c009d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73f3fff640 1 --2- 172.21.15.66:0/1645872723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f4154d30 0x7f73f40cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73f2ffd640 1 -- 172.21.15.66:0/1645872723 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604844410 0 0) 0x7f73f40cc790 con 0x7f73f415ba80 2024-03-30T13:10:09.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e6061640 1 -- 172.21.15.66:0/2058412954 >> v1:172.21.15.117:6789/0 conn(0x7f73d8009270 legacy=0x7f73d8053a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:10:09.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e6061640 1 -- 172.21.15.66:0/2058412954 >> v1:172.21.15.66:6789/0 conn(0x7f73d8008de0 legacy=0x7f73d8038d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73f2ffd640 1 -- 172.21.15.66:0/1645872723 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73dc003440 con 0x7f73f415ba80 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f084d022640 1 -- start start 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f385359e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f384c03aa50 0x7f384c058a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f385359e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f384c0eb0b0 0x7f384c0572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f385359e640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f384c055850 con 0x7f384c04cb20 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f385359e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f384c0eb500 con 0x7f384c03aa50 2024-03-30T13:10:09.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f385359e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f384c0ebfc0 con 0x7f384c0eb0b0 2024-03-30T13:10:09.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7f73e6061640 1 -- 172.21.15.66:0/2058412954 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d80085e0 con 0x7f73d8054970 2024-03-30T13:10:09.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c03a6a0 con 0x7f224c0516d0 2024-03-30T13:10:09.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f224c055910 con 0x7f224c00ab30 2024-03-30T13:10:09.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f224c054d00 con 0x7f224c0e1300 2024-03-30T13:10:09.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f73f3fff640 1 -- 172.21.15.66:0/1645872723 >> v1:172.21.15.117:6789/0 conn(0x7f73f40d06f0 legacy=0x7f73f40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f73f3fff640 1 -- 172.21.15.66:0/1645872723 >> v1:172.21.15.66:6789/0 conn(0x7f73f415ba80 legacy=0x7f73f40cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f73f3fff640 1 -- 172.21.15.66:0/1645872723 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f40cd010 con 0x7f73f4154d30 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7fa09e3d3640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa0900e4500 0x7fa090057690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7fa09e3d3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa090038020 con 0x7fa09003bd50 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7fa09e3d3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa090038300 con 0x7fa0900e48e0 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.874+0000 7fa09e3d3640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa0900072e0 con 0x7fa0900e4500 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7fa09d3d1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa0900e4500 0x7fa090057690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7fa09d3d1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa0900e4500 0x7fa090057690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34426/0 (socket says 172.21.15.66:34426) 2024-03-30T13:10:09.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1869665337 learned_addr learned my addr 172.21.15.66:0/1869665337 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f3851d9b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f384c04cb20 0x7f384c03ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53972/0 (socket says 172.21.15.66:53972) 2024-03-30T13:10:09.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f3851d9b640 1 -- 172.21.15.66:0/2603460010 learned_addr learned my addr 172.21.15.66:0/2603460010 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:09.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f3852d9d640 1 --2- 172.21.15.66:0/2603460010 >> v2:172.21.15.117:3300/0 conn(0x7f384c0eb0b0 0x7f384c0572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f385259c640 1 --2- 172.21.15.66:0/2603460010 >> v2:172.21.15.66:3300/0 conn(0x7f384c03aa50 0x7f384c058a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f385159a640 1 -- 172.21.15.66:0/2603460010 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3666619385 0 0) 0x7f384c055850 con 0x7f384c04cb20 2024-03-30T13:10:09.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f385159a640 1 -- 172.21.15.66:0/2603460010 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f383c003520 con 0x7f384c04cb20 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1869665337 >> v1:172.21.15.117:6789/0 conn(0x7fa0900e48e0 legacy=0x7fa090054940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1869665337 >> v1:172.21.15.66:6789/0 conn(0x7fa09003bd50 legacy=0x7fa09003c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1869665337 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0900075c0 con 0x7fa0900e4500 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225899b640 1 -- 172.21.15.66:0/1987824652 >> v2:172.21.15.117:3300/0 conn(0x7f224c00ab30 msgr2=0x7f224c009d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225899b640 1 --2- 172.21.15.66:0/1987824652 >> v2:172.21.15.117:3300/0 conn(0x7f224c00ab30 0x7f224c009d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225899b640 1 -- 172.21.15.66:0/1987824652 >> v1:172.21.15.66:6789/0 conn(0x7f224c0516d0 legacy=0x7f224c055da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f225899b640 1 -- 172.21.15.66:0/1987824652 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c054fe0 con 0x7f224c0e1300 2024-03-30T13:10:09.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f73e6061640 1 --2- 172.21.15.66:0/2058412954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8050290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009870 tx=0x7f73dc02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef66518c8f523fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f3852d9d640 1 -- 172.21.15.66:0/2603460010 >> v2:172.21.15.66:3300/0 conn(0x7f384c03aa50 msgr2=0x7f384c058a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f3852d9d640 1 --2- 172.21.15.66:0/2603460010 >> v2:172.21.15.66:3300/0 conn(0x7f384c03aa50 0x7f384c058a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f3852d9d640 1 -- 172.21.15.66:0/2603460010 >> v1:172.21.15.66:6789/0 conn(0x7f384c04cb20 legacy=0x7f384c03ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.875+0000 7f084d022640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08481625b0 0x7f084806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f3852d9d640 1 -- 172.21.15.66:0/2603460010 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c0eb720 con 0x7f384c0eb0b0 2024-03-30T13:10:09.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f385259c640 1 --2- 172.21.15.66:0/2603460010 >> v2:172.21.15.66:3300/0 conn(0x7f384c03aa50 0x7f384c058a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:09.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f73f3fff640 1 --2- 172.21.15.66:0/1645872723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f4154d30 0x7f73f40cb230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73e40097d0 tx=0x7f73e402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc236a7701ba7992 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f225899b640 1 --2- 172.21.15.66:0/1987824652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 0x7f224c03a160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f224800dc90 tx=0x7f224802fd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5596db08f26837 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f084d022640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08481670e0 con 0x7f08481662b0 2024-03-30T13:10:09.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f0846d9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08481625b0 0x7f084806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f0846d9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08481625b0 0x7f084806daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35130/0 (socket says 172.21.15.66:35130) 2024-03-30T13:10:09.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.876+0000 7f73f2ffd640 1 -- 172.21.15.66:0/1645872723 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4035020 con 0x7f73f4154d30 2024-03-30T13:10:09.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f73f2ffd640 1 -- 172.21.15.66:0/1645872723 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4031800 con 0x7f73f4154d30 2024-03-30T13:10:09.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f73f2ffd640 1 -- 172.21.15.66:0/1645872723 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4031bd0 con 0x7f73f4154d30 2024-03-30T13:10:09.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f0846d9d640 1 -- 172.21.15.66:0/2206313785 learned_addr learned my addr 172.21.15.66:0/2206313785 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f22537fe640 1 -- 172.21.15.66:0/1987824652 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480308a0 con 0x7f224c0e1300 2024-03-30T13:10:09.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f22537fe640 1 -- 172.21.15.66:0/1987824652 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2248030a40 con 0x7f224c0e1300 2024-03-30T13:10:09.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08480ca720 con 0x7f08480cf3b0 2024-03-30T13:10:09.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f08480caa00 con 0x7f08481625b0 2024-03-30T13:10:09.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f22537fe640 1 -- 172.21.15.66:0/1987824652 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248030e10 con 0x7f224c0e1300 2024-03-30T13:10:09.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7fa09d3d1640 1 --2- 172.21.15.66:0/1869665337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa090057690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa09806abd0 tx=0x7fa0980a7020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442950ee585c7b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f7500be0640 1 Processor -- start 2024-03-30T13:10:09.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f0846d9d640 1 -- 172.21.15.66:0/2206313785 >> v1:172.21.15.117:6789/0 conn(0x7f08480cf3b0 legacy=0x7f08480cc770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:10:09.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f0846d9d640 1 -- 172.21.15.66:0/2206313785 >> v1:172.21.15.66:6789/0 conn(0x7f08481662b0 legacy=0x7f08480cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7fa08ffff640 1 -- 172.21.15.66:0/1869665337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980b4070 con 0x7fa0900e4500 2024-03-30T13:10:09.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f0846d9d640 1 -- 172.21.15.66:0/2206313785 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08480cd8c0 con 0x7f08481625b0 2024-03-30T13:10:09.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f7500be0640 1 -- start start 2024-03-30T13:10:09.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f73e505f640 1 -- 172.21.15.66:0/2058412954 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc035020 con 0x7f73d8054970 2024-03-30T13:10:09.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f73e505f640 1 -- 172.21.15.66:0/2058412954 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc031870 con 0x7f73d8054970 2024-03-30T13:10:09.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7f73e505f640 1 -- 172.21.15.66:0/2058412954 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc031c40 con 0x7f73d8054970 2024-03-30T13:10:09.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.877+0000 7fa08ffff640 1 -- 172.21.15.66:0/1869665337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980af440 con 0x7fa0900e4500 2024-03-30T13:10:09.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 msgr2=0x7f224c03a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 --2- 172.21.15.66:0/1987824652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 0x7f224c03a160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f224800dc90 tx=0x7f224802fd00 comp rx=0 tx=0).stop 2024-03-30T13:10:09.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7fa08ffff640 1 -- 172.21.15.66:0/1869665337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980af8f0 con 0x7fa0900e4500 2024-03-30T13:10:09.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 shutdown_connections 2024-03-30T13:10:09.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 --2- 172.21.15.66:0/1987824652 >> v2:172.21.15.117:3300/0 conn(0x7f224c00ab30 0x7f224c009d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 --2- 172.21.15.66:0/1987824652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0e1300 0x7f224c03a160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 >> 172.21.15.66:0/1987824652 conn(0x7f224c056cc0 msgr2=0x7f224c0570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1869665337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 msgr2=0x7fa090057690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 shutdown_connections 2024-03-30T13:10:09.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1869665337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa090057690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa09806abd0 tx=0x7fa0980a7020 comp rx=0 tx=0).stop 2024-03-30T13:10:09.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f7500be0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 0x7f74fc15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f7500be0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74fc0cf860 0x7f74fc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f7500be0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74fc15e880 con 0x7f74fc0ccae0 2024-03-30T13:10:09.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f7500be0640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f74fc15ee40 con 0x7f74fc0cf860 2024-03-30T13:10:09.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f7500be0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f74fc15eb60 con 0x7f74fc0b22f0 2024-03-30T13:10:09.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f225999d640 1 -- 172.21.15.66:0/1987824652 wait complete. 2024-03-30T13:10:09.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1869665337 shutdown_connections 2024-03-30T13:10:09.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1869665337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa090057690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1869665337 >> 172.21.15.66:0/1869665337 conn(0x7fa090020040 msgr2=0x7fa090056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f0846d9d640 1 --2- 172.21.15.66:0/2206313785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08481625b0 0x7f084806daf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f083c0098a0 tx=0x7f083c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af547d42eaa995c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f73e82c5640 1 -- 172.21.15.66:0/2058412954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 msgr2=0x7f73d8050290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2058412954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8050290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009870 tx=0x7f73dc02b910 comp rx=0 tx=0).stop 2024-03-30T13:10:09.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.878+0000 7f3852d9d640 1 --2- 172.21.15.66:0/2603460010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c0eb0b0 0x7f384c0572e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f384c007440 tx=0x7f384802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af63725c964c04b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f74faffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74fc0cf860 0x7f74fc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f74fb7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 0x7f74fc15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f385159a640 1 -- 172.21.15.66:0/2603460010 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38480315a0 con 0x7f384c0eb0b0 2024-03-30T13:10:09.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 -- 172.21.15.66:0/1645872723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f4154d30 msgr2=0x7f73f40cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 --2- 172.21.15.66:0/1645872723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f4154d30 0x7f73f40cb230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73e40097d0 tx=0x7f73e402b8a0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f385159a640 1 -- 172.21.15.66:0/2603460010 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848037070 con 0x7f384c0eb0b0 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f385159a640 1 -- 172.21.15.66:0/2603460010 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848031960 con 0x7f384c0eb0b0 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73e82c5640 1 -- 172.21.15.66:0/2058412954 shutdown_connections 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2058412954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8050290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73e82c5640 1 -- 172.21.15.66:0/2058412954 >> 172.21.15.66:0/2058412954 conn(0x7f73d8055c40 msgr2=0x7f73d8056050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73e82c5640 1 -- 172.21.15.66:0/2058412954 shutdown_connections 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73e82c5640 1 -- 172.21.15.66:0/2058412954 wait complete. 2024-03-30T13:10:09.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f0845d9b640 1 -- 172.21.15.66:0/2206313785 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c034070 con 0x7f08481625b0 2024-03-30T13:10:09.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 -- 172.21.15.66:0/1645872723 shutdown_connections 2024-03-30T13:10:09.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 --2- 172.21.15.66:0/1645872723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f4154d30 0x7f73f40cb230 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 -- 172.21.15.66:0/1645872723 >> 172.21.15.66:0/1645872723 conn(0x7f73f406d610 msgr2=0x7f73f406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f74fb7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 0x7f74fc15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35146/0 (socket says 172.21.15.66:35146) 2024-03-30T13:10:09.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f0845d9b640 1 -- 172.21.15.66:0/2206313785 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c030840 con 0x7f08481625b0 2024-03-30T13:10:09.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f74fb7fe640 1 -- 172.21.15.66:0/2522131624 learned_addr learned my addr 172.21.15.66:0/2522131624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f74faffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74fc0cf860 0x7f74fc15df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34448/0 (socket says 172.21.15.66:34448) 2024-03-30T13:10:09.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1869665337 shutdown_connections 2024-03-30T13:10:09.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1869665337 wait complete. 2024-03-30T13:10:09.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 -- 172.21.15.66:0/1645872723 shutdown_connections 2024-03-30T13:10:09.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f74faffd640 1 -- 172.21.15.66:0/2522131624 >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 msgr2=0x7f74fc15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.879+0000 7f73fa862640 1 -- 172.21.15.66:0/1645872723 wait complete. 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f74faffd640 1 --2- 172.21.15.66:0/2522131624 >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 0x7f74fc15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f74faffd640 1 -- 172.21.15.66:0/2522131624 >> v1:172.21.15.117:6789/0 conn(0x7f74fc0ccae0 legacy=0x7f74fc06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f74faffd640 1 -- 172.21.15.66:0/2522131624 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc15aac0 con 0x7f74fc0cf860 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08481625b0 msgr2=0x7f084806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f084d022640 1 --2- 172.21.15.66:0/2206313785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08481625b0 0x7f084806daf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f083c0098a0 tx=0x7f083c02b980 comp rx=0 tx=0).stop 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 shutdown_connections 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f084d022640 1 --2- 172.21.15.66:0/2206313785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08481625b0 0x7f084806daf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 >> 172.21.15.66:0/2206313785 conn(0x7f08480b1800 msgr2=0x7f08480b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.880+0000 7f74fb7fe640 1 --2- 172.21.15.66:0/2522131624 >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 0x7f74fc15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:09.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f225999d640 1 Processor -- start 2024-03-30T13:10:09.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f385359e640 1 -- 172.21.15.66:0/2603460010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c0eb0b0 msgr2=0x7f384c0572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f385359e640 1 --2- 172.21.15.66:0/2603460010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c0eb0b0 0x7f384c0572e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f384c007440 tx=0x7f384802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:09.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f225999d640 1 -- start start 2024-03-30T13:10:09.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f73e82c5640 1 Processor -- start 2024-03-30T13:10:09.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7fa09e3d3640 1 Processor -- start 2024-03-30T13:10:09.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f73e82c5640 1 -- start start 2024-03-30T13:10:09.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7fa09e3d3640 1 -- start start 2024-03-30T13:10:09.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f73fa862640 1 Processor -- start 2024-03-30T13:10:09.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f74faffd640 1 --2- 172.21.15.66:0/2522131624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0cf860 0x7f74fc15df10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74fc15efc0 tx=0x7f74f002bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9eb434061a8951 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 shutdown_connections 2024-03-30T13:10:09.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f084d022640 1 -- 172.21.15.66:0/2206313785 wait complete. 2024-03-30T13:10:09.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.881+0000 7f73fa862640 1 -- start start 2024-03-30T13:10:09.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f74fa7fc640 1 -- 172.21.15.66:0/2522131624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f002d6e0 con 0x7f74fc0cf860 2024-03-30T13:10:09.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f74fa7fc640 1 -- 172.21.15.66:0/2522131624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f002d880 con 0x7f74fc0cf860 2024-03-30T13:10:09.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f74fa7fc640 1 -- 172.21.15.66:0/2522131624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f002dc50 con 0x7f74fc0cf860 2024-03-30T13:10:09.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f385359e640 1 -- 172.21.15.66:0/2603460010 shutdown_connections 2024-03-30T13:10:09.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f385359e640 1 --2- 172.21.15.66:0/2603460010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c0eb0b0 0x7f384c0572e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7fa09e3d3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 0x7fa0900db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f385359e640 1 --2- 172.21.15.66:0/2603460010 >> v2:172.21.15.66:3300/0 conn(0x7f384c03aa50 0x7f384c058a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f385359e640 1 -- 172.21.15.66:0/2603460010 >> 172.21.15.66:0/2603460010 conn(0x7f384c058fd0 msgr2=0x7f384c0593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f225999d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c00ab30 0x7f224c0dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7fa09e3d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa0900d2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09d3d1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 0x7fa0900db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09d3d1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 0x7fa0900db5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39356/0 (socket says 172.21.15.66:39356) 2024-03-30T13:10:09.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225899b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c00ab30 0x7f224c0dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225899b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c00ab30 0x7f224c0dbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35162/0 (socket says 172.21.15.66:35162) 2024-03-30T13:10:09.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225899b640 1 -- 172.21.15.66:0/232052518 learned_addr learned my addr 172.21.15.66:0/232052518 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f385359e640 1 -- 172.21.15.66:0/2603460010 shutdown_connections 2024-03-30T13:10:09.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f385359e640 1 -- 172.21.15.66:0/2603460010 wait complete. 2024-03-30T13:10:09.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0516d0 0x7f224c0d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 -- 172.21.15.66:0/2522131624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0cf860 msgr2=0x7f74fc15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 --2- 172.21.15.66:0/2522131624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0cf860 0x7f74fc15df10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74fc15efc0 tx=0x7f74f002bb30 comp rx=0 tx=0).stop 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 -- 172.21.15.66:0/2522131624 shutdown_connections 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 --2- 172.21.15.66:0/2522131624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0cf860 0x7f74fc15df10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 --2- 172.21.15.66:0/2522131624 >> v2:172.21.15.117:3300/0 conn(0x7f74fc0b22f0 0x7f74fc15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 -- 172.21.15.66:0/2522131624 >> 172.21.15.66:0/2522131624 conn(0x7f74fc0b1800 msgr2=0x7f74fc0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224c0e1300 0x7f224c0da010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f2253fff640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0516d0 0x7f224c0d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1728116945 learned_addr learned my addr 172.21.15.66:0/1728116945 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900e48e0 0x7fa090005e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 -- 172.21.15.66:0/2522131624 shutdown_connections 2024-03-30T13:10:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f224c054890 con 0x7f224c0e1300 2024-03-30T13:10:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0555a0 con 0x7f224c00ab30 2024-03-30T13:10:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900ec1e0 con 0x7fa0900e4500 2024-03-30T13:10:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f224c054a10 con 0x7f224c0516d0 2024-03-30T13:10:09.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f225919c640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224c0e1300 0x7f224c0da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f7500be0640 1 -- 172.21.15.66:0/2522131624 wait complete. 2024-03-30T13:10:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09dbd2640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900e48e0 0x7fa090005e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900ec060 con 0x7fa0900e48e0 2024-03-30T13:10:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa0900ec360 con 0x7fa09003bd50 2024-03-30T13:10:09.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7fa09cbd0640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa0900d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e82c5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d8008de0 0x7f73d80d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e82c5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d8009270 0x7f73d8047410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e82c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8047950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e82c5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73d80507d0 con 0x7f73d8008de0 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e82c5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f73d80512c0 con 0x7f73d8054970 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e82c5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f73d8039480 con 0x7f73d8009270 2024-03-30T13:10:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e6862640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8047950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e6862640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8047950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35148/0 (socket says 172.21.15.66:35148) 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.882+0000 7f73e6862640 1 -- 172.21.15.66:0/2208392426 learned_addr learned my addr 172.21.15.66:0/2208392426 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e6862640 1 -- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d8009270 msgr2=0x7f73d8047410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e5860640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d8009270 0x7f73d8047410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e6862640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d8009270 0x7f73d8047410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e6862640 1 -- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d8008de0 msgr2=0x7f73d80d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e6061640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d8008de0 0x7f73d80d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e6862640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d8008de0 0x7f73d80d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e6862640 1 -- 172.21.15.66:0/2208392426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d80d07a0 con 0x7f73d8054970 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.883+0000 7f73e5860640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d8009270 0x7f73d8047410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900e48e0 msgr2=0x7fa090005e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73e6061640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d8008de0 0x7f73d80d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7fa09d3d1640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900e48e0 0x7fa090005e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 msgr2=0x7fa0900d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7fa09d3d1640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa0900d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7fa09d3d1640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0900dbaf0 con 0x7fa09003bd50 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f2253fff640 1 -- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c00ab30 msgr2=0x7f224c0dbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f2253fff640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c00ab30 0x7f224c0dbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f2253fff640 1 -- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224c0e1300 msgr2=0x7f224c0da010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f2253fff640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224c0e1300 0x7f224c0da010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f2253fff640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c008880 con 0x7f224c0516d0 2024-03-30T13:10:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f084d022640 1 Processor -- start 2024-03-30T13:10:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f084d022640 1 -- start start 2024-03-30T13:10:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73fa862640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f40d06f0 0x7f73f4152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f2253fff640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0516d0 0x7f224c0d9ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f224000e870 tx=0x7f224000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73fa862640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 0x7f73f41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73fa862640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f415ba80 0x7f73f415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73fa862640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4155110 con 0x7f73f4154d30 2024-03-30T13:10:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73fa862640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40cb770 con 0x7f73f415ba80 2024-03-30T13:10:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73e6862640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8047950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e0050e40 tx=0x7f73e0074420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.884+0000 7f73fa862640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f73f4150f10 con 0x7f73f40d06f0 2024-03-30T13:10:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7fa09d3d1640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 0x7fa0900db5b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa09806a520 tx=0x7fa098075560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f7500be0640 1 Processor -- start 2024-03-30T13:10:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 0x7f73f41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 0x7f73f41509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34474/0 (socket says 172.21.15.66:34474) 2024-03-30T13:10:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f37fe640 1 -- 172.21.15.66:0/3432491596 learned_addr learned my addr 172.21.15.66:0/3432491596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f3fff640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f40d06f0 0x7f73f4152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f7500be0640 1 -- start start 2024-03-30T13:10:09.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980be070 con 0x7fa09003bd50 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f084d022640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 0x7f08480b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f084d022640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08481625b0 0x7f08481ed140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f3fff640 1 -- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f415ba80 msgr2=0x7f73f415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f3fff640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f415ba80 0x7f73f415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f3fff640 1 -- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 msgr2=0x7f73f41509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f3fff640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 0x7f73f41509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f3fff640 1 -- 172.21.15.66:0/3432491596 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f41511b0 con 0x7f73f40d06f0 2024-03-30T13:10:09.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f73f37fe640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 0x7f73f41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0900d2c80 con 0x7fa09003bd50 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0900d3240 con 0x7fa09003bd50 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240017070 con 0x7f224c0516d0 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224000ee60 con 0x7f224c0516d0 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224000cb60 con 0x7f224c0516d0 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c008b60 con 0x7f224c0516d0 2024-03-30T13:10:09.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c009120 con 0x7f224c0516d0 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f7500be0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 0x7f74fc1fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f7500be0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc0ccae0 0x7f74fc1fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f7500be0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74fc0cf860 0x7f74fc1f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f7500be0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc15e450 con 0x7f74fc0b22f0 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f7500be0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc15b800 con 0x7f74fc0ccae0 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f7500be0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74fc15e5d0 con 0x7f74fc0cf860 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f74fb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 0x7f74fc1fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f74fb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 0x7f74fc1fbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34498/0 (socket says 172.21.15.66:34498) 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f74fb7fe640 1 -- 172.21.15.66:0/3993442888 learned_addr learned my addr 172.21.15.66:0/3993442888 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f74fb7fe640 1 -- 172.21.15.66:0/3993442888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74fc0cf860 msgr2=0x7f74fc1f2340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f74fb7fe640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74fc0cf860 0x7f74fc1f2340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f74fb7fe640 1 -- 172.21.15.66:0/3993442888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc0ccae0 msgr2=0x7f74fc1fc3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f74fb7fe640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc0ccae0 0x7f74fc1fc3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f74fb7fe640 1 -- 172.21.15.66:0/3993442888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc0c4d10 con 0x7f74fc0b22f0 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f385359e640 1 Processor -- start 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.885+0000 7f385359e640 1 -- start start 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f385359e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 0x7f384c0d8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f385359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c04cb20 0x7f384c0e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f385359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 0x7f384c0e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f3852d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 0x7f384c0e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f3852d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 0x7f384c0e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34504/0 (socket says 172.21.15.66:34504) 2024-03-30T13:10:09.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f385259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 0x7f384c0d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f385259c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 0x7f384c0d8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:39384/0 (socket says 172.21.15.66:39384) 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f3852d9d640 1 -- 172.21.15.66:0/2361072202 learned_addr learned my addr 172.21.15.66:0/2361072202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0e7870 con 0x7f384c0eb0b0 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0ebce0 con 0x7f384c04cb20 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f384c057820 con 0x7f384c03aa50 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e006dc60 con 0x7f73d8054970 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e00a0070 con 0x7f73d8054970 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00749e0 con 0x7f73d8054970 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d8006f40 con 0x7f73d8054970 2024-03-30T13:10:09.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d8049750 con 0x7f73d8054970 2024-03-30T13:10:09.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f74fb7fe640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 0x7f74fc1fbe80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00e840 tx=0x7f74ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f73f3fff640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f40d06f0 0x7f73f4152e20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73e4033000 tx=0x7f73e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c0057d0 con 0x7f224c0516d0 2024-03-30T13:10:09.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f3851d9b640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c04cb20 0x7f384c0e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f224001a070 con 0x7f224c0516d0 2024-03-30T13:10:09.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f22517fa640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f223806c600 0x7f223806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f224008bb00 con 0x7f224c0516d0 2024-03-30T13:10:09.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f084d022640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08481662b0 0x7f08480b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00f040 con 0x7f74fc0b22f0 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec004230 con 0x7f74fc0b22f0 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f084d022640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f084806e030 con 0x7f08481625b0 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f084d022640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08480cde80 con 0x7f08480cf3b0 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f084d022640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f08480cce80 con 0x7f08481662b0 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 0x7f08480b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00c6d0 con 0x7f74fc0b22f0 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 0x7f08480b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35190/0 (socket says 172.21.15.66:35190) 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 -- 172.21.15.66:0/2338676305 learned_addr learned my addr 172.21.15.66:0/2338676305 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 -- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08481662b0 msgr2=0x7f08480b4b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08481662b0 0x7f08480b4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 -- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08481625b0 msgr2=0x7f08481ed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08481625b0 0x7f08481ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.886+0000 7f0846d9d640 1 -- 172.21.15.66:0/2338676305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08481edb00 con 0x7f08480cf3b0 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f0846d9d640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 0x7f08480b2b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f083c0099c0 tx=0x7f083c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c048070 con 0x7f08480cf3b0 2024-03-30T13:10:09.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f084d022640 1 -- 172.21.15.66:0/2338676305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08481edde0 con 0x7f08480cf3b0 2024-03-30T13:10:09.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f084d022640 1 -- 172.21.15.66:0/2338676305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08481ee3a0 con 0x7f08480cf3b0 2024-03-30T13:10:09.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc0c3c70 con 0x7f74fc0b22f0 2024-03-30T13:10:09.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e403bcf0 con 0x7f73f40d06f0 2024-03-30T13:10:09.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f385259c640 1 -- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c04cb20 msgr2=0x7f384c0e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f385259c640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c04cb20 0x7f384c0e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f385259c640 1 -- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 msgr2=0x7f384c0e5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:09.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c02f5a0 con 0x7f08480cf3b0 2024-03-30T13:10:09.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c034040 con 0x7f08480cf3b0 2024-03-30T13:10:09.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f385259c640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 0x7f384c0e5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:09.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f385259c640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c052190 con 0x7f384c03aa50 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f3852d9d640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 0x7f384c0e5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e40453f0 con 0x7f73f40d06f0 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f4151490 con 0x7f73f40d06f0 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f4151a50 con 0x7f73f40d06f0 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40457e0 con 0x7f73f40d06f0 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d4005290 con 0x7f73f40d06f0 2024-03-30T13:10:09.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f73e0074b80 con 0x7f73d8054970 2024-03-30T13:10:09.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.889+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c8005290 con 0x7f73d8054970 2024-03-30T13:10:09.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.887+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa090005390 con 0x7fa09003bd50 2024-03-30T13:10:09.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980af440 con 0x7fa09003bd50 2024-03-30T13:10:09.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.888+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980be070 con 0x7fa09003bd50 2024-03-30T13:10:09.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc0c4230 con 0x7f74fc0b22f0 2024-03-30T13:10:09.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f225899b640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f223806c600 0x7f223806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f73d6ffd640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73c406c5b0 0x7f73c406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f73e01179d0 con 0x7f73d8054970 2024-03-30T13:10:09.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f385259c640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 0x7f384c0d8560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f384400e7e0 tx=0x7f384400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f73e6061640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73c406c5b0 0x7f73c406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f083c04b070 con 0x7f08480cf3b0 2024-03-30T13:10:09.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.890+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0828005290 con 0x7f08480cf3b0 2024-03-30T13:10:09.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.891+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc0cf4b0 con 0x7f74fc0b22f0 2024-03-30T13:10:09.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.891+0000 7f73e6061640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73c406c5b0 0x7f73c406ea70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73dc036000 tx=0x7f73dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.891+0000 7f225899b640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f223806c600 0x7f223806eac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f224802fd40 tx=0x7f2248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.891+0000 7f082f7fe640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f083407d610 0x7f083407fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.891+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f083c0adcc0 con 0x7f08480cf3b0 2024-03-30T13:10:09.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.891+0000 7f084659c640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f083407d610 0x7f083407fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f74ec00c870 con 0x7f74fc0b22f0 2024-03-30T13:10:09.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f084659c640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f083407d610 0x7f083407fad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08400012e0 tx=0x7f0840007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f74e3fff640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74e806c580 0x7f74e806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f74ec08b780 con 0x7f74fc0b22f0 2024-03-30T13:10:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f74faffd640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74e806c580 0x7f74e806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f74faffd640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74e806c580 0x7f74e806ea40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74f0036000 tx=0x7f74f0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.892+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2240059770 con 0x7f224c0516d0 2024-03-30T13:10:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.893+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400f040 con 0x7f384c03aa50 2024-03-30T13:10:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.893+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3844004030 con 0x7f384c03aa50 2024-03-30T13:10:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.893+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844004420 con 0x7f384c03aa50 2024-03-30T13:10:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.894+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c052470 con 0x7f384c03aa50 2024-03-30T13:10:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.894+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c0e7d70 con 0x7f384c03aa50 2024-03-30T13:10:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.894+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f73e404c060 con 0x7f73f40d06f0 2024-03-30T13:10:09.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.895+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f73d80d07a0 con 0x7f73d8054970 2024-03-30T13:10:09.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.895+0000 7f73f0ff9640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73e0074e10 0x7f73e00772d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.895+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa0980c1070 con 0x7fa09003bd50 2024-03-30T13:10:09.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.895+0000 7f73f37fe640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73e0074e10 0x7f73e00772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.895+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f73e40adbd0 con 0x7f73f40d06f0 2024-03-30T13:10:09.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.895+0000 7fa08dffb640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa07c06c930 0x7fa07c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.896+0000 7f73f37fe640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73e0074e10 0x7f73e00772d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73f414dc90 tx=0x7f73e8005e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.896+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa0980ae770 con 0x7fa09003bd50 2024-03-30T13:10:09.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.896+0000 7fa09cbd0640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa07c06c930 0x7fa07c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.896+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c0057d0 con 0x7f384c03aa50 2024-03-30T13:10:09.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.896+0000 7fa09cbd0640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa07c06c930 0x7fa07c06edf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa094007fd0 tx=0x7fa094009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.897+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa0980f7590 con 0x7fa09003bd50 2024-03-30T13:10:09.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.899+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f083c07bf10 con 0x7f08480cf3b0 2024-03-30T13:10:09.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.902+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f73e40ade20 con 0x7f73f40d06f0 2024-03-30T13:10:09.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.901+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f74fc0cf4b0 con 0x7f74fc0b22f0 2024-03-30T13:10:09.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.903+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f384400c6d0 con 0x7f384c03aa50 2024-03-30T13:10:09.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.903+0000 7f3842ffd640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f383406c530 0x7f383406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:09.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.904+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f384408bb40 con 0x7f384c03aa50 2024-03-30T13:10:09.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.904+0000 7f3851d9b640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f383406c530 0x7f383406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:09.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.904+0000 7f3851d9b640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f383406c530 0x7f383406e9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38540510d0 tx=0x7f385406b4b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:09.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:09.906+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3844017020 con 0x7f384c03aa50 2024-03-30T13:10:10.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.259+0000 7f403fae0640 1 -- 172.21.15.66:0/3702639252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4038075900 con 0x7f40380ccd30 2024-03-30T13:10:10.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.260+0000 7f402e7fc640 1 -- 172.21.15.66:0/3702639252 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4030058e20 con 0x7f40380ccd30 2024-03-30T13:10:10.263 INFO:teuthology.orchestra.run.smithi066.stdout:64424509449 2024-03-30T13:10:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.263+0000 7f401bfff640 1 -- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f401c070a40 msgr2=0x7f401c072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.263+0000 7f401bfff640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f401c070a40 0x7f401c072f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4034036000 tx=0x7f4034030000 comp rx=0 tx=0).stop 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.263+0000 7f401bfff640 1 -- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 msgr2=0x7f4038158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.263+0000 7f401bfff640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 0x7f4038158fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4030001a50 tx=0x7f4030001f10 comp rx=0 tx=0).stop 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 -- 172.21.15.66:0/3702639252 shutdown_connections 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f401c070a40 0x7f401c072f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40380c68b0 0x7f40380b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f403815a720 0x7f40380c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 --2- 172.21.15.66:0/3702639252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40380ccd30 0x7f4038158fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 -- 172.21.15.66:0/3702639252 >> 172.21.15.66:0/3702639252 conn(0x7f40380b1c20 msgr2=0x7f40380ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 -- 172.21.15.66:0/3702639252 shutdown_connections 2024-03-30T13:10:10.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.264+0000 7f401bfff640 1 -- 172.21.15.66:0/3702639252 wait complete. 2024-03-30T13:10:10.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-03-30T13:10:10.285 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.300+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f74fc075890 con 0x7f74fc0b22f0 2024-03-30T13:10:10.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.301+0000 7f74e3fff640 1 -- 172.21.15.66:0/3993442888 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74ec017020 con 0x7f74fc0b22f0 2024-03-30T13:10:10.304 INFO:teuthology.orchestra.run.smithi066.stdout:55834574859 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74e806c580 msgr2=0x7f74e806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74e806c580 0x7f74e806ea40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74f0036000 tx=0x7f74f0008030 comp rx=0 tx=0).stop 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 msgr2=0x7f74fc1fbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 0x7f74fc1fbe80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00e840 tx=0x7f74ec00ed00 comp rx=0 tx=0).stop 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 shutdown_connections 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74e806c580 0x7f74e806ea40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74fc0cf860 0x7f74fc1f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74fc0ccae0 0x7f74fc1fc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 --2- 172.21.15.66:0/3993442888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74fc0b22f0 0x7f74fc1fbe80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 >> 172.21.15.66:0/3993442888 conn(0x7f74fc0b1800 msgr2=0x7f74fc072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 shutdown_connections 2024-03-30T13:10:10.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.303+0000 7f7500be0640 1 -- 172.21.15.66:0/3993442888 wait complete. 2024-03-30T13:10:10.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-03-30T13:10:10.318 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.329+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f224c0e09b0 con 0x7f224c0516d0 2024-03-30T13:10:10.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.329+0000 7f22517fa640 1 -- 172.21.15.66:0/232052518 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f224005da70 con 0x7f224c0516d0 2024-03-30T13:10:10.333 INFO:teuthology.orchestra.run.smithi066.stdout:64424509449 2024-03-30T13:10:10.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.331+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f73c8005580 con 0x7f73d8054970 2024-03-30T13:10:10.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.331+0000 7f73d6ffd640 1 -- 172.21.15.66:0/2208392426 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73e00e5d20 con 0x7f73d8054970 2024-03-30T13:10:10.335 INFO:teuthology.orchestra.run.smithi066.stdout:60129542153 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f223806c600 msgr2=0x7f223806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f223806c600 0x7f223806eac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f224802fd40 tx=0x7f2248008040 comp rx=0 tx=0).stop 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0516d0 msgr2=0x7f224c0d9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0516d0 0x7f224c0d9ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f224000e870 tx=0x7f224000ed30 comp rx=0 tx=0).stop 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 shutdown_connections 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f223806c600 0x7f223806eac0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224c0e1300 0x7f224c0da010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224c0516d0 0x7f224c0d9ad0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 --2- 172.21.15.66:0/232052518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f224c00ab30 0x7f224c0dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 >> 172.21.15.66:0/232052518 conn(0x7f224c056cc0 msgr2=0x7f224c051bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 shutdown_connections 2024-03-30T13:10:10.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.332+0000 7f225999d640 1 -- 172.21.15.66:0/232052518 wait complete. 2024-03-30T13:10:10.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.334+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f73d4005850 con 0x7f73f40d06f0 2024-03-30T13:10:10.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73c406c5b0 msgr2=0x7f73c406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73c406c5b0 0x7f73c406ea70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73dc036000 tx=0x7f73dc008030 comp rx=0 tx=0).stop 2024-03-30T13:10:10.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 msgr2=0x7f73d8047950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8047950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e0050e40 tx=0x7f73e0074420 comp rx=0 tx=0).stop 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 shutdown_connections 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73c406c5b0 0x7f73c406ea70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73d8054970 0x7f73d8047950 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73d8009270 0x7f73d8047410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 --2- 172.21.15.66:0/2208392426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73d8008de0 0x7f73d80d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 >> 172.21.15.66:0/2208392426 conn(0x7f73d8055c40 msgr2=0x7f73d80376f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.340 INFO:teuthology.orchestra.run.smithi066.stdout:60129542154 2024-03-30T13:10:10.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73f0ff9640 1 -- 172.21.15.66:0/3432491596 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73e4040070 con 0x7f73f40d06f0 2024-03-30T13:10:10.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 shutdown_connections 2024-03-30T13:10:10.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.335+0000 7f73e82c5640 1 -- 172.21.15.66:0/2208392426 wait complete. 2024-03-30T13:10:10.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.336+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f384c00df70 con 0x7f384c03aa50 2024-03-30T13:10:10.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.336+0000 7f3842ffd640 1 -- 172.21.15.66:0/2361072202 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3844059840 con 0x7f384c03aa50 2024-03-30T13:10:10.341 INFO:teuthology.orchestra.run.smithi066.stdout:64424509450 2024-03-30T13:10:10.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.337+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0828005580 con 0x7f08480cf3b0 2024-03-30T13:10:10.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.337+0000 7f082f7fe640 1 -- 172.21.15.66:0/2338676305 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f083c07b810 con 0x7f08480cf3b0 2024-03-30T13:10:10.342 INFO:teuthology.orchestra.run.smithi066.stdout:64424509452 2024-03-30T13:10:10.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa0900058e0 con 0x7fa09003bd50 2024-03-30T13:10:10.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73e0074e10 msgr2=0x7f73e00772d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7fa08dffb640 1 -- 172.21.15.66:0/1728116945 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0980f6e90 con 0x7fa09003bd50 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73e0074e10 0x7f73e00772d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73f414dc90 tx=0x7f73e8005e70 comp rx=0 tx=0).stop 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f40d06f0 msgr2=0x7f73f4152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f40d06f0 0x7f73f4152e20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73e4033000 tx=0x7f73e40046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 shutdown_connections 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f73e0074e10 0x7f73e00772d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73f415ba80 0x7f73f415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f4154d30 0x7f73f41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 --2- 172.21.15.66:0/3432491596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73f40d06f0 0x7f73f4152e20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 >> 172.21.15.66:0/3432491596 conn(0x7f73f406d610 msgr2=0x7f73f4071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.344 INFO:teuthology.orchestra.run.smithi066.stdout:64424509452 2024-03-30T13:10:10.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 shutdown_connections 2024-03-30T13:10:10.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.338+0000 7f73fa862640 1 -- 172.21.15.66:0/3432491596 wait complete. 2024-03-30T13:10:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f383406c530 msgr2=0x7f383406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f383406c530 0x7f383406e9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38540510d0 tx=0x7f385406b4b0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 msgr2=0x7f384c0d8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 0x7f384c0d8560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f384400e7e0 tx=0x7f384400eca0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 shutdown_connections 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f383406c530 0x7f383406e9f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f384c0eb0b0 0x7f384c0e5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f384c04cb20 0x7f384c0e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 --2- 172.21.15.66:0/2361072202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f384c03aa50 0x7f384c0d8560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 >> 172.21.15.66:0/2361072202 conn(0x7f384c058fd0 msgr2=0x7f384c006770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 shutdown_connections 2024-03-30T13:10:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.339+0000 7f385359e640 1 -- 172.21.15.66:0/2361072202 wait complete. 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.340+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f083407d610 msgr2=0x7f083407fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.340+0000 7f082d7fa640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f083407d610 0x7f083407fad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08400012e0 tx=0x7f0840007d30 comp rx=0 tx=0).stop 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 msgr2=0x7f08480b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 0x7f08480b2b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f083c0099c0 tx=0x7f083c004730 comp rx=0 tx=0).stop 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 shutdown_connections 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f083407d610 0x7f083407fad0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08481662b0 0x7f08480b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08481625b0 0x7f08481ed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 --2- 172.21.15.66:0/2338676305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08480cf3b0 0x7f08480b2b80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 >> 172.21.15.66:0/2338676305 conn(0x7f08480b1800 msgr2=0x7f0848071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 shutdown_connections 2024-03-30T13:10:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.341+0000 7f082d7fa640 1 -- 172.21.15.66:0/2338676305 wait complete. 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa07c06c930 msgr2=0x7fa07c06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa07c06c930 0x7fa07c06edf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa094007fd0 tx=0x7fa094009040 comp rx=0 tx=0).stop 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 msgr2=0x7fa0900db5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 0x7fa0900db5b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa09806a520 tx=0x7fa098075560 comp rx=0 tx=0).stop 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 shutdown_connections 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa07c06c930 0x7fa07c06edf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900e48e0 0x7fa090005e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0900e4500 0x7fa0900d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 --2- 172.21.15.66:0/1728116945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa09003bd50 0x7fa0900db5b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 >> 172.21.15.66:0/1728116945 conn(0x7fa090020040 msgr2=0x7fa090034de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 shutdown_connections 2024-03-30T13:10:10.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.342+0000 7fa09e3d3640 1 -- 172.21.15.66:0/1728116945 wait complete. 2024-03-30T13:10:10.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.4 2024-03-30T13:10:10.355 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-03-30T13:10:10.360 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.3 2024-03-30T13:10:10.362 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.1 2024-03-30T13:10:10.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.2 2024-03-30T13:10:10.363 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.363 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.0 2024-03-30T13:10:10.364 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:10.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T13:10:10.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:10:10.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.507+0000 7f8a4a89a640 1 Processor -- start 2024-03-30T13:10:10.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.507+0000 7f8a4a89a640 1 -- start start 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.507+0000 7f8a4a89a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4415e950 0x7f8a440d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.507+0000 7f8a4a89a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a440d36f0 con 0x7f8a440d48c0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.507+0000 7f8a4a89a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a440d39d0 con 0x7f8a440d30d0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.507+0000 7f8a4a89a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8a440d3cb0 con 0x7f8a4415e950 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a437fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8a440d48c0 0x7f8a440d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53986/0 (socket says 172.21.15.66:53986) 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a437fe640 1 -- 172.21.15.66:0/482363747 learned_addr learned my addr 172.21.15.66:0/482363747 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a43fff640 1 --2- 172.21.15.66:0/482363747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4415e950 0x7f8a440d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902903435 0 0) 0x7f8a440d36f0 con 0x7f8a440d48c0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a2c003520 con 0x7f8a440d48c0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3290008051 0 0) 0x7f8a440d39d0 con 0x7f8a440d30d0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a440d36f0 con 0x7f8a440d30d0 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a43fff640 1 -- 172.21.15.66:0/482363747 >> v1:172.21.15.117:6789/0 conn(0x7f8a440d30d0 legacy=0x7f8a440b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a43fff640 1 -- 172.21.15.66:0/482363747 >> v1:172.21.15.66:6789/0 conn(0x7f8a440d48c0 legacy=0x7f8a440d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.508+0000 7f8a43fff640 1 -- 172.21.15.66:0/482363747 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a440b7ac0 con 0x7f8a4415e950 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a43fff640 1 --2- 172.21.15.66:0/482363747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4415e950 0x7f8a440d4380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8a340097d0 tx=0x7f8a3402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90ed618b5dbcb4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34034070 con 0x7f8a4415e950 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a34030880 con 0x7f8a4415e950 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a42ffd640 1 -- 172.21.15.66:0/482363747 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34030c50 con 0x7f8a4415e950 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 -- 172.21.15.66:0/482363747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4415e950 msgr2=0x7f8a440d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/482363747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4415e950 0x7f8a440d4380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8a340097d0 tx=0x7f8a3402b9c0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 -- 172.21.15.66:0/482363747 shutdown_connections 2024-03-30T13:10:10.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/482363747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a4415e950 0x7f8a440d4380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 -- 172.21.15.66:0/482363747 >> 172.21.15.66:0/482363747 conn(0x7f8a440676e0 msgr2=0x7f8a44067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 -- 172.21.15.66:0/482363747 shutdown_connections 2024-03-30T13:10:10.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.509+0000 7f8a4a89a640 1 -- 172.21.15.66:0/482363747 wait complete. 2024-03-30T13:10:10.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 Processor -- start 2024-03-30T13:10:10.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 -- start start 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a440d30d0 0x7f8a4415f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 0x7f8a441f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4415e950 0x7f8a4415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4415ed30 con 0x7f8a440d48c0 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a440d4d00 con 0x7f8a4415e950 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.511+0000 7f8a4a89a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4415eeb0 con 0x7f8a440d30d0 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 0x7f8a441f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 0x7f8a441f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34520/0 (socket says 172.21.15.66:34520) 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 -- 172.21.15.66:0/3579190795 learned_addr learned my addr 172.21.15.66:0/3579190795 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a43fff640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a440d30d0 0x7f8a4415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a48e37640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4415e950 0x7f8a4415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 -- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a440d30d0 msgr2=0x7f8a4415f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a440d30d0 0x7f8a4415f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 -- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4415e950 msgr2=0x7f8a4415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4415e950 0x7f8a4415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 -- 172.21.15.66:0/3579190795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a44160cd0 con 0x7f8a440d48c0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a437fe640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 0x7f8a441f4e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800c530 tx=0x7f8a3800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38017070 con 0x7f8a440d48c0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a38005ce0 con 0x7f8a440d48c0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38007270 con 0x7f8a440d48c0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4415fc30 con 0x7f8a440d48c0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.512+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441601f0 con 0x7f8a440d48c0 2024-03-30T13:10:10.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.513+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a44075a10 con 0x7f8a440d48c0 2024-03-30T13:10:10.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.514+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8a3801a070 con 0x7f8a440d48c0 2024-03-30T13:10:10.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.514+0000 7f8a40ff9640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8a3006c530 0x7f8a3006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.514+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8a3808ac60 con 0x7f8a440d48c0 2024-03-30T13:10:10.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.514+0000 7f8a43fff640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8a3006c530 0x7f8a3006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.515+0000 7f8a43fff640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8a3006c530 0x7f8a3006e9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a34036000 tx=0x7f8a3403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:10.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.519+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8a380588f0 con 0x7f8a440d48c0 2024-03-30T13:10:10.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.698+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a440b6500 con 0x7f8a3006c530 2024-03-30T13:10:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.706+0000 7f8a40ff9640 1 -- 172.21.15.66:0/3579190795 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f8a440b6500 con 0x7f8a3006c530 2024-03-30T13:10:10.713 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:10:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8a3006c530 msgr2=0x7f8a3006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8a3006c530 0x7f8a3006e9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a34036000 tx=0x7f8a3403b040 comp rx=0 tx=0).stop 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 msgr2=0x7f8a441f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 0x7f8a441f4e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800c530 tx=0x7f8a3800c9f0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 shutdown_connections 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8a3006c530 0x7f8a3006e9f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4415e950 0x7f8a4415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a440d48c0 0x7f8a441f4e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 --2- 172.21.15.66:0/3579190795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a440d30d0 0x7f8a4415f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 >> 172.21.15.66:0/3579190795 conn(0x7f8a440676e0 msgr2=0x7f8a440cb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 shutdown_connections 2024-03-30T13:10:10.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.709+0000 7f8a4a89a640 1 -- 172.21.15.66:0/3579190795 wait complete. 2024-03-30T13:10:10.723 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-30T13:10:08.799254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219500,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755220,"statfs":{"total":773094113280,"available":772869345280,"internally_reserved":0,"allocated":3416064,"data_stored":2064256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939160+0000","last_change":"2024-03-30T13:09:42.124458+0000","last_active":"2024-03-30T13:09:49.939160+0000","last_peered":"2024-03-30T13:09:49.939160+0000","last_clean":"2024-03-30T13:09:49.939160+0000","last_became_active":"2024-03-30T13:09:42.124158+0000","last_became_peered":"2024-03-30T13:09:42.124158+0000","last_unstale":"2024-03-30T13:09:49.939160+0000","last_undegraded":"2024-03-30T13:09:49.939160+0000","last_fullsized":"2024-03-30T13:09:49.939160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132823+0000","last_change":"2024-03-30T13:09:43.127433+0000","last_active":"2024-03-30T13:09:49.132823+0000","last_peered":"2024-03-30T13:09:49.132823+0000","last_clean":"2024-03-30T13:09:49.132823+0000","last_became_active":"2024-03-30T13:09:43.127127+0000","last_became_peered":"2024-03-30T13:09:43.127127+0000","last_unstale":"2024-03-30T13:09:49.132823+0000","last_undegraded":"2024-03-30T13:09:49.132823+0000","last_fullsized":"2024-03-30T13:09:49.132823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132184+0000","last_change":"2024-03-30T13:09:47.572430+0000","last_active":"2024-03-30T13:09:49.132184+0000","last_peered":"2024-03-30T13:09:49.132184+0000","last_clean":"2024-03-30T13:09:49.132184+0000","last_became_active":"2024-03-30T13:09:47.572059+0000","last_became_peered":"2024-03-30T13:09:47.572059+0000","last_unstale":"2024-03-30T13:09:49.132184+0000","last_undegraded":"2024-03-30T13:09:49.132184+0000","last_fullsized":"2024-03-30T13:09:49.132184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:31:37.218184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132943+0000","last_change":"2024-03-30T13:09:42.126685+0000","last_active":"2024-03-30T13:09:49.132943+0000","last_peered":"2024-03-30T13:09:49.132943+0000","last_clean":"2024-03-30T13:09:49.132943+0000","last_became_active":"2024-03-30T13:09:42.126383+0000","last_became_peered":"2024-03-30T13:09:42.126383+0000","last_unstale":"2024-03-30T13:09:49.132943+0000","last_undegraded":"2024-03-30T13:09:49.132943+0000","last_fullsized":"2024-03-30T13:09:49.132943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938945+0000","last_change":"2024-03-30T13:09:43.132452+0000","last_active":"2024-03-30T13:09:49.938945+0000","last_peered":"2024-03-30T13:09:49.938945+0000","last_clean":"2024-03-30T13:09:49.938945+0000","last_became_active":"2024-03-30T13:09:43.132158+0000","last_became_peered":"2024-03-30T13:09:43.132158+0000","last_unstale":"2024-03-30T13:09:49.938945+0000","last_undegraded":"2024-03-30T13:09:49.938945+0000","last_fullsized":"2024-03-30T13:09:49.938945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131666+0000","last_change":"2024-03-30T13:09:47.157090+0000","last_active":"2024-03-30T13:09:49.131666+0000","last_peered":"2024-03-30T13:09:49.131666+0000","last_clean":"2024-03-30T13:09:49.131666+0000","last_became_active":"2024-03-30T13:09:47.156676+0000","last_became_peered":"2024-03-30T13:09:47.156676+0000","last_unstale":"2024-03-30T13:09:49.131666+0000","last_undegraded":"2024-03-30T13:09:49.131666+0000","last_fullsized":"2024-03-30T13:09:49.131666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938826+0000","last_change":"2024-03-30T13:09:42.127138+0000","last_active":"2024-03-30T13:09:49.938826+0000","last_peered":"2024-03-30T13:09:49.938826+0000","last_clean":"2024-03-30T13:09:49.938826+0000","last_became_active":"2024-03-30T13:09:42.126853+0000","last_became_peered":"2024-03-30T13:09:42.126853+0000","last_unstale":"2024-03-30T13:09:49.938826+0000","last_undegraded":"2024-03-30T13:09:49.938826+0000","last_fullsized":"2024-03-30T13:09:49.938826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938126+0000","last_change":"2024-03-30T13:09:43.137336+0000","last_active":"2024-03-30T13:09:49.938126+0000","last_peered":"2024-03-30T13:09:49.938126+0000","last_clean":"2024-03-30T13:09:49.938126+0000","last_became_active":"2024-03-30T13:09:43.137057+0000","last_became_peered":"2024-03-30T13:09:43.137057+0000","last_unstale":"2024-03-30T13:09:49.938126+0000","last_undegraded":"2024-03-30T13:09:49.938126+0000","last_fullsized":"2024-03-30T13:09:49.938126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:20.284742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132127+0000","last_change":"2024-03-30T13:09:47.149659+0000","last_active":"2024-03-30T13:09:49.132127+0000","last_peered":"2024-03-30T13:09:49.132127+0000","last_clean":"2024-03-30T13:09:49.132127+0000","last_became_active":"2024-03-30T13:09:47.149298+0000","last_became_peered":"2024-03-30T13:09:47.149298+0000","last_unstale":"2024-03-30T13:09:49.132127+0000","last_undegraded":"2024-03-30T13:09:49.132127+0000","last_fullsized":"2024-03-30T13:09:49.132127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:48.863738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938057+0000","last_change":"2024-03-30T13:09:42.126941+0000","last_active":"2024-03-30T13:09:49.938057+0000","last_peered":"2024-03-30T13:09:49.938057+0000","last_clean":"2024-03-30T13:09:49.938057+0000","last_became_active":"2024-03-30T13:09:42.126646+0000","last_became_peered":"2024-03-30T13:09:42.126646+0000","last_unstale":"2024-03-30T13:09:49.938057+0000","last_undegraded":"2024-03-30T13:09:49.938057+0000","last_fullsized":"2024-03-30T13:09:49.938057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938364+0000","last_change":"2024-03-30T13:09:43.129721+0000","last_active":"2024-03-30T13:09:49.938364+0000","last_peered":"2024-03-30T13:09:49.938364+0000","last_clean":"2024-03-30T13:09:49.938364+0000","last_became_active":"2024-03-30T13:09:43.128999+0000","last_became_peered":"2024-03-30T13:09:43.128999+0000","last_unstale":"2024-03-30T13:09:49.938364+0000","last_undegraded":"2024-03-30T13:09:49.938364+0000","last_fullsized":"2024-03-30T13:09:49.938364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:28:48.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131877+0000","last_change":"2024-03-30T13:09:47.149841+0000","last_active":"2024-03-30T13:09:49.131877+0000","last_peered":"2024-03-30T13:09:49.131877+0000","last_clean":"2024-03-30T13:09:49.131877+0000","last_became_active":"2024-03-30T13:09:47.148965+0000","last_became_peered":"2024-03-30T13:09:47.148965+0000","last_unstale":"2024-03-30T13:09:49.131877+0000","last_undegraded":"2024-03-30T13:09:49.131877+0000","last_fullsized":"2024-03-30T13:09:49.131877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:08.872490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295198+0000","last_change":"2024-03-30T13:09:43.127058+0000","last_active":"2024-03-30T13:09:49.295198+0000","last_peered":"2024-03-30T13:09:49.295198+0000","last_clean":"2024-03-30T13:09:49.295198+0000","last_became_active":"2024-03-30T13:09:43.126777+0000","last_became_peered":"2024-03-30T13:09:43.126777+0000","last_unstale":"2024-03-30T13:09:49.295198+0000","last_undegraded":"2024-03-30T13:09:49.295198+0000","last_fullsized":"2024-03-30T13:09:49.295198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938548+0000","last_change":"2024-03-30T13:09:42.124505+0000","last_active":"2024-03-30T13:09:49.938548+0000","last_peered":"2024-03-30T13:09:49.938548+0000","last_clean":"2024-03-30T13:09:49.938548+0000","last_became_active":"2024-03-30T13:09:42.124243+0000","last_became_peered":"2024-03-30T13:09:42.124243+0000","last_unstale":"2024-03-30T13:09:49.938548+0000","last_undegraded":"2024-03-30T13:09:49.938548+0000","last_fullsized":"2024-03-30T13:09:49.938548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:47.055563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132399+0000","last_change":"2024-03-30T13:09:47.146122+0000","last_active":"2024-03-30T13:09:49.132399+0000","last_peered":"2024-03-30T13:09:49.132399+0000","last_clean":"2024-03-30T13:09:49.132399+0000","last_became_active":"2024-03-30T13:09:47.145870+0000","last_became_peered":"2024-03-30T13:09:47.145870+0000","last_unstale":"2024-03-30T13:09:49.132399+0000","last_undegraded":"2024-03-30T13:09:49.132399+0000","last_fullsized":"2024-03-30T13:09:49.132399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132985+0000","last_change":"2024-03-30T13:09:43.130941+0000","last_active":"2024-03-30T13:09:49.132985+0000","last_peered":"2024-03-30T13:09:49.132985+0000","last_clean":"2024-03-30T13:09:49.132985+0000","last_became_active":"2024-03-30T13:09:43.130335+0000","last_became_peered":"2024-03-30T13:09:43.130335+0000","last_unstale":"2024-03-30T13:09:49.132985+0000","last_undegraded":"2024-03-30T13:09:49.132985+0000","last_fullsized":"2024-03-30T13:09:49.132985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938369+0000","last_change":"2024-03-30T13:09:42.124102+0000","last_active":"2024-03-30T13:09:49.938369+0000","last_peered":"2024-03-30T13:09:49.938369+0000","last_clean":"2024-03-30T13:09:49.938369+0000","last_became_active":"2024-03-30T13:09:42.123746+0000","last_became_peered":"2024-03-30T13:09:42.123746+0000","last_unstale":"2024-03-30T13:09:49.938369+0000","last_undegraded":"2024-03-30T13:09:49.938369+0000","last_fullsized":"2024-03-30T13:09:49.938369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132249+0000","last_change":"2024-03-30T13:09:47.153343+0000","last_active":"2024-03-30T13:09:49.132249+0000","last_peered":"2024-03-30T13:09:49.132249+0000","last_clean":"2024-03-30T13:09:49.132249+0000","last_became_active":"2024-03-30T13:09:47.152894+0000","last_became_peered":"2024-03-30T13:09:47.152894+0000","last_unstale":"2024-03-30T13:09:49.132249+0000","last_undegraded":"2024-03-30T13:09:49.132249+0000","last_fullsized":"2024-03-30T13:09:49.132249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132418+0000","last_change":"2024-03-30T13:09:43.117715+0000","last_active":"2024-03-30T13:09:49.132418+0000","last_peered":"2024-03-30T13:09:49.132418+0000","last_clean":"2024-03-30T13:09:49.132418+0000","last_became_active":"2024-03-30T13:09:43.116768+0000","last_became_peered":"2024-03-30T13:09:43.116768+0000","last_unstale":"2024-03-30T13:09:49.132418+0000","last_undegraded":"2024-03-30T13:09:49.132418+0000","last_fullsized":"2024-03-30T13:09:49.132418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938237+0000","last_change":"2024-03-30T13:09:42.125545+0000","last_active":"2024-03-30T13:09:49.938237+0000","last_peered":"2024-03-30T13:09:49.938237+0000","last_clean":"2024-03-30T13:09:49.938237+0000","last_became_active":"2024-03-30T13:09:42.125280+0000","last_became_peered":"2024-03-30T13:09:42.125280+0000","last_unstale":"2024-03-30T13:09:49.938237+0000","last_undegraded":"2024-03-30T13:09:49.938237+0000","last_fullsized":"2024-03-30T13:09:49.938237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938821+0000","last_change":"2024-03-30T13:09:47.571186+0000","last_active":"2024-03-30T13:09:49.938821+0000","last_peered":"2024-03-30T13:09:49.938821+0000","last_clean":"2024-03-30T13:09:49.938821+0000","last_became_active":"2024-03-30T13:09:47.570755+0000","last_became_peered":"2024-03-30T13:09:47.570755+0000","last_unstale":"2024-03-30T13:09:49.938821+0000","last_undegraded":"2024-03-30T13:09:49.938821+0000","last_fullsized":"2024-03-30T13:09:49.938821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131859+0000","last_change":"2024-03-30T13:09:43.115171+0000","last_active":"2024-03-30T13:09:49.131859+0000","last_peered":"2024-03-30T13:09:49.131859+0000","last_clean":"2024-03-30T13:09:49.131859+0000","last_became_active":"2024-03-30T13:09:43.113821+0000","last_became_peered":"2024-03-30T13:09:43.113821+0000","last_unstale":"2024-03-30T13:09:49.131859+0000","last_undegraded":"2024-03-30T13:09:49.131859+0000","last_fullsized":"2024-03-30T13:09:49.131859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:04.366501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938219+0000","last_change":"2024-03-30T13:09:42.125753+0000","last_active":"2024-03-30T13:09:49.938219+0000","last_peered":"2024-03-30T13:09:49.938219+0000","last_clean":"2024-03-30T13:09:49.938219+0000","last_became_active":"2024-03-30T13:09:42.125505+0000","last_became_peered":"2024-03-30T13:09:42.125505+0000","last_unstale":"2024-03-30T13:09:49.938219+0000","last_undegraded":"2024-03-30T13:09:49.938219+0000","last_fullsized":"2024-03-30T13:09:49.938219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:43:24.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132292+0000","last_change":"2024-03-30T13:09:47.149806+0000","last_active":"2024-03-30T13:09:49.132292+0000","last_peered":"2024-03-30T13:09:49.132292+0000","last_clean":"2024-03-30T13:09:49.132292+0000","last_became_active":"2024-03-30T13:09:47.149427+0000","last_became_peered":"2024-03-30T13:09:47.149427+0000","last_unstale":"2024-03-30T13:09:49.132292+0000","last_undegraded":"2024-03-30T13:09:49.132292+0000","last_fullsized":"2024-03-30T13:09:49.132292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939301+0000","last_change":"2024-03-30T13:09:43.122135+0000","last_active":"2024-03-30T13:09:49.939301+0000","last_peered":"2024-03-30T13:09:49.939301+0000","last_clean":"2024-03-30T13:09:49.939301+0000","last_became_active":"2024-03-30T13:09:43.121877+0000","last_became_peered":"2024-03-30T13:09:43.121877+0000","last_unstale":"2024-03-30T13:09:49.939301+0000","last_undegraded":"2024-03-30T13:09:49.939301+0000","last_fullsized":"2024-03-30T13:09:49.939301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295852+0000","last_change":"2024-03-30T13:09:42.124452+0000","last_active":"2024-03-30T13:09:49.295852+0000","last_peered":"2024-03-30T13:09:49.295852+0000","last_clean":"2024-03-30T13:09:49.295852+0000","last_became_active":"2024-03-30T13:09:42.124188+0000","last_became_peered":"2024-03-30T13:09:42.124188+0000","last_unstale":"2024-03-30T13:09:49.295852+0000","last_undegraded":"2024-03-30T13:09:49.295852+0000","last_fullsized":"2024-03-30T13:09:49.295852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131805+0000","last_change":"2024-03-30T13:09:47.571007+0000","last_active":"2024-03-30T13:09:49.131805+0000","last_peered":"2024-03-30T13:09:49.131805+0000","last_clean":"2024-03-30T13:09:49.131805+0000","last_became_active":"2024-03-30T13:09:47.570641+0000","last_became_peered":"2024-03-30T13:09:47.570641+0000","last_unstale":"2024-03-30T13:09:49.131805+0000","last_undegraded":"2024-03-30T13:09:49.131805+0000","last_fullsized":"2024-03-30T13:09:49.131805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133129+0000","last_change":"2024-03-30T13:09:43.123324+0000","last_active":"2024-03-30T13:09:49.133129+0000","last_peered":"2024-03-30T13:09:49.133129+0000","last_clean":"2024-03-30T13:09:49.133129+0000","last_became_active":"2024-03-30T13:09:43.122970+0000","last_became_peered":"2024-03-30T13:09:43.122970+0000","last_unstale":"2024-03-30T13:09:49.133129+0000","last_undegraded":"2024-03-30T13:09:49.133129+0000","last_fullsized":"2024-03-30T13:09:49.133129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938610+0000","last_change":"2024-03-30T13:09:42.112467+0000","last_active":"2024-03-30T13:09:49.938610+0000","last_peered":"2024-03-30T13:09:49.938610+0000","last_clean":"2024-03-30T13:09:49.938610+0000","last_became_active":"2024-03-30T13:09:42.112119+0000","last_became_peered":"2024-03-30T13:09:42.112119+0000","last_unstale":"2024-03-30T13:09:49.938610+0000","last_undegraded":"2024-03-30T13:09:49.938610+0000","last_fullsized":"2024-03-30T13:09:49.938610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:45.606451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132205+0000","last_change":"2024-03-30T13:09:47.571907+0000","last_active":"2024-03-30T13:09:49.132205+0000","last_peered":"2024-03-30T13:09:49.132205+0000","last_clean":"2024-03-30T13:09:49.132205+0000","last_became_active":"2024-03-30T13:09:47.571466+0000","last_became_peered":"2024-03-30T13:09:47.571466+0000","last_unstale":"2024-03-30T13:09:49.132205+0000","last_undegraded":"2024-03-30T13:09:49.132205+0000","last_fullsized":"2024-03-30T13:09:49.132205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:36:14.998133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938844+0000","last_change":"2024-03-30T13:09:43.114536+0000","last_active":"2024-03-30T13:09:49.938844+0000","last_peered":"2024-03-30T13:09:49.938844+0000","last_clean":"2024-03-30T13:09:49.938844+0000","last_became_active":"2024-03-30T13:09:43.114268+0000","last_became_peered":"2024-03-30T13:09:43.114268+0000","last_unstale":"2024-03-30T13:09:49.938844+0000","last_undegraded":"2024-03-30T13:09:49.938844+0000","last_fullsized":"2024-03-30T13:09:49.938844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938931+0000","last_change":"2024-03-30T13:09:42.125506+0000","last_active":"2024-03-30T13:09:49.938931+0000","last_peered":"2024-03-30T13:09:49.938931+0000","last_clean":"2024-03-30T13:09:49.938931+0000","last_became_active":"2024-03-30T13:09:42.125297+0000","last_became_peered":"2024-03-30T13:09:42.125297+0000","last_unstale":"2024-03-30T13:09:49.938931+0000","last_undegraded":"2024-03-30T13:09:49.938931+0000","last_fullsized":"2024-03-30T13:09:49.938931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:58.545296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938573+0000","last_change":"2024-03-30T13:09:47.570730+0000","last_active":"2024-03-30T13:09:49.938573+0000","last_peered":"2024-03-30T13:09:49.938573+0000","last_clean":"2024-03-30T13:09:49.938573+0000","last_became_active":"2024-03-30T13:09:47.570239+0000","last_became_peered":"2024-03-30T13:09:47.570239+0000","last_unstale":"2024-03-30T13:09:49.938573+0000","last_undegraded":"2024-03-30T13:09:49.938573+0000","last_fullsized":"2024-03-30T13:09:49.938573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938464+0000","last_change":"2024-03-30T13:09:43.127560+0000","last_active":"2024-03-30T13:09:49.938464+0000","last_peered":"2024-03-30T13:09:49.938464+0000","last_clean":"2024-03-30T13:09:49.938464+0000","last_became_active":"2024-03-30T13:09:43.127245+0000","last_became_peered":"2024-03-30T13:09:43.127245+0000","last_unstale":"2024-03-30T13:09:49.938464+0000","last_undegraded":"2024-03-30T13:09:49.938464+0000","last_fullsized":"2024-03-30T13:09:49.938464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:37.548735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938862+0000","last_change":"2024-03-30T13:09:42.116193+0000","last_active":"2024-03-30T13:09:49.938862+0000","last_peered":"2024-03-30T13:09:49.938862+0000","last_clean":"2024-03-30T13:09:49.938862+0000","last_became_active":"2024-03-30T13:09:42.115609+0000","last_became_peered":"2024-03-30T13:09:42.115609+0000","last_unstale":"2024-03-30T13:09:49.938862+0000","last_undegraded":"2024-03-30T13:09:49.938862+0000","last_fullsized":"2024-03-30T13:09:49.938862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938004+0000","last_change":"2024-03-30T13:09:47.159058+0000","last_active":"2024-03-30T13:09:49.938004+0000","last_peered":"2024-03-30T13:09:49.938004+0000","last_clean":"2024-03-30T13:09:49.938004+0000","last_became_active":"2024-03-30T13:09:47.158646+0000","last_became_peered":"2024-03-30T13:09:47.158646+0000","last_unstale":"2024-03-30T13:09:49.938004+0000","last_undegraded":"2024-03-30T13:09:49.938004+0000","last_fullsized":"2024-03-30T13:09:49.938004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:51.919084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132471+0000","last_change":"2024-03-30T13:09:43.121780+0000","last_active":"2024-03-30T13:09:49.132471+0000","last_peered":"2024-03-30T13:09:49.132471+0000","last_clean":"2024-03-30T13:09:49.132471+0000","last_became_active":"2024-03-30T13:09:43.121519+0000","last_became_peered":"2024-03-30T13:09:43.121519+0000","last_unstale":"2024-03-30T13:09:49.132471+0000","last_undegraded":"2024-03-30T13:09:49.132471+0000","last_fullsized":"2024-03-30T13:09:49.132471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:48.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938327+0000","last_change":"2024-03-30T13:09:42.122015+0000","last_active":"2024-03-30T13:09:49.938327+0000","last_peered":"2024-03-30T13:09:49.938327+0000","last_clean":"2024-03-30T13:09:49.938327+0000","last_became_active":"2024-03-30T13:09:42.121721+0000","last_became_peered":"2024-03-30T13:09:42.121721+0000","last_unstale":"2024-03-30T13:09:49.938327+0000","last_undegraded":"2024-03-30T13:09:49.938327+0000","last_fullsized":"2024-03-30T13:09:49.938327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938675+0000","last_change":"2024-03-30T13:09:47.159041+0000","last_active":"2024-03-30T13:09:49.938675+0000","last_peered":"2024-03-30T13:09:49.938675+0000","last_clean":"2024-03-30T13:09:49.938675+0000","last_became_active":"2024-03-30T13:09:47.158466+0000","last_became_peered":"2024-03-30T13:09:47.158466+0000","last_unstale":"2024-03-30T13:09:49.938675+0000","last_undegraded":"2024-03-30T13:09:49.938675+0000","last_fullsized":"2024-03-30T13:09:49.938675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938050+0000","last_change":"2024-03-30T13:09:43.111943+0000","last_active":"2024-03-30T13:09:49.938050+0000","last_peered":"2024-03-30T13:09:49.938050+0000","last_clean":"2024-03-30T13:09:49.938050+0000","last_became_active":"2024-03-30T13:09:43.111677+0000","last_became_peered":"2024-03-30T13:09:43.111677+0000","last_unstale":"2024-03-30T13:09:49.938050+0000","last_undegraded":"2024-03-30T13:09:49.938050+0000","last_fullsized":"2024-03-30T13:09:49.938050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:33:22.810971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821050+0000","last_change":"2024-03-30T13:09:42.121133+0000","last_active":"2024-03-30T13:09:49.821050+0000","last_peered":"2024-03-30T13:09:49.821050+0000","last_clean":"2024-03-30T13:09:49.821050+0000","last_became_active":"2024-03-30T13:09:42.120851+0000","last_became_peered":"2024-03-30T13:09:42.120851+0000","last_unstale":"2024-03-30T13:09:49.821050+0000","last_undegraded":"2024-03-30T13:09:49.821050+0000","last_fullsized":"2024-03-30T13:09:49.821050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938890+0000","last_change":"2024-03-30T13:09:47.155307+0000","last_active":"2024-03-30T13:09:49.938890+0000","last_peered":"2024-03-30T13:09:49.938890+0000","last_clean":"2024-03-30T13:09:49.938890+0000","last_became_active":"2024-03-30T13:09:47.154875+0000","last_became_peered":"2024-03-30T13:09:47.154875+0000","last_unstale":"2024-03-30T13:09:49.938890+0000","last_undegraded":"2024-03-30T13:09:49.938890+0000","last_fullsized":"2024-03-30T13:09:49.938890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:17:44.422352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937941+0000","last_change":"2024-03-30T13:09:43.128352+0000","last_active":"2024-03-30T13:09:49.937941+0000","last_peered":"2024-03-30T13:09:49.937941+0000","last_clean":"2024-03-30T13:09:49.937941+0000","last_became_active":"2024-03-30T13:09:43.128073+0000","last_became_peered":"2024-03-30T13:09:43.128073+0000","last_unstale":"2024-03-30T13:09:49.937941+0000","last_undegraded":"2024-03-30T13:09:49.937941+0000","last_fullsized":"2024-03-30T13:09:49.937941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:41.389848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938326+0000","last_change":"2024-03-30T13:09:42.124601+0000","last_active":"2024-03-30T13:09:49.938326+0000","last_peered":"2024-03-30T13:09:49.938326+0000","last_clean":"2024-03-30T13:09:49.938326+0000","last_became_active":"2024-03-30T13:09:42.124335+0000","last_became_peered":"2024-03-30T13:09:42.124335+0000","last_unstale":"2024-03-30T13:09:49.938326+0000","last_undegraded":"2024-03-30T13:09:49.938326+0000","last_fullsized":"2024-03-30T13:09:49.938326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:02:06.738441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939206+0000","last_change":"2024-03-30T13:09:47.572608+0000","last_active":"2024-03-30T13:09:49.939206+0000","last_peered":"2024-03-30T13:09:49.939206+0000","last_clean":"2024-03-30T13:09:49.939206+0000","last_became_active":"2024-03-30T13:09:47.571945+0000","last_became_peered":"2024-03-30T13:09:47.571945+0000","last_unstale":"2024-03-30T13:09:49.939206+0000","last_undegraded":"2024-03-30T13:09:49.939206+0000","last_fullsized":"2024-03-30T13:09:49.939206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:06:37.115753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132459+0000","last_change":"2024-03-30T13:09:43.125963+0000","last_active":"2024-03-30T13:09:49.132459+0000","last_peered":"2024-03-30T13:09:49.132459+0000","last_clean":"2024-03-30T13:09:49.132459+0000","last_became_active":"2024-03-30T13:09:43.125682+0000","last_became_peered":"2024-03-30T13:09:43.125682+0000","last_unstale":"2024-03-30T13:09:49.132459+0000","last_undegraded":"2024-03-30T13:09:49.132459+0000","last_fullsized":"2024-03-30T13:09:49.132459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938090+0000","last_change":"2024-03-30T13:09:42.119717+0000","last_active":"2024-03-30T13:09:49.938090+0000","last_peered":"2024-03-30T13:09:49.938090+0000","last_clean":"2024-03-30T13:09:49.938090+0000","last_became_active":"2024-03-30T13:09:42.119185+0000","last_became_peered":"2024-03-30T13:09:42.119185+0000","last_unstale":"2024-03-30T13:09:49.938090+0000","last_undegraded":"2024-03-30T13:09:49.938090+0000","last_fullsized":"2024-03-30T13:09:49.938090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938410+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:09:49.938410+0000","last_peered":"2024-03-30T13:09:49.938410+0000","last_clean":"2024-03-30T13:09:49.938410+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:09:49.938410+0000","last_undegraded":"2024-03-30T13:09:49.938410+0000","last_fullsized":"2024-03-30T13:09:49.938410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295610+0000","last_change":"2024-03-30T13:09:47.572345+0000","last_active":"2024-03-30T13:09:49.295610+0000","last_peered":"2024-03-30T13:09:49.295610+0000","last_clean":"2024-03-30T13:09:49.295610+0000","last_became_active":"2024-03-30T13:09:47.571956+0000","last_became_peered":"2024-03-30T13:09:47.571956+0000","last_unstale":"2024-03-30T13:09:49.295610+0000","last_undegraded":"2024-03-30T13:09:49.295610+0000","last_fullsized":"2024-03-30T13:09:49.295610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938718+0000","last_change":"2024-03-30T13:09:43.134581+0000","last_active":"2024-03-30T13:09:49.938718+0000","last_peered":"2024-03-30T13:09:49.938718+0000","last_clean":"2024-03-30T13:09:49.938718+0000","last_became_active":"2024-03-30T13:09:43.131462+0000","last_became_peered":"2024-03-30T13:09:43.131462+0000","last_unstale":"2024-03-30T13:09:49.938718+0000","last_undegraded":"2024-03-30T13:09:49.938718+0000","last_fullsized":"2024-03-30T13:09:49.938718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:49:52.247198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821188+0000","last_change":"2024-03-30T13:09:42.119614+0000","last_active":"2024-03-30T13:09:49.821188+0000","last_peered":"2024-03-30T13:09:49.821188+0000","last_clean":"2024-03-30T13:09:49.821188+0000","last_became_active":"2024-03-30T13:09:42.119252+0000","last_became_peered":"2024-03-30T13:09:42.119252+0000","last_unstale":"2024-03-30T13:09:49.821188+0000","last_undegraded":"2024-03-30T13:09:49.821188+0000","last_fullsized":"2024-03-30T13:09:49.821188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295320+0000","last_change":"2024-03-30T13:09:47.571383+0000","last_active":"2024-03-30T13:09:49.295320+0000","last_peered":"2024-03-30T13:09:49.295320+0000","last_clean":"2024-03-30T13:09:49.295320+0000","last_became_active":"2024-03-30T13:09:47.570998+0000","last_became_peered":"2024-03-30T13:09:47.570998+0000","last_unstale":"2024-03-30T13:09:49.295320+0000","last_undegraded":"2024-03-30T13:09:49.295320+0000","last_fullsized":"2024-03-30T13:09:49.295320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938152+0000","last_change":"2024-03-30T13:09:43.127405+0000","last_active":"2024-03-30T13:09:49.938152+0000","last_peered":"2024-03-30T13:09:49.938152+0000","last_clean":"2024-03-30T13:09:49.938152+0000","last_became_active":"2024-03-30T13:09:43.122914+0000","last_became_peered":"2024-03-30T13:09:43.122914+0000","last_unstale":"2024-03-30T13:09:49.938152+0000","last_undegraded":"2024-03-30T13:09:49.938152+0000","last_fullsized":"2024-03-30T13:09:49.938152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:51.506154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938749+0000","last_change":"2024-03-30T13:09:42.113530+0000","last_active":"2024-03-30T13:09:49.938749+0000","last_peered":"2024-03-30T13:09:49.938749+0000","last_clean":"2024-03-30T13:09:49.938749+0000","last_became_active":"2024-03-30T13:09:42.113114+0000","last_became_peered":"2024-03-30T13:09:42.113114+0000","last_unstale":"2024-03-30T13:09:49.938749+0000","last_undegraded":"2024-03-30T13:09:49.938749+0000","last_fullsized":"2024-03-30T13:09:49.938749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:59:09.359720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938814+0000","last_change":"2024-03-30T13:09:47.156241+0000","last_active":"2024-03-30T13:09:49.938814+0000","last_peered":"2024-03-30T13:09:49.938814+0000","last_clean":"2024-03-30T13:09:49.938814+0000","last_became_active":"2024-03-30T13:09:47.155884+0000","last_became_peered":"2024-03-30T13:09:47.155884+0000","last_unstale":"2024-03-30T13:09:49.938814+0000","last_undegraded":"2024-03-30T13:09:49.938814+0000","last_fullsized":"2024-03-30T13:09:49.938814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939042+0000","last_change":"2024-03-30T13:09:43.126029+0000","last_active":"2024-03-30T13:09:49.939042+0000","last_peered":"2024-03-30T13:09:49.939042+0000","last_clean":"2024-03-30T13:09:49.939042+0000","last_became_active":"2024-03-30T13:09:43.125724+0000","last_became_peered":"2024-03-30T13:09:43.125724+0000","last_unstale":"2024-03-30T13:09:49.939042+0000","last_undegraded":"2024-03-30T13:09:49.939042+0000","last_fullsized":"2024-03-30T13:09:49.939042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:44.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821402+0000","last_change":"2024-03-30T13:09:42.125370+0000","last_active":"2024-03-30T13:09:49.821402+0000","last_peered":"2024-03-30T13:09:49.821402+0000","last_clean":"2024-03-30T13:09:49.821402+0000","last_became_active":"2024-03-30T13:09:42.124450+0000","last_became_peered":"2024-03-30T13:09:42.124450+0000","last_unstale":"2024-03-30T13:09:49.821402+0000","last_undegraded":"2024-03-30T13:09:49.821402+0000","last_fullsized":"2024-03-30T13:09:49.821402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937709+0000","last_change":"2024-03-30T13:09:47.571108+0000","last_active":"2024-03-30T13:09:49.937709+0000","last_peered":"2024-03-30T13:09:49.937709+0000","last_clean":"2024-03-30T13:09:49.937709+0000","last_became_active":"2024-03-30T13:09:47.570725+0000","last_became_peered":"2024-03-30T13:09:47.570725+0000","last_unstale":"2024-03-30T13:09:49.937709+0000","last_undegraded":"2024-03-30T13:09:49.937709+0000","last_fullsized":"2024-03-30T13:09:49.937709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938937+0000","last_change":"2024-03-30T13:09:43.125121+0000","last_active":"2024-03-30T13:09:49.938937+0000","last_peered":"2024-03-30T13:09:49.938937+0000","last_clean":"2024-03-30T13:09:49.938937+0000","last_became_active":"2024-03-30T13:09:43.124795+0000","last_became_peered":"2024-03-30T13:09:43.124795+0000","last_unstale":"2024-03-30T13:09:49.938937+0000","last_undegraded":"2024-03-30T13:09:49.938937+0000","last_fullsized":"2024-03-30T13:09:49.938937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:55.220836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132420+0000","last_change":"2024-03-30T13:09:42.118818+0000","last_active":"2024-03-30T13:09:49.132420+0000","last_peered":"2024-03-30T13:09:49.132420+0000","last_clean":"2024-03-30T13:09:49.132420+0000","last_became_active":"2024-03-30T13:09:42.118529+0000","last_became_peered":"2024-03-30T13:09:42.118529+0000","last_unstale":"2024-03-30T13:09:49.132420+0000","last_undegraded":"2024-03-30T13:09:49.132420+0000","last_fullsized":"2024-03-30T13:09:49.132420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:40:03.615310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131582+0000","last_change":"2024-03-30T13:09:47.149385+0000","last_active":"2024-03-30T13:09:49.131582+0000","last_peered":"2024-03-30T13:09:49.131582+0000","last_clean":"2024-03-30T13:09:49.131582+0000","last_became_active":"2024-03-30T13:09:47.149030+0000","last_became_peered":"2024-03-30T13:09:47.149030+0000","last_unstale":"2024-03-30T13:09:49.131582+0000","last_undegraded":"2024-03-30T13:09:49.131582+0000","last_fullsized":"2024-03-30T13:09:49.131582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.118404+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.117853+0000","last_became_peered":"2024-03-30T13:09:43.117853+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820864+0000","last_change":"2024-03-30T13:09:42.117594+0000","last_active":"2024-03-30T13:09:49.820864+0000","last_peered":"2024-03-30T13:09:49.820864+0000","last_clean":"2024-03-30T13:09:49.820864+0000","last_became_active":"2024-03-30T13:09:42.117276+0000","last_became_peered":"2024-03-30T13:09:42.117276+0000","last_unstale":"2024-03-30T13:09:49.820864+0000","last_undegraded":"2024-03-30T13:09:49.820864+0000","last_fullsized":"2024-03-30T13:09:49.820864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939158+0000","last_change":"2024-03-30T13:09:47.155090+0000","last_active":"2024-03-30T13:09:49.939158+0000","last_peered":"2024-03-30T13:09:49.939158+0000","last_clean":"2024-03-30T13:09:49.939158+0000","last_became_active":"2024-03-30T13:09:47.154070+0000","last_became_peered":"2024-03-30T13:09:47.154070+0000","last_unstale":"2024-03-30T13:09:49.939158+0000","last_undegraded":"2024-03-30T13:09:49.939158+0000","last_fullsized":"2024-03-30T13:09:49.939158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295287+0000","last_change":"2024-03-30T13:09:43.127827+0000","last_active":"2024-03-30T13:09:49.295287+0000","last_peered":"2024-03-30T13:09:49.295287+0000","last_clean":"2024-03-30T13:09:49.295287+0000","last_became_active":"2024-03-30T13:09:43.127217+0000","last_became_peered":"2024-03-30T13:09:43.127217+0000","last_unstale":"2024-03-30T13:09:49.295287+0000","last_undegraded":"2024-03-30T13:09:49.295287+0000","last_fullsized":"2024-03-30T13:09:49.295287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295274+0000","last_change":"2024-03-30T13:09:42.112507+0000","last_active":"2024-03-30T13:09:49.295274+0000","last_peered":"2024-03-30T13:09:49.295274+0000","last_clean":"2024-03-30T13:09:49.295274+0000","last_became_active":"2024-03-30T13:09:42.112227+0000","last_became_peered":"2024-03-30T13:09:42.112227+0000","last_unstale":"2024-03-30T13:09:49.295274+0000","last_undegraded":"2024-03-30T13:09:49.295274+0000","last_fullsized":"2024-03-30T13:09:49.295274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133062+0000","last_change":"2024-03-30T13:09:47.572882+0000","last_active":"2024-03-30T13:09:49.133062+0000","last_peered":"2024-03-30T13:09:49.133062+0000","last_clean":"2024-03-30T13:09:49.133062+0000","last_became_active":"2024-03-30T13:09:47.572527+0000","last_became_peered":"2024-03-30T13:09:47.572527+0000","last_unstale":"2024-03-30T13:09:49.133062+0000","last_undegraded":"2024-03-30T13:09:49.133062+0000","last_fullsized":"2024-03-30T13:09:49.133062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:18.356964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938472+0000","last_change":"2024-03-30T13:09:43.125989+0000","last_active":"2024-03-30T13:09:49.938472+0000","last_peered":"2024-03-30T13:09:49.938472+0000","last_clean":"2024-03-30T13:09:49.938472+0000","last_became_active":"2024-03-30T13:09:43.125685+0000","last_became_peered":"2024-03-30T13:09:43.125685+0000","last_unstale":"2024-03-30T13:09:49.938472+0000","last_undegraded":"2024-03-30T13:09:49.938472+0000","last_fullsized":"2024-03-30T13:09:49.938472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:23.535266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938231+0000","last_change":"2024-03-30T13:09:42.122605+0000","last_active":"2024-03-30T13:09:49.938231+0000","last_peered":"2024-03-30T13:09:49.938231+0000","last_clean":"2024-03-30T13:09:49.938231+0000","last_became_active":"2024-03-30T13:09:42.122338+0000","last_became_peered":"2024-03-30T13:09:42.122338+0000","last_unstale":"2024-03-30T13:09:49.938231+0000","last_undegraded":"2024-03-30T13:09:49.938231+0000","last_fullsized":"2024-03-30T13:09:49.938231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133046+0000","last_change":"2024-03-30T13:09:47.571300+0000","last_active":"2024-03-30T13:09:49.133046+0000","last_peered":"2024-03-30T13:09:49.133046+0000","last_clean":"2024-03-30T13:09:49.133046+0000","last_became_active":"2024-03-30T13:09:47.570892+0000","last_became_peered":"2024-03-30T13:09:47.570892+0000","last_unstale":"2024-03-30T13:09:49.133046+0000","last_undegraded":"2024-03-30T13:09:49.133046+0000","last_fullsized":"2024-03-30T13:09:49.133046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:09.284589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938281+0000","last_change":"2024-03-30T13:09:43.125239+0000","last_active":"2024-03-30T13:09:49.938281+0000","last_peered":"2024-03-30T13:09:49.938281+0000","last_clean":"2024-03-30T13:09:49.938281+0000","last_became_active":"2024-03-30T13:09:43.124815+0000","last_became_peered":"2024-03-30T13:09:43.124815+0000","last_unstale":"2024-03-30T13:09:49.938281+0000","last_undegraded":"2024-03-30T13:09:49.938281+0000","last_fullsized":"2024-03-30T13:09:49.938281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939006+0000","last_change":"2024-03-30T13:09:42.113307+0000","last_active":"2024-03-30T13:09:49.939006+0000","last_peered":"2024-03-30T13:09:49.939006+0000","last_clean":"2024-03-30T13:09:49.939006+0000","last_became_active":"2024-03-30T13:09:42.112968+0000","last_became_peered":"2024-03-30T13:09:42.112968+0000","last_unstale":"2024-03-30T13:09:49.939006+0000","last_undegraded":"2024-03-30T13:09:49.939006+0000","last_fullsized":"2024-03-30T13:09:49.939006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938380+0000","last_change":"2024-03-30T13:09:47.572011+0000","last_active":"2024-03-30T13:09:49.938380+0000","last_peered":"2024-03-30T13:09:49.938380+0000","last_clean":"2024-03-30T13:09:49.938380+0000","last_became_active":"2024-03-30T13:09:47.571632+0000","last_became_peered":"2024-03-30T13:09:47.571632+0000","last_unstale":"2024-03-30T13:09:49.938380+0000","last_undegraded":"2024-03-30T13:09:49.938380+0000","last_fullsized":"2024-03-30T13:09:49.938380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938714+0000","last_change":"2024-03-30T13:09:43.129993+0000","last_active":"2024-03-30T13:09:49.938714+0000","last_peered":"2024-03-30T13:09:49.938714+0000","last_clean":"2024-03-30T13:09:49.938714+0000","last_became_active":"2024-03-30T13:09:43.128265+0000","last_became_peered":"2024-03-30T13:09:43.128265+0000","last_unstale":"2024-03-30T13:09:49.938714+0000","last_undegraded":"2024-03-30T13:09:49.938714+0000","last_fullsized":"2024-03-30T13:09:49.938714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:28:32.691360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:52.955617+0000","last_change":"2024-03-30T13:09:42.120865+0000","last_active":"2024-03-30T13:09:52.955617+0000","last_peered":"2024-03-30T13:09:52.955617+0000","last_clean":"2024-03-30T13:09:52.955617+0000","last_became_active":"2024-03-30T13:09:42.120589+0000","last_became_peered":"2024-03-30T13:09:42.120589+0000","last_unstale":"2024-03-30T13:09:52.955617+0000","last_undegraded":"2024-03-30T13:09:52.955617+0000","last_fullsized":"2024-03-30T13:09:52.955617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:03.359370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938224+0000","last_change":"2024-03-30T13:09:47.155432+0000","last_active":"2024-03-30T13:09:49.938224+0000","last_peered":"2024-03-30T13:09:49.938224+0000","last_clean":"2024-03-30T13:09:49.938224+0000","last_became_active":"2024-03-30T13:09:47.155044+0000","last_became_peered":"2024-03-30T13:09:47.155044+0000","last_unstale":"2024-03-30T13:09:49.938224+0000","last_undegraded":"2024-03-30T13:09:49.938224+0000","last_fullsized":"2024-03-30T13:09:49.938224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:16.148467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132299+0000","last_change":"2024-03-30T13:09:43.115819+0000","last_active":"2024-03-30T13:09:49.132299+0000","last_peered":"2024-03-30T13:09:49.132299+0000","last_clean":"2024-03-30T13:09:49.132299+0000","last_became_active":"2024-03-30T13:09:43.115563+0000","last_became_peered":"2024-03-30T13:09:43.115563+0000","last_unstale":"2024-03-30T13:09:49.132299+0000","last_undegraded":"2024-03-30T13:09:49.132299+0000","last_fullsized":"2024-03-30T13:09:49.132299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:59.552427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938506+0000","last_change":"2024-03-30T13:09:42.121667+0000","last_active":"2024-03-30T13:09:49.938506+0000","last_peered":"2024-03-30T13:09:49.938506+0000","last_clean":"2024-03-30T13:09:49.938506+0000","last_became_active":"2024-03-30T13:09:42.121384+0000","last_became_peered":"2024-03-30T13:09:42.121384+0000","last_unstale":"2024-03-30T13:09:49.938506+0000","last_undegraded":"2024-03-30T13:09:49.938506+0000","last_fullsized":"2024-03-30T13:09:49.938506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132454+0000","last_change":"2024-03-30T13:09:47.147513+0000","last_active":"2024-03-30T13:09:49.132454+0000","last_peered":"2024-03-30T13:09:49.132454+0000","last_clean":"2024-03-30T13:09:49.132454+0000","last_became_active":"2024-03-30T13:09:47.147126+0000","last_became_peered":"2024-03-30T13:09:47.147126+0000","last_unstale":"2024-03-30T13:09:49.132454+0000","last_undegraded":"2024-03-30T13:09:49.132454+0000","last_fullsized":"2024-03-30T13:09:49.132454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939064+0000","last_change":"2024-03-30T13:09:43.130550+0000","last_active":"2024-03-30T13:09:49.939064+0000","last_peered":"2024-03-30T13:09:49.939064+0000","last_clean":"2024-03-30T13:09:49.939064+0000","last_became_active":"2024-03-30T13:09:43.130297+0000","last_became_peered":"2024-03-30T13:09:43.130297+0000","last_unstale":"2024-03-30T13:09:49.939064+0000","last_undegraded":"2024-03-30T13:09:49.939064+0000","last_fullsized":"2024-03-30T13:09:49.939064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938180+0000","last_change":"2024-03-30T13:09:42.122699+0000","last_active":"2024-03-30T13:09:49.938180+0000","last_peered":"2024-03-30T13:09:49.938180+0000","last_clean":"2024-03-30T13:09:49.938180+0000","last_became_active":"2024-03-30T13:09:42.122407+0000","last_became_peered":"2024-03-30T13:09:42.122407+0000","last_unstale":"2024-03-30T13:09:49.938180+0000","last_undegraded":"2024-03-30T13:09:49.938180+0000","last_fullsized":"2024-03-30T13:09:49.938180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132389+0000","last_change":"2024-03-30T13:09:47.160317+0000","last_active":"2024-03-30T13:09:49.132389+0000","last_peered":"2024-03-30T13:09:49.132389+0000","last_clean":"2024-03-30T13:09:49.132389+0000","last_became_active":"2024-03-30T13:09:47.159818+0000","last_became_peered":"2024-03-30T13:09:47.159818+0000","last_unstale":"2024-03-30T13:09:49.132389+0000","last_undegraded":"2024-03-30T13:09:49.132389+0000","last_fullsized":"2024-03-30T13:09:49.132389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:22:15.869872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938158+0000","last_change":"2024-03-30T13:09:43.128528+0000","last_active":"2024-03-30T13:09:49.938158+0000","last_peered":"2024-03-30T13:09:49.938158+0000","last_clean":"2024-03-30T13:09:49.938158+0000","last_became_active":"2024-03-30T13:09:43.128239+0000","last_became_peered":"2024-03-30T13:09:43.128239+0000","last_unstale":"2024-03-30T13:09:49.938158+0000","last_undegraded":"2024-03-30T13:09:49.938158+0000","last_fullsized":"2024-03-30T13:09:49.938158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:38:56.141275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295744+0000","last_change":"2024-03-30T13:09:42.118540+0000","last_active":"2024-03-30T13:09:49.295744+0000","last_peered":"2024-03-30T13:09:49.295744+0000","last_clean":"2024-03-30T13:09:49.295744+0000","last_became_active":"2024-03-30T13:09:42.118090+0000","last_became_peered":"2024-03-30T13:09:42.118090+0000","last_unstale":"2024-03-30T13:09:49.295744+0000","last_undegraded":"2024-03-30T13:09:49.295744+0000","last_fullsized":"2024-03-30T13:09:49.295744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132435+0000","last_change":"2024-03-30T13:09:47.571367+0000","last_active":"2024-03-30T13:09:49.132435+0000","last_peered":"2024-03-30T13:09:49.132435+0000","last_clean":"2024-03-30T13:09:49.132435+0000","last_became_active":"2024-03-30T13:09:47.570973+0000","last_became_peered":"2024-03-30T13:09:47.570973+0000","last_unstale":"2024-03-30T13:09:49.132435+0000","last_undegraded":"2024-03-30T13:09:49.132435+0000","last_fullsized":"2024-03-30T13:09:49.132435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:33:15.965915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939033+0000","last_change":"2024-03-30T13:09:47.572196+0000","last_active":"2024-03-30T13:09:49.939033+0000","last_peered":"2024-03-30T13:09:49.939033+0000","last_clean":"2024-03-30T13:09:49.939033+0000","last_became_active":"2024-03-30T13:09:47.571378+0000","last_became_peered":"2024-03-30T13:09:47.571378+0000","last_unstale":"2024-03-30T13:09:49.939033+0000","last_undegraded":"2024-03-30T13:09:49.939033+0000","last_fullsized":"2024-03-30T13:09:49.939033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938039+0000","last_change":"2024-03-30T13:09:43.125653+0000","last_active":"2024-03-30T13:09:49.938039+0000","last_peered":"2024-03-30T13:09:49.938039+0000","last_clean":"2024-03-30T13:09:49.938039+0000","last_became_active":"2024-03-30T13:09:43.125382+0000","last_became_peered":"2024-03-30T13:09:43.125382+0000","last_unstale":"2024-03-30T13:09:49.938039+0000","last_undegraded":"2024-03-30T13:09:49.938039+0000","last_fullsized":"2024-03-30T13:09:49.938039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:25:10.034049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937816+0000","last_change":"2024-03-30T13:09:42.115855+0000","last_active":"2024-03-30T13:09:49.937816+0000","last_peered":"2024-03-30T13:09:49.937816+0000","last_clean":"2024-03-30T13:09:49.937816+0000","last_became_active":"2024-03-30T13:09:42.115369+0000","last_became_peered":"2024-03-30T13:09:42.115369+0000","last_unstale":"2024-03-30T13:09:49.937816+0000","last_undegraded":"2024-03-30T13:09:49.937816+0000","last_fullsized":"2024-03-30T13:09:49.937816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295715+0000","last_change":"2024-03-30T13:09:47.156344+0000","last_active":"2024-03-30T13:09:49.295715+0000","last_peered":"2024-03-30T13:09:49.295715+0000","last_clean":"2024-03-30T13:09:49.295715+0000","last_became_active":"2024-03-30T13:09:47.155917+0000","last_became_peered":"2024-03-30T13:09:47.155917+0000","last_unstale":"2024-03-30T13:09:49.295715+0000","last_undegraded":"2024-03-30T13:09:49.295715+0000","last_fullsized":"2024-03-30T13:09:49.295715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132295+0000","last_change":"2024-03-30T13:09:43.128241+0000","last_active":"2024-03-30T13:09:49.132295+0000","last_peered":"2024-03-30T13:09:49.132295+0000","last_clean":"2024-03-30T13:09:49.132295+0000","last_became_active":"2024-03-30T13:09:43.127971+0000","last_became_peered":"2024-03-30T13:09:43.127971+0000","last_unstale":"2024-03-30T13:09:49.132295+0000","last_undegraded":"2024-03-30T13:09:49.132295+0000","last_fullsized":"2024-03-30T13:09:49.132295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132655+0000","last_change":"2024-03-30T13:09:42.122509+0000","last_active":"2024-03-30T13:09:49.132655+0000","last_peered":"2024-03-30T13:09:49.132655+0000","last_clean":"2024-03-30T13:09:49.132655+0000","last_became_active":"2024-03-30T13:09:42.122130+0000","last_became_peered":"2024-03-30T13:09:42.122130+0000","last_unstale":"2024-03-30T13:09:49.132655+0000","last_undegraded":"2024-03-30T13:09:49.132655+0000","last_fullsized":"2024-03-30T13:09:49.132655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295373+0000","last_change":"2024-03-30T13:09:42.115453+0000","last_active":"2024-03-30T13:09:49.295373+0000","last_peered":"2024-03-30T13:09:49.295373+0000","last_clean":"2024-03-30T13:09:49.295373+0000","last_became_active":"2024-03-30T13:09:42.115182+0000","last_became_peered":"2024-03-30T13:09:42.115182+0000","last_unstale":"2024-03-30T13:09:49.295373+0000","last_undegraded":"2024-03-30T13:09:49.295373+0000","last_fullsized":"2024-03-30T13:09:49.295373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:47:12.895083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938966+0000","last_change":"2024-03-30T13:09:43.127679+0000","last_active":"2024-03-30T13:09:49.938966+0000","last_peered":"2024-03-30T13:09:49.938966+0000","last_clean":"2024-03-30T13:09:49.938966+0000","last_became_active":"2024-03-30T13:09:43.127421+0000","last_became_peered":"2024-03-30T13:09:43.127421+0000","last_unstale":"2024-03-30T13:09:49.938966+0000","last_undegraded":"2024-03-30T13:09:49.938966+0000","last_fullsized":"2024-03-30T13:09:49.938966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821070+0000","last_change":"2024-03-30T13:09:47.154699+0000","last_active":"2024-03-30T13:09:49.821070+0000","last_peered":"2024-03-30T13:09:49.821070+0000","last_clean":"2024-03-30T13:09:49.821070+0000","last_became_active":"2024-03-30T13:09:47.154291+0000","last_became_peered":"2024-03-30T13:09:47.154291+0000","last_unstale":"2024-03-30T13:09:49.821070+0000","last_undegraded":"2024-03-30T13:09:49.821070+0000","last_fullsized":"2024-03-30T13:09:49.821070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:56:47.829620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133085+0000","last_change":"2024-03-30T13:09:42.124644+0000","last_active":"2024-03-30T13:09:49.133085+0000","last_peered":"2024-03-30T13:09:49.133085+0000","last_clean":"2024-03-30T13:09:49.133085+0000","last_became_active":"2024-03-30T13:09:42.124355+0000","last_became_peered":"2024-03-30T13:09:42.124355+0000","last_unstale":"2024-03-30T13:09:49.133085+0000","last_undegraded":"2024-03-30T13:09:49.133085+0000","last_fullsized":"2024-03-30T13:09:49.133085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132040+0000","last_change":"2024-03-30T13:09:43.113299+0000","last_active":"2024-03-30T13:09:49.132040+0000","last_peered":"2024-03-30T13:09:49.132040+0000","last_clean":"2024-03-30T13:09:49.132040+0000","last_became_active":"2024-03-30T13:09:43.113031+0000","last_became_peered":"2024-03-30T13:09:43.113031+0000","last_unstale":"2024-03-30T13:09:49.132040+0000","last_undegraded":"2024-03-30T13:09:49.132040+0000","last_fullsized":"2024-03-30T13:09:49.132040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:32.715713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132575+0000","last_change":"2024-03-30T13:09:47.156912+0000","last_active":"2024-03-30T13:09:49.132575+0000","last_peered":"2024-03-30T13:09:49.132575+0000","last_clean":"2024-03-30T13:09:49.132575+0000","last_became_active":"2024-03-30T13:09:47.156492+0000","last_became_peered":"2024-03-30T13:09:47.156492+0000","last_unstale":"2024-03-30T13:09:49.132575+0000","last_undegraded":"2024-03-30T13:09:49.132575+0000","last_fullsized":"2024-03-30T13:09:49.132575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:55:36.886194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295186+0000","last_change":"2024-03-30T13:09:43.127904+0000","last_active":"2024-03-30T13:09:49.295186+0000","last_peered":"2024-03-30T13:09:49.295186+0000","last_clean":"2024-03-30T13:09:49.295186+0000","last_became_active":"2024-03-30T13:09:43.127563+0000","last_became_peered":"2024-03-30T13:09:43.127563+0000","last_unstale":"2024-03-30T13:09:49.295186+0000","last_undegraded":"2024-03-30T13:09:49.295186+0000","last_fullsized":"2024-03-30T13:09:49.295186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820892+0000","last_change":"2024-03-30T13:09:43.131561+0000","last_active":"2024-03-30T13:09:49.820892+0000","last_peered":"2024-03-30T13:09:49.820892+0000","last_clean":"2024-03-30T13:09:49.820892+0000","last_became_active":"2024-03-30T13:09:43.129703+0000","last_became_peered":"2024-03-30T13:09:43.129703+0000","last_unstale":"2024-03-30T13:09:49.820892+0000","last_undegraded":"2024-03-30T13:09:49.820892+0000","last_fullsized":"2024-03-30T13:09:49.820892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:46.613583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938131+0000","last_change":"2024-03-30T13:09:43.129678+0000","last_active":"2024-03-30T13:09:49.938131+0000","last_peered":"2024-03-30T13:09:49.938131+0000","last_clean":"2024-03-30T13:09:49.938131+0000","last_became_active":"2024-03-30T13:09:43.129405+0000","last_became_peered":"2024-03-30T13:09:43.129405+0000","last_unstale":"2024-03-30T13:09:49.938131+0000","last_undegraded":"2024-03-30T13:09:49.938131+0000","last_fullsized":"2024-03-30T13:09:49.938131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:13:29.545084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938913+0000","last_change":"2024-03-30T13:09:43.135202+0000","last_active":"2024-03-30T13:09:49.938913+0000","last_peered":"2024-03-30T13:09:49.938913+0000","last_clean":"2024-03-30T13:09:49.938913+0000","last_became_active":"2024-03-30T13:09:43.134895+0000","last_became_peered":"2024-03-30T13:09:43.134895+0000","last_unstale":"2024-03-30T13:09:49.938913+0000","last_undegraded":"2024-03-30T13:09:49.938913+0000","last_fullsized":"2024-03-30T13:09:49.938913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133173+0000","last_change":"2024-03-30T13:09:43.127414+0000","last_active":"2024-03-30T13:09:49.133173+0000","last_peered":"2024-03-30T13:09:49.133173+0000","last_clean":"2024-03-30T13:09:49.133173+0000","last_became_active":"2024-03-30T13:09:43.127077+0000","last_became_peered":"2024-03-30T13:09:43.127077+0000","last_unstale":"2024-03-30T13:09:49.133173+0000","last_undegraded":"2024-03-30T13:09:49.133173+0000","last_fullsized":"2024-03-30T13:09:49.133173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938324+0000","last_change":"2024-03-30T13:09:43.119060+0000","last_active":"2024-03-30T13:09:49.938324+0000","last_peered":"2024-03-30T13:09:49.938324+0000","last_clean":"2024-03-30T13:09:49.938324+0000","last_became_active":"2024-03-30T13:09:43.118585+0000","last_became_peered":"2024-03-30T13:09:43.118585+0000","last_unstale":"2024-03-30T13:09:49.938324+0000","last_undegraded":"2024-03-30T13:09:49.938324+0000","last_fullsized":"2024-03-30T13:09:49.938324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295833+0000","last_change":"2024-03-30T13:09:43.122310+0000","last_active":"2024-03-30T13:09:49.295833+0000","last_peered":"2024-03-30T13:09:49.295833+0000","last_clean":"2024-03-30T13:09:49.295833+0000","last_became_active":"2024-03-30T13:09:43.121999+0000","last_became_peered":"2024-03-30T13:09:43.121999+0000","last_unstale":"2024-03-30T13:09:49.295833+0000","last_undegraded":"2024-03-30T13:09:49.295833+0000","last_fullsized":"2024-03-30T13:09:49.295833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133127+0000","last_change":"2024-03-30T13:09:43.122476+0000","last_active":"2024-03-30T13:09:49.133127+0000","last_peered":"2024-03-30T13:09:49.133127+0000","last_clean":"2024-03-30T13:09:49.133127+0000","last_became_active":"2024-03-30T13:09:43.120315+0000","last_became_peered":"2024-03-30T13:09:43.120315+0000","last_unstale":"2024-03-30T13:09:49.133127+0000","last_undegraded":"2024-03-30T13:09:49.133127+0000","last_fullsized":"2024-03-30T13:09:49.133127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937780+0000","last_change":"2024-03-30T13:09:43.128432+0000","last_active":"2024-03-30T13:09:49.937780+0000","last_peered":"2024-03-30T13:09:49.937780+0000","last_clean":"2024-03-30T13:09:49.937780+0000","last_became_active":"2024-03-30T13:09:43.128013+0000","last_became_peered":"2024-03-30T13:09:43.128013+0000","last_unstale":"2024-03-30T13:09:49.937780+0000","last_undegraded":"2024-03-30T13:09:49.937780+0000","last_fullsized":"2024-03-30T13:09:49.937780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132485+0000","last_change":"2024-03-30T13:09:43.120476+0000","last_active":"2024-03-30T13:09:49.132485+0000","last_peered":"2024-03-30T13:09:49.132485+0000","last_clean":"2024-03-30T13:09:49.132485+0000","last_became_active":"2024-03-30T13:09:43.120212+0000","last_became_peered":"2024-03-30T13:09:43.120212+0000","last_unstale":"2024-03-30T13:09:49.132485+0000","last_undegraded":"2024-03-30T13:09:49.132485+0000","last_fullsized":"2024-03-30T13:09:49.132485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:50:23.461880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938025+0000","last_change":"2024-03-30T13:09:43.137167+0000","last_active":"2024-03-30T13:09:49.938025+0000","last_peered":"2024-03-30T13:09:49.938025+0000","last_clean":"2024-03-30T13:09:49.938025+0000","last_became_active":"2024-03-30T13:09:43.136900+0000","last_became_peered":"2024-03-30T13:09:43.136900+0000","last_unstale":"2024-03-30T13:09:49.938025+0000","last_undegraded":"2024-03-30T13:09:49.938025+0000","last_fullsized":"2024-03-30T13:09:49.938025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:15:29.671082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124147+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.123860+0000","last_became_peered":"2024-03-30T13:09:43.123860+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:32:55.194498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295535+0000","last_change":"2024-03-30T13:09:43.127624+0000","last_active":"2024-03-30T13:09:49.295535+0000","last_peered":"2024-03-30T13:09:49.295535+0000","last_clean":"2024-03-30T13:09:49.295535+0000","last_became_active":"2024-03-30T13:09:43.127340+0000","last_became_peered":"2024-03-30T13:09:43.127340+0000","last_unstale":"2024-03-30T13:09:49.295535+0000","last_undegraded":"2024-03-30T13:09:49.295535+0000","last_fullsized":"2024-03-30T13:09:49.295535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938857+0000","last_change":"2024-03-30T13:09:43.122985+0000","last_active":"2024-03-30T13:09:49.938857+0000","last_peered":"2024-03-30T13:09:49.938857+0000","last_clean":"2024-03-30T13:09:49.938857+0000","last_became_active":"2024-03-30T13:09:43.122689+0000","last_became_peered":"2024-03-30T13:09:43.122689+0000","last_unstale":"2024-03-30T13:09:49.938857+0000","last_undegraded":"2024-03-30T13:09:49.938857+0000","last_fullsized":"2024-03-30T13:09:49.938857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:41.817012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821341+0000","last_change":"2024-03-30T13:09:43.131062+0000","last_active":"2024-03-30T13:09:49.821341+0000","last_peered":"2024-03-30T13:09:49.821341+0000","last_clean":"2024-03-30T13:09:49.821341+0000","last_became_active":"2024-03-30T13:09:43.130727+0000","last_became_peered":"2024-03-30T13:09:43.130727+0000","last_unstale":"2024-03-30T13:09:49.821341+0000","last_undegraded":"2024-03-30T13:09:49.821341+0000","last_fullsized":"2024-03-30T13:09:49.821341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:03:31.612660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938468+0000","last_change":"2024-03-30T13:09:43.137027+0000","last_active":"2024-03-30T13:09:49.938468+0000","last_peered":"2024-03-30T13:09:49.938468+0000","last_clean":"2024-03-30T13:09:49.938468+0000","last_became_active":"2024-03-30T13:09:43.136755+0000","last_became_peered":"2024-03-30T13:09:43.136755+0000","last_unstale":"2024-03-30T13:09:49.938468+0000","last_undegraded":"2024-03-30T13:09:49.938468+0000","last_fullsized":"2024-03-30T13:09:49.938468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:38:26.087735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937690+0000","last_change":"2024-03-30T13:09:43.132197+0000","last_active":"2024-03-30T13:09:49.937690+0000","last_peered":"2024-03-30T13:09:49.937690+0000","last_clean":"2024-03-30T13:09:49.937690+0000","last_became_active":"2024-03-30T13:09:43.131927+0000","last_became_peered":"2024-03-30T13:09:43.131927+0000","last_unstale":"2024-03-30T13:09:49.937690+0000","last_undegraded":"2024-03-30T13:09:49.937690+0000","last_fullsized":"2024-03-30T13:09:49.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938139+0000","last_change":"2024-03-30T13:09:43.109275+0000","last_active":"2024-03-30T13:09:49.938139+0000","last_peered":"2024-03-30T13:09:49.938139+0000","last_clean":"2024-03-30T13:09:49.938139+0000","last_became_active":"2024-03-30T13:09:43.108964+0000","last_became_peered":"2024-03-30T13:09:43.108964+0000","last_unstale":"2024-03-30T13:09:49.938139+0000","last_undegraded":"2024-03-30T13:09:49.938139+0000","last_fullsized":"2024-03-30T13:09:49.938139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:48.847357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938228+0000","last_change":"2024-03-30T13:09:43.121915+0000","last_active":"2024-03-30T13:09:49.938228+0000","last_peered":"2024-03-30T13:09:49.938228+0000","last_clean":"2024-03-30T13:09:49.938228+0000","last_became_active":"2024-03-30T13:09:43.121625+0000","last_became_peered":"2024-03-30T13:09:43.121625+0000","last_unstale":"2024-03-30T13:09:49.938228+0000","last_undegraded":"2024-03-30T13:09:49.938228+0000","last_fullsized":"2024-03-30T13:09:49.938228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132890+0000","last_change":"2024-03-30T13:09:43.131187+0000","last_active":"2024-03-30T13:09:49.132890+0000","last_peered":"2024-03-30T13:09:49.132890+0000","last_clean":"2024-03-30T13:09:49.132890+0000","last_became_active":"2024-03-30T13:09:43.130682+0000","last_became_peered":"2024-03-30T13:09:43.130682+0000","last_unstale":"2024-03-30T13:09:49.132890+0000","last_undegraded":"2024-03-30T13:09:49.132890+0000","last_fullsized":"2024-03-30T13:09:49.132890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132240+0000","last_change":"2024-03-30T13:09:43.123005+0000","last_active":"2024-03-30T13:09:49.132240+0000","last_peered":"2024-03-30T13:09:49.132240+0000","last_clean":"2024-03-30T13:09:49.132240+0000","last_became_active":"2024-03-30T13:09:43.122761+0000","last_became_peered":"2024-03-30T13:09:43.122761+0000","last_unstale":"2024-03-30T13:09:49.132240+0000","last_undegraded":"2024-03-30T13:09:49.132240+0000","last_fullsized":"2024-03-30T13:09:49.132240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:48.062694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295578+0000","last_change":"2024-03-30T13:09:43.122715+0000","last_active":"2024-03-30T13:09:49.295578+0000","last_peered":"2024-03-30T13:09:49.295578+0000","last_clean":"2024-03-30T13:09:49.295578+0000","last_became_active":"2024-03-30T13:09:43.122445+0000","last_became_peered":"2024-03-30T13:09:43.122445+0000","last_unstale":"2024-03-30T13:09:49.295578+0000","last_undegraded":"2024-03-30T13:09:49.295578+0000","last_fullsized":"2024-03-30T13:09:49.295578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295783+0000","last_change":"2024-03-30T13:09:43.130345+0000","last_active":"2024-03-30T13:09:49.295783+0000","last_peered":"2024-03-30T13:09:49.295783+0000","last_clean":"2024-03-30T13:09:49.295783+0000","last_became_active":"2024-03-30T13:09:43.130062+0000","last_became_peered":"2024-03-30T13:09:43.130062+0000","last_unstale":"2024-03-30T13:09:49.295783+0000","last_undegraded":"2024-03-30T13:09:49.295783+0000","last_fullsized":"2024-03-30T13:09:49.295783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939037+0000","last_change":"2024-03-30T13:09:43.135018+0000","last_active":"2024-03-30T13:09:49.939037+0000","last_peered":"2024-03-30T13:09:49.939037+0000","last_clean":"2024-03-30T13:09:49.939037+0000","last_became_active":"2024-03-30T13:09:43.134354+0000","last_became_peered":"2024-03-30T13:09:43.134354+0000","last_unstale":"2024-03-30T13:09:49.939037+0000","last_undegraded":"2024-03-30T13:09:49.939037+0000","last_fullsized":"2024-03-30T13:09:49.939037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:04.849718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132188+0000","last_change":"2024-03-30T13:09:43.123158+0000","last_active":"2024-03-30T13:09:49.132188+0000","last_peered":"2024-03-30T13:09:49.132188+0000","last_clean":"2024-03-30T13:09:49.132188+0000","last_became_active":"2024-03-30T13:09:43.122830+0000","last_became_peered":"2024-03-30T13:09:43.122830+0000","last_unstale":"2024-03-30T13:09:49.132188+0000","last_undegraded":"2024-03-30T13:09:49.132188+0000","last_fullsized":"2024-03-30T13:09:49.132188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124628+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.124364+0000","last_became_peered":"2024-03-30T13:09:43.124364+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937937+0000","last_change":"2024-03-30T13:09:43.132387+0000","last_active":"2024-03-30T13:09:49.937937+0000","last_peered":"2024-03-30T13:09:49.937937+0000","last_clean":"2024-03-30T13:09:49.937937+0000","last_became_active":"2024-03-30T13:09:43.132110+0000","last_became_peered":"2024-03-30T13:09:43.132110+0000","last_unstale":"2024-03-30T13:09:49.937937+0000","last_undegraded":"2024-03-30T13:09:49.937937+0000","last_fullsized":"2024-03-30T13:09:49.937937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938308+0000","last_change":"2024-03-30T13:09:43.124528+0000","last_active":"2024-03-30T13:09:49.938308+0000","last_peered":"2024-03-30T13:09:49.938308+0000","last_clean":"2024-03-30T13:09:49.938308+0000","last_became_active":"2024-03-30T13:09:43.124023+0000","last_became_peered":"2024-03-30T13:09:43.124023+0000","last_unstale":"2024-03-30T13:09:49.938308+0000","last_undegraded":"2024-03-30T13:09:49.938308+0000","last_fullsized":"2024-03-30T13:09:49.938308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938061+0000","last_change":"2024-03-30T13:09:43.127663+0000","last_active":"2024-03-30T13:09:49.938061+0000","last_peered":"2024-03-30T13:09:49.938061+0000","last_clean":"2024-03-30T13:09:49.938061+0000","last_became_active":"2024-03-30T13:09:43.127180+0000","last_became_peered":"2024-03-30T13:09:43.127180+0000","last_unstale":"2024-03-30T13:09:49.938061+0000","last_undegraded":"2024-03-30T13:09:49.938061+0000","last_fullsized":"2024-03-30T13:09:49.938061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:36:33.418013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.121807+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.121498+0000","last_became_peered":"2024-03-30T13:09:43.121498+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938554+0000","last_change":"2024-03-30T13:09:43.124024+0000","last_active":"2024-03-30T13:09:49.938554+0000","last_peered":"2024-03-30T13:09:49.938554+0000","last_clean":"2024-03-30T13:09:49.938554+0000","last_became_active":"2024-03-30T13:09:43.123748+0000","last_became_peered":"2024-03-30T13:09:43.123748+0000","last_unstale":"2024-03-30T13:09:49.938554+0000","last_undegraded":"2024-03-30T13:09:49.938554+0000","last_fullsized":"2024-03-30T13:09:49.938554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938980+0000","last_change":"2024-03-30T13:09:43.122707+0000","last_active":"2024-03-30T13:09:49.938980+0000","last_peered":"2024-03-30T13:09:49.938980+0000","last_clean":"2024-03-30T13:09:49.938980+0000","last_became_active":"2024-03-30T13:09:43.122418+0000","last_became_peered":"2024-03-30T13:09:43.122418+0000","last_unstale":"2024-03-30T13:09:49.938980+0000","last_undegraded":"2024-03-30T13:09:49.938980+0000","last_fullsized":"2024-03-30T13:09:49.938980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:49.970892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":700230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":109406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":109280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":109440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":109370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:10:10.729 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:10:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.869+0000 7f9b1be93640 1 Processor -- start 2024-03-30T13:10:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.869+0000 7f9b1be93640 1 -- start start 2024-03-30T13:10:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1be93640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b140b7530 0x7f9b140d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1be93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1be93640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b140b9d20 con 0x7f9b1415e4f0 2024-03-30T13:10:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1be93640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9b140d3430 con 0x7f9b140b7530 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1be93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9b140ba000 con 0x7f9b140b79c0 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1942e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b140b7530 0x7f9b140d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b19c2f640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9b1415e4f0 0x7f9b140cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53990/0 (socket says 172.21.15.66:53990) 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b19c2f640 1 -- 172.21.15.66:0/941748481 learned_addr learned my addr 172.21.15.66:0/941748481 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.870+0000 7f9b1a430640 1 --2- 172.21.15.66:0/941748481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.871+0000 7f9b1942e640 1 -- 172.21.15.66:0/941748481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 msgr2=0x7f9b140d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.871+0000 7f9b1942e640 1 --2- 172.21.15.66:0/941748481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.871+0000 7f9b1942e640 1 -- 172.21.15.66:0/941748481 >> v1:172.21.15.66:6789/0 conn(0x7f9b1415e4f0 legacy=0x7f9b140cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.871+0000 7f9b18c2d640 1 -- 172.21.15.66:0/941748481 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794885606 0 0) 0x7f9b140b9d20 con 0x7f9b1415e4f0 2024-03-30T13:10:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.871+0000 7f9b1942e640 1 -- 172.21.15.66:0/941748481 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b140ba2e0 con 0x7f9b140b7530 2024-03-30T13:10:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.871+0000 7f9b1942e640 1 --2- 172.21.15.66:0/941748481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b140d29b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097a0 tx=0x7f9b0802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad36b4af106cf09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b18c2d640 1 -- 172.21.15.66:0/941748481 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080315a0 con 0x7f9b140b7530 2024-03-30T13:10:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b18c2d640 1 -- 172.21.15.66:0/941748481 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b08037070 con 0x7f9b140b7530 2024-03-30T13:10:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b18c2d640 1 -- 172.21.15.66:0/941748481 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08031960 con 0x7f9b140b7530 2024-03-30T13:10:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 -- 172.21.15.66:0/941748481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 msgr2=0x7f9b140d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 --2- 172.21.15.66:0/941748481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b140d29b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b080097a0 tx=0x7f9b0802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 -- 172.21.15.66:0/941748481 shutdown_connections 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 --2- 172.21.15.66:0/941748481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 --2- 172.21.15.66:0/941748481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b140d29b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 -- 172.21.15.66:0/941748481 >> 172.21.15.66:0/941748481 conn(0x7f9b140676e0 msgr2=0x7f9b14067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 -- 172.21.15.66:0/941748481 shutdown_connections 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.872+0000 7f9b1be93640 1 -- 172.21.15.66:0/941748481 wait complete. 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.873+0000 7f9b1be93640 1 Processor -- start 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.873+0000 7f9b1be93640 1 -- start start 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1be93640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b141f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1be93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1be93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b1415e4f0 0x7f9b140cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1be93640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1415ea50 con 0x7f9b140b7530 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1be93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1415e8d0 con 0x7f9b1415e4f0 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1be93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9b140d3d00 con 0x7f9b140b79c0 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b141f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b141f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34538/0 (socket says 172.21.15.66:34538) 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1942e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 -- 172.21.15.66:0/265108743 learned_addr learned my addr 172.21.15.66:0/265108743 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b1a430640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b1415e4f0 0x7f9b140cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 -- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 msgr2=0x7f9b140b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 -- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b1415e4f0 msgr2=0x7f9b140cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.874+0000 7f9b19c2f640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b1415e4f0 0x7f9b140cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b19c2f640 1 -- 172.21.15.66:0/265108743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b140b5d00 con 0x7f9b140b7530 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b1a430640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b1415e4f0 0x7f9b140cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b19c2f640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b141f6ca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c400 tx=0x7f9b0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c017070 con 0x7f9b140b7530 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0c005ce0 con 0x7f9b140b7530 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b140b5fe0 con 0x7f9b140b7530 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c007310 con 0x7f9b140b7530 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.875+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b140b6880 con 0x7f9b140b7530 2024-03-30T13:10:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.876+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b14075a10 con 0x7f9b140b7530 2024-03-30T13:10:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.876+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b0c01a070 con 0x7f9b140b7530 2024-03-30T13:10:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.877+0000 7f9b127fc640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b0006c530 0x7f9b0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.877+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9b0c08ad80 con 0x7f9b140b7530 2024-03-30T13:10:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.877+0000 7f9b1942e640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b0006c530 0x7f9b0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.877+0000 7f9b1942e640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b0006c530 0x7f9b0006e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b08036000 tx=0x7f9b08008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:10.881+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b0c058a10 con 0x7f9b140b7530 2024-03-30T13:10:11.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.061+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b140ce810 con 0x7f9b0006c530 2024-03-30T13:10:11.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.068+0000 7f9b127fc640 1 -- 172.21.15.66:0/265108743 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f9b140ce810 con 0x7f9b0006c530 2024-03-30T13:10:11.077 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:11.079 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:10:11.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b0006c530 msgr2=0x7f9b0006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b0006c530 0x7f9b0006e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b08036000 tx=0x7f9b08008030 comp rx=0 tx=0).stop 2024-03-30T13:10:11.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 msgr2=0x7f9b141f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b141f6ca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c400 tx=0x7f9b0c00c8c0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 shutdown_connections 2024-03-30T13:10:11.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9b0006c530 0x7f9b0006e9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b1415e4f0 0x7f9b140cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b140b79c0 0x7f9b140b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 --2- 172.21.15.66:0/265108743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b140b7530 0x7f9b141f6ca0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 >> 172.21.15.66:0/265108743 conn(0x7f9b140676e0 msgr2=0x7f9b140cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:11.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 shutdown_connections 2024-03-30T13:10:11.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.072+0000 7f9b1be93640 1 -- 172.21.15.66:0/265108743 wait complete. 2024-03-30T13:10:11.086 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-03-30T13:10:10.799779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219500,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755220,"statfs":{"total":773094113280,"available":772869345280,"internally_reserved":0,"allocated":3416064,"data_stored":2064256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939160+0000","last_change":"2024-03-30T13:09:42.124458+0000","last_active":"2024-03-30T13:09:49.939160+0000","last_peered":"2024-03-30T13:09:49.939160+0000","last_clean":"2024-03-30T13:09:49.939160+0000","last_became_active":"2024-03-30T13:09:42.124158+0000","last_became_peered":"2024-03-30T13:09:42.124158+0000","last_unstale":"2024-03-30T13:09:49.939160+0000","last_undegraded":"2024-03-30T13:09:49.939160+0000","last_fullsized":"2024-03-30T13:09:49.939160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132823+0000","last_change":"2024-03-30T13:09:43.127433+0000","last_active":"2024-03-30T13:09:49.132823+0000","last_peered":"2024-03-30T13:09:49.132823+0000","last_clean":"2024-03-30T13:09:49.132823+0000","last_became_active":"2024-03-30T13:09:43.127127+0000","last_became_peered":"2024-03-30T13:09:43.127127+0000","last_unstale":"2024-03-30T13:09:49.132823+0000","last_undegraded":"2024-03-30T13:09:49.132823+0000","last_fullsized":"2024-03-30T13:09:49.132823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132184+0000","last_change":"2024-03-30T13:09:47.572430+0000","last_active":"2024-03-30T13:09:49.132184+0000","last_peered":"2024-03-30T13:09:49.132184+0000","last_clean":"2024-03-30T13:09:49.132184+0000","last_became_active":"2024-03-30T13:09:47.572059+0000","last_became_peered":"2024-03-30T13:09:47.572059+0000","last_unstale":"2024-03-30T13:09:49.132184+0000","last_undegraded":"2024-03-30T13:09:49.132184+0000","last_fullsized":"2024-03-30T13:09:49.132184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:31:37.218184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132943+0000","last_change":"2024-03-30T13:09:42.126685+0000","last_active":"2024-03-30T13:09:49.132943+0000","last_peered":"2024-03-30T13:09:49.132943+0000","last_clean":"2024-03-30T13:09:49.132943+0000","last_became_active":"2024-03-30T13:09:42.126383+0000","last_became_peered":"2024-03-30T13:09:42.126383+0000","last_unstale":"2024-03-30T13:09:49.132943+0000","last_undegraded":"2024-03-30T13:09:49.132943+0000","last_fullsized":"2024-03-30T13:09:49.132943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:02:07.521481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938945+0000","last_change":"2024-03-30T13:09:43.132452+0000","last_active":"2024-03-30T13:09:49.938945+0000","last_peered":"2024-03-30T13:09:49.938945+0000","last_clean":"2024-03-30T13:09:49.938945+0000","last_became_active":"2024-03-30T13:09:43.132158+0000","last_became_peered":"2024-03-30T13:09:43.132158+0000","last_unstale":"2024-03-30T13:09:49.938945+0000","last_undegraded":"2024-03-30T13:09:49.938945+0000","last_fullsized":"2024-03-30T13:09:49.938945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131666+0000","last_change":"2024-03-30T13:09:47.157090+0000","last_active":"2024-03-30T13:09:49.131666+0000","last_peered":"2024-03-30T13:09:49.131666+0000","last_clean":"2024-03-30T13:09:49.131666+0000","last_became_active":"2024-03-30T13:09:47.156676+0000","last_became_peered":"2024-03-30T13:09:47.156676+0000","last_unstale":"2024-03-30T13:09:49.131666+0000","last_undegraded":"2024-03-30T13:09:49.131666+0000","last_fullsized":"2024-03-30T13:09:49.131666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938826+0000","last_change":"2024-03-30T13:09:42.127138+0000","last_active":"2024-03-30T13:09:49.938826+0000","last_peered":"2024-03-30T13:09:49.938826+0000","last_clean":"2024-03-30T13:09:49.938826+0000","last_became_active":"2024-03-30T13:09:42.126853+0000","last_became_peered":"2024-03-30T13:09:42.126853+0000","last_unstale":"2024-03-30T13:09:49.938826+0000","last_undegraded":"2024-03-30T13:09:49.938826+0000","last_fullsized":"2024-03-30T13:09:49.938826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938126+0000","last_change":"2024-03-30T13:09:43.137336+0000","last_active":"2024-03-30T13:09:49.938126+0000","last_peered":"2024-03-30T13:09:49.938126+0000","last_clean":"2024-03-30T13:09:49.938126+0000","last_became_active":"2024-03-30T13:09:43.137057+0000","last_became_peered":"2024-03-30T13:09:43.137057+0000","last_unstale":"2024-03-30T13:09:49.938126+0000","last_undegraded":"2024-03-30T13:09:49.938126+0000","last_fullsized":"2024-03-30T13:09:49.938126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:20.284742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132127+0000","last_change":"2024-03-30T13:09:47.149659+0000","last_active":"2024-03-30T13:09:49.132127+0000","last_peered":"2024-03-30T13:09:49.132127+0000","last_clean":"2024-03-30T13:09:49.132127+0000","last_became_active":"2024-03-30T13:09:47.149298+0000","last_became_peered":"2024-03-30T13:09:47.149298+0000","last_unstale":"2024-03-30T13:09:49.132127+0000","last_undegraded":"2024-03-30T13:09:49.132127+0000","last_fullsized":"2024-03-30T13:09:49.132127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:48.863738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938057+0000","last_change":"2024-03-30T13:09:42.126941+0000","last_active":"2024-03-30T13:09:49.938057+0000","last_peered":"2024-03-30T13:09:49.938057+0000","last_clean":"2024-03-30T13:09:49.938057+0000","last_became_active":"2024-03-30T13:09:42.126646+0000","last_became_peered":"2024-03-30T13:09:42.126646+0000","last_unstale":"2024-03-30T13:09:49.938057+0000","last_undegraded":"2024-03-30T13:09:49.938057+0000","last_fullsized":"2024-03-30T13:09:49.938057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938364+0000","last_change":"2024-03-30T13:09:43.129721+0000","last_active":"2024-03-30T13:09:49.938364+0000","last_peered":"2024-03-30T13:09:49.938364+0000","last_clean":"2024-03-30T13:09:49.938364+0000","last_became_active":"2024-03-30T13:09:43.128999+0000","last_became_peered":"2024-03-30T13:09:43.128999+0000","last_unstale":"2024-03-30T13:09:49.938364+0000","last_undegraded":"2024-03-30T13:09:49.938364+0000","last_fullsized":"2024-03-30T13:09:49.938364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:28:48.670825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131877+0000","last_change":"2024-03-30T13:09:47.149841+0000","last_active":"2024-03-30T13:09:49.131877+0000","last_peered":"2024-03-30T13:09:49.131877+0000","last_clean":"2024-03-30T13:09:49.131877+0000","last_became_active":"2024-03-30T13:09:47.148965+0000","last_became_peered":"2024-03-30T13:09:47.148965+0000","last_unstale":"2024-03-30T13:09:49.131877+0000","last_undegraded":"2024-03-30T13:09:49.131877+0000","last_fullsized":"2024-03-30T13:09:49.131877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:08.872490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295198+0000","last_change":"2024-03-30T13:09:43.127058+0000","last_active":"2024-03-30T13:09:49.295198+0000","last_peered":"2024-03-30T13:09:49.295198+0000","last_clean":"2024-03-30T13:09:49.295198+0000","last_became_active":"2024-03-30T13:09:43.126777+0000","last_became_peered":"2024-03-30T13:09:43.126777+0000","last_unstale":"2024-03-30T13:09:49.295198+0000","last_undegraded":"2024-03-30T13:09:49.295198+0000","last_fullsized":"2024-03-30T13:09:49.295198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938548+0000","last_change":"2024-03-30T13:09:42.124505+0000","last_active":"2024-03-30T13:09:49.938548+0000","last_peered":"2024-03-30T13:09:49.938548+0000","last_clean":"2024-03-30T13:09:49.938548+0000","last_became_active":"2024-03-30T13:09:42.124243+0000","last_became_peered":"2024-03-30T13:09:42.124243+0000","last_unstale":"2024-03-30T13:09:49.938548+0000","last_undegraded":"2024-03-30T13:09:49.938548+0000","last_fullsized":"2024-03-30T13:09:49.938548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:47.055563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132399+0000","last_change":"2024-03-30T13:09:47.146122+0000","last_active":"2024-03-30T13:09:49.132399+0000","last_peered":"2024-03-30T13:09:49.132399+0000","last_clean":"2024-03-30T13:09:49.132399+0000","last_became_active":"2024-03-30T13:09:47.145870+0000","last_became_peered":"2024-03-30T13:09:47.145870+0000","last_unstale":"2024-03-30T13:09:49.132399+0000","last_undegraded":"2024-03-30T13:09:49.132399+0000","last_fullsized":"2024-03-30T13:09:49.132399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132985+0000","last_change":"2024-03-30T13:09:43.130941+0000","last_active":"2024-03-30T13:09:49.132985+0000","last_peered":"2024-03-30T13:09:49.132985+0000","last_clean":"2024-03-30T13:09:49.132985+0000","last_became_active":"2024-03-30T13:09:43.130335+0000","last_became_peered":"2024-03-30T13:09:43.130335+0000","last_unstale":"2024-03-30T13:09:49.132985+0000","last_undegraded":"2024-03-30T13:09:49.132985+0000","last_fullsized":"2024-03-30T13:09:49.132985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938369+0000","last_change":"2024-03-30T13:09:42.124102+0000","last_active":"2024-03-30T13:09:49.938369+0000","last_peered":"2024-03-30T13:09:49.938369+0000","last_clean":"2024-03-30T13:09:49.938369+0000","last_became_active":"2024-03-30T13:09:42.123746+0000","last_became_peered":"2024-03-30T13:09:42.123746+0000","last_unstale":"2024-03-30T13:09:49.938369+0000","last_undegraded":"2024-03-30T13:09:49.938369+0000","last_fullsized":"2024-03-30T13:09:49.938369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132249+0000","last_change":"2024-03-30T13:09:47.153343+0000","last_active":"2024-03-30T13:09:49.132249+0000","last_peered":"2024-03-30T13:09:49.132249+0000","last_clean":"2024-03-30T13:09:49.132249+0000","last_became_active":"2024-03-30T13:09:47.152894+0000","last_became_peered":"2024-03-30T13:09:47.152894+0000","last_unstale":"2024-03-30T13:09:49.132249+0000","last_undegraded":"2024-03-30T13:09:49.132249+0000","last_fullsized":"2024-03-30T13:09:49.132249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132418+0000","last_change":"2024-03-30T13:09:43.117715+0000","last_active":"2024-03-30T13:09:49.132418+0000","last_peered":"2024-03-30T13:09:49.132418+0000","last_clean":"2024-03-30T13:09:49.132418+0000","last_became_active":"2024-03-30T13:09:43.116768+0000","last_became_peered":"2024-03-30T13:09:43.116768+0000","last_unstale":"2024-03-30T13:09:49.132418+0000","last_undegraded":"2024-03-30T13:09:49.132418+0000","last_fullsized":"2024-03-30T13:09:49.132418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938237+0000","last_change":"2024-03-30T13:09:42.125545+0000","last_active":"2024-03-30T13:09:49.938237+0000","last_peered":"2024-03-30T13:09:49.938237+0000","last_clean":"2024-03-30T13:09:49.938237+0000","last_became_active":"2024-03-30T13:09:42.125280+0000","last_became_peered":"2024-03-30T13:09:42.125280+0000","last_unstale":"2024-03-30T13:09:49.938237+0000","last_undegraded":"2024-03-30T13:09:49.938237+0000","last_fullsized":"2024-03-30T13:09:49.938237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938821+0000","last_change":"2024-03-30T13:09:47.571186+0000","last_active":"2024-03-30T13:09:49.938821+0000","last_peered":"2024-03-30T13:09:49.938821+0000","last_clean":"2024-03-30T13:09:49.938821+0000","last_became_active":"2024-03-30T13:09:47.570755+0000","last_became_peered":"2024-03-30T13:09:47.570755+0000","last_unstale":"2024-03-30T13:09:49.938821+0000","last_undegraded":"2024-03-30T13:09:49.938821+0000","last_fullsized":"2024-03-30T13:09:49.938821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131859+0000","last_change":"2024-03-30T13:09:43.115171+0000","last_active":"2024-03-30T13:09:49.131859+0000","last_peered":"2024-03-30T13:09:49.131859+0000","last_clean":"2024-03-30T13:09:49.131859+0000","last_became_active":"2024-03-30T13:09:43.113821+0000","last_became_peered":"2024-03-30T13:09:43.113821+0000","last_unstale":"2024-03-30T13:09:49.131859+0000","last_undegraded":"2024-03-30T13:09:49.131859+0000","last_fullsized":"2024-03-30T13:09:49.131859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:04.366501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938219+0000","last_change":"2024-03-30T13:09:42.125753+0000","last_active":"2024-03-30T13:09:49.938219+0000","last_peered":"2024-03-30T13:09:49.938219+0000","last_clean":"2024-03-30T13:09:49.938219+0000","last_became_active":"2024-03-30T13:09:42.125505+0000","last_became_peered":"2024-03-30T13:09:42.125505+0000","last_unstale":"2024-03-30T13:09:49.938219+0000","last_undegraded":"2024-03-30T13:09:49.938219+0000","last_fullsized":"2024-03-30T13:09:49.938219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:43:24.738116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132292+0000","last_change":"2024-03-30T13:09:47.149806+0000","last_active":"2024-03-30T13:09:49.132292+0000","last_peered":"2024-03-30T13:09:49.132292+0000","last_clean":"2024-03-30T13:09:49.132292+0000","last_became_active":"2024-03-30T13:09:47.149427+0000","last_became_peered":"2024-03-30T13:09:47.149427+0000","last_unstale":"2024-03-30T13:09:49.132292+0000","last_undegraded":"2024-03-30T13:09:49.132292+0000","last_fullsized":"2024-03-30T13:09:49.132292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939301+0000","last_change":"2024-03-30T13:09:43.122135+0000","last_active":"2024-03-30T13:09:49.939301+0000","last_peered":"2024-03-30T13:09:49.939301+0000","last_clean":"2024-03-30T13:09:49.939301+0000","last_became_active":"2024-03-30T13:09:43.121877+0000","last_became_peered":"2024-03-30T13:09:43.121877+0000","last_unstale":"2024-03-30T13:09:49.939301+0000","last_undegraded":"2024-03-30T13:09:49.939301+0000","last_fullsized":"2024-03-30T13:09:49.939301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295852+0000","last_change":"2024-03-30T13:09:42.124452+0000","last_active":"2024-03-30T13:09:49.295852+0000","last_peered":"2024-03-30T13:09:49.295852+0000","last_clean":"2024-03-30T13:09:49.295852+0000","last_became_active":"2024-03-30T13:09:42.124188+0000","last_became_peered":"2024-03-30T13:09:42.124188+0000","last_unstale":"2024-03-30T13:09:49.295852+0000","last_undegraded":"2024-03-30T13:09:49.295852+0000","last_fullsized":"2024-03-30T13:09:49.295852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:47.331912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131805+0000","last_change":"2024-03-30T13:09:47.571007+0000","last_active":"2024-03-30T13:09:49.131805+0000","last_peered":"2024-03-30T13:09:49.131805+0000","last_clean":"2024-03-30T13:09:49.131805+0000","last_became_active":"2024-03-30T13:09:47.570641+0000","last_became_peered":"2024-03-30T13:09:47.570641+0000","last_unstale":"2024-03-30T13:09:49.131805+0000","last_undegraded":"2024-03-30T13:09:49.131805+0000","last_fullsized":"2024-03-30T13:09:49.131805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133129+0000","last_change":"2024-03-30T13:09:43.123324+0000","last_active":"2024-03-30T13:09:49.133129+0000","last_peered":"2024-03-30T13:09:49.133129+0000","last_clean":"2024-03-30T13:09:49.133129+0000","last_became_active":"2024-03-30T13:09:43.122970+0000","last_became_peered":"2024-03-30T13:09:43.122970+0000","last_unstale":"2024-03-30T13:09:49.133129+0000","last_undegraded":"2024-03-30T13:09:49.133129+0000","last_fullsized":"2024-03-30T13:09:49.133129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938610+0000","last_change":"2024-03-30T13:09:42.112467+0000","last_active":"2024-03-30T13:09:49.938610+0000","last_peered":"2024-03-30T13:09:49.938610+0000","last_clean":"2024-03-30T13:09:49.938610+0000","last_became_active":"2024-03-30T13:09:42.112119+0000","last_became_peered":"2024-03-30T13:09:42.112119+0000","last_unstale":"2024-03-30T13:09:49.938610+0000","last_undegraded":"2024-03-30T13:09:49.938610+0000","last_fullsized":"2024-03-30T13:09:49.938610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:45.606451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132205+0000","last_change":"2024-03-30T13:09:47.571907+0000","last_active":"2024-03-30T13:09:49.132205+0000","last_peered":"2024-03-30T13:09:49.132205+0000","last_clean":"2024-03-30T13:09:49.132205+0000","last_became_active":"2024-03-30T13:09:47.571466+0000","last_became_peered":"2024-03-30T13:09:47.571466+0000","last_unstale":"2024-03-30T13:09:49.132205+0000","last_undegraded":"2024-03-30T13:09:49.132205+0000","last_fullsized":"2024-03-30T13:09:49.132205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:36:14.998133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938844+0000","last_change":"2024-03-30T13:09:43.114536+0000","last_active":"2024-03-30T13:09:49.938844+0000","last_peered":"2024-03-30T13:09:49.938844+0000","last_clean":"2024-03-30T13:09:49.938844+0000","last_became_active":"2024-03-30T13:09:43.114268+0000","last_became_peered":"2024-03-30T13:09:43.114268+0000","last_unstale":"2024-03-30T13:09:49.938844+0000","last_undegraded":"2024-03-30T13:09:49.938844+0000","last_fullsized":"2024-03-30T13:09:49.938844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938931+0000","last_change":"2024-03-30T13:09:42.125506+0000","last_active":"2024-03-30T13:09:49.938931+0000","last_peered":"2024-03-30T13:09:49.938931+0000","last_clean":"2024-03-30T13:09:49.938931+0000","last_became_active":"2024-03-30T13:09:42.125297+0000","last_became_peered":"2024-03-30T13:09:42.125297+0000","last_unstale":"2024-03-30T13:09:49.938931+0000","last_undegraded":"2024-03-30T13:09:49.938931+0000","last_fullsized":"2024-03-30T13:09:49.938931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:58.545296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938573+0000","last_change":"2024-03-30T13:09:47.570730+0000","last_active":"2024-03-30T13:09:49.938573+0000","last_peered":"2024-03-30T13:09:49.938573+0000","last_clean":"2024-03-30T13:09:49.938573+0000","last_became_active":"2024-03-30T13:09:47.570239+0000","last_became_peered":"2024-03-30T13:09:47.570239+0000","last_unstale":"2024-03-30T13:09:49.938573+0000","last_undegraded":"2024-03-30T13:09:49.938573+0000","last_fullsized":"2024-03-30T13:09:49.938573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:22:25.802912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938464+0000","last_change":"2024-03-30T13:09:43.127560+0000","last_active":"2024-03-30T13:09:49.938464+0000","last_peered":"2024-03-30T13:09:49.938464+0000","last_clean":"2024-03-30T13:09:49.938464+0000","last_became_active":"2024-03-30T13:09:43.127245+0000","last_became_peered":"2024-03-30T13:09:43.127245+0000","last_unstale":"2024-03-30T13:09:49.938464+0000","last_undegraded":"2024-03-30T13:09:49.938464+0000","last_fullsized":"2024-03-30T13:09:49.938464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:37.548735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938862+0000","last_change":"2024-03-30T13:09:42.116193+0000","last_active":"2024-03-30T13:09:49.938862+0000","last_peered":"2024-03-30T13:09:49.938862+0000","last_clean":"2024-03-30T13:09:49.938862+0000","last_became_active":"2024-03-30T13:09:42.115609+0000","last_became_peered":"2024-03-30T13:09:42.115609+0000","last_unstale":"2024-03-30T13:09:49.938862+0000","last_undegraded":"2024-03-30T13:09:49.938862+0000","last_fullsized":"2024-03-30T13:09:49.938862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938004+0000","last_change":"2024-03-30T13:09:47.159058+0000","last_active":"2024-03-30T13:09:49.938004+0000","last_peered":"2024-03-30T13:09:49.938004+0000","last_clean":"2024-03-30T13:09:49.938004+0000","last_became_active":"2024-03-30T13:09:47.158646+0000","last_became_peered":"2024-03-30T13:09:47.158646+0000","last_unstale":"2024-03-30T13:09:49.938004+0000","last_undegraded":"2024-03-30T13:09:49.938004+0000","last_fullsized":"2024-03-30T13:09:49.938004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:51.919084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132471+0000","last_change":"2024-03-30T13:09:43.121780+0000","last_active":"2024-03-30T13:09:49.132471+0000","last_peered":"2024-03-30T13:09:49.132471+0000","last_clean":"2024-03-30T13:09:49.132471+0000","last_became_active":"2024-03-30T13:09:43.121519+0000","last_became_peered":"2024-03-30T13:09:43.121519+0000","last_unstale":"2024-03-30T13:09:49.132471+0000","last_undegraded":"2024-03-30T13:09:49.132471+0000","last_fullsized":"2024-03-30T13:09:49.132471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:48.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938327+0000","last_change":"2024-03-30T13:09:42.122015+0000","last_active":"2024-03-30T13:09:49.938327+0000","last_peered":"2024-03-30T13:09:49.938327+0000","last_clean":"2024-03-30T13:09:49.938327+0000","last_became_active":"2024-03-30T13:09:42.121721+0000","last_became_peered":"2024-03-30T13:09:42.121721+0000","last_unstale":"2024-03-30T13:09:49.938327+0000","last_undegraded":"2024-03-30T13:09:49.938327+0000","last_fullsized":"2024-03-30T13:09:49.938327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:53:08.078627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938675+0000","last_change":"2024-03-30T13:09:47.159041+0000","last_active":"2024-03-30T13:09:49.938675+0000","last_peered":"2024-03-30T13:09:49.938675+0000","last_clean":"2024-03-30T13:09:49.938675+0000","last_became_active":"2024-03-30T13:09:47.158466+0000","last_became_peered":"2024-03-30T13:09:47.158466+0000","last_unstale":"2024-03-30T13:09:49.938675+0000","last_undegraded":"2024-03-30T13:09:49.938675+0000","last_fullsized":"2024-03-30T13:09:49.938675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938050+0000","last_change":"2024-03-30T13:09:43.111943+0000","last_active":"2024-03-30T13:09:49.938050+0000","last_peered":"2024-03-30T13:09:49.938050+0000","last_clean":"2024-03-30T13:09:49.938050+0000","last_became_active":"2024-03-30T13:09:43.111677+0000","last_became_peered":"2024-03-30T13:09:43.111677+0000","last_unstale":"2024-03-30T13:09:49.938050+0000","last_undegraded":"2024-03-30T13:09:49.938050+0000","last_fullsized":"2024-03-30T13:09:49.938050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:33:22.810971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821050+0000","last_change":"2024-03-30T13:09:42.121133+0000","last_active":"2024-03-30T13:09:49.821050+0000","last_peered":"2024-03-30T13:09:49.821050+0000","last_clean":"2024-03-30T13:09:49.821050+0000","last_became_active":"2024-03-30T13:09:42.120851+0000","last_became_peered":"2024-03-30T13:09:42.120851+0000","last_unstale":"2024-03-30T13:09:49.821050+0000","last_undegraded":"2024-03-30T13:09:49.821050+0000","last_fullsized":"2024-03-30T13:09:49.821050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938890+0000","last_change":"2024-03-30T13:09:47.155307+0000","last_active":"2024-03-30T13:09:49.938890+0000","last_peered":"2024-03-30T13:09:49.938890+0000","last_clean":"2024-03-30T13:09:49.938890+0000","last_became_active":"2024-03-30T13:09:47.154875+0000","last_became_peered":"2024-03-30T13:09:47.154875+0000","last_unstale":"2024-03-30T13:09:49.938890+0000","last_undegraded":"2024-03-30T13:09:49.938890+0000","last_fullsized":"2024-03-30T13:09:49.938890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:17:44.422352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937941+0000","last_change":"2024-03-30T13:09:43.128352+0000","last_active":"2024-03-30T13:09:49.937941+0000","last_peered":"2024-03-30T13:09:49.937941+0000","last_clean":"2024-03-30T13:09:49.937941+0000","last_became_active":"2024-03-30T13:09:43.128073+0000","last_became_peered":"2024-03-30T13:09:43.128073+0000","last_unstale":"2024-03-30T13:09:49.937941+0000","last_undegraded":"2024-03-30T13:09:49.937941+0000","last_fullsized":"2024-03-30T13:09:49.937941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:41.389848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938326+0000","last_change":"2024-03-30T13:09:42.124601+0000","last_active":"2024-03-30T13:09:49.938326+0000","last_peered":"2024-03-30T13:09:49.938326+0000","last_clean":"2024-03-30T13:09:49.938326+0000","last_became_active":"2024-03-30T13:09:42.124335+0000","last_became_peered":"2024-03-30T13:09:42.124335+0000","last_unstale":"2024-03-30T13:09:49.938326+0000","last_undegraded":"2024-03-30T13:09:49.938326+0000","last_fullsized":"2024-03-30T13:09:49.938326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:02:06.738441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939206+0000","last_change":"2024-03-30T13:09:47.572608+0000","last_active":"2024-03-30T13:09:49.939206+0000","last_peered":"2024-03-30T13:09:49.939206+0000","last_clean":"2024-03-30T13:09:49.939206+0000","last_became_active":"2024-03-30T13:09:47.571945+0000","last_became_peered":"2024-03-30T13:09:47.571945+0000","last_unstale":"2024-03-30T13:09:49.939206+0000","last_undegraded":"2024-03-30T13:09:49.939206+0000","last_fullsized":"2024-03-30T13:09:49.939206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:06:37.115753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132459+0000","last_change":"2024-03-30T13:09:43.125963+0000","last_active":"2024-03-30T13:09:49.132459+0000","last_peered":"2024-03-30T13:09:49.132459+0000","last_clean":"2024-03-30T13:09:49.132459+0000","last_became_active":"2024-03-30T13:09:43.125682+0000","last_became_peered":"2024-03-30T13:09:43.125682+0000","last_unstale":"2024-03-30T13:09:49.132459+0000","last_undegraded":"2024-03-30T13:09:49.132459+0000","last_fullsized":"2024-03-30T13:09:49.132459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938090+0000","last_change":"2024-03-30T13:09:42.119717+0000","last_active":"2024-03-30T13:09:49.938090+0000","last_peered":"2024-03-30T13:09:49.938090+0000","last_clean":"2024-03-30T13:09:49.938090+0000","last_became_active":"2024-03-30T13:09:42.119185+0000","last_became_peered":"2024-03-30T13:09:42.119185+0000","last_unstale":"2024-03-30T13:09:49.938090+0000","last_undegraded":"2024-03-30T13:09:49.938090+0000","last_fullsized":"2024-03-30T13:09:49.938090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938410+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:09:49.938410+0000","last_peered":"2024-03-30T13:09:49.938410+0000","last_clean":"2024-03-30T13:09:49.938410+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:09:49.938410+0000","last_undegraded":"2024-03-30T13:09:49.938410+0000","last_fullsized":"2024-03-30T13:09:49.938410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295610+0000","last_change":"2024-03-30T13:09:47.572345+0000","last_active":"2024-03-30T13:09:49.295610+0000","last_peered":"2024-03-30T13:09:49.295610+0000","last_clean":"2024-03-30T13:09:49.295610+0000","last_became_active":"2024-03-30T13:09:47.571956+0000","last_became_peered":"2024-03-30T13:09:47.571956+0000","last_unstale":"2024-03-30T13:09:49.295610+0000","last_undegraded":"2024-03-30T13:09:49.295610+0000","last_fullsized":"2024-03-30T13:09:49.295610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:45.839783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938718+0000","last_change":"2024-03-30T13:09:43.134581+0000","last_active":"2024-03-30T13:09:49.938718+0000","last_peered":"2024-03-30T13:09:49.938718+0000","last_clean":"2024-03-30T13:09:49.938718+0000","last_became_active":"2024-03-30T13:09:43.131462+0000","last_became_peered":"2024-03-30T13:09:43.131462+0000","last_unstale":"2024-03-30T13:09:49.938718+0000","last_undegraded":"2024-03-30T13:09:49.938718+0000","last_fullsized":"2024-03-30T13:09:49.938718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:49:52.247198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821188+0000","last_change":"2024-03-30T13:09:42.119614+0000","last_active":"2024-03-30T13:09:49.821188+0000","last_peered":"2024-03-30T13:09:49.821188+0000","last_clean":"2024-03-30T13:09:49.821188+0000","last_became_active":"2024-03-30T13:09:42.119252+0000","last_became_peered":"2024-03-30T13:09:42.119252+0000","last_unstale":"2024-03-30T13:09:49.821188+0000","last_undegraded":"2024-03-30T13:09:49.821188+0000","last_fullsized":"2024-03-30T13:09:49.821188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295320+0000","last_change":"2024-03-30T13:09:47.571383+0000","last_active":"2024-03-30T13:09:49.295320+0000","last_peered":"2024-03-30T13:09:49.295320+0000","last_clean":"2024-03-30T13:09:49.295320+0000","last_became_active":"2024-03-30T13:09:47.570998+0000","last_became_peered":"2024-03-30T13:09:47.570998+0000","last_unstale":"2024-03-30T13:09:49.295320+0000","last_undegraded":"2024-03-30T13:09:49.295320+0000","last_fullsized":"2024-03-30T13:09:49.295320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938152+0000","last_change":"2024-03-30T13:09:43.127405+0000","last_active":"2024-03-30T13:09:49.938152+0000","last_peered":"2024-03-30T13:09:49.938152+0000","last_clean":"2024-03-30T13:09:49.938152+0000","last_became_active":"2024-03-30T13:09:43.122914+0000","last_became_peered":"2024-03-30T13:09:43.122914+0000","last_unstale":"2024-03-30T13:09:49.938152+0000","last_undegraded":"2024-03-30T13:09:49.938152+0000","last_fullsized":"2024-03-30T13:09:49.938152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:51.506154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938749+0000","last_change":"2024-03-30T13:09:42.113530+0000","last_active":"2024-03-30T13:09:49.938749+0000","last_peered":"2024-03-30T13:09:49.938749+0000","last_clean":"2024-03-30T13:09:49.938749+0000","last_became_active":"2024-03-30T13:09:42.113114+0000","last_became_peered":"2024-03-30T13:09:42.113114+0000","last_unstale":"2024-03-30T13:09:49.938749+0000","last_undegraded":"2024-03-30T13:09:49.938749+0000","last_fullsized":"2024-03-30T13:09:49.938749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:59:09.359720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938814+0000","last_change":"2024-03-30T13:09:47.156241+0000","last_active":"2024-03-30T13:09:49.938814+0000","last_peered":"2024-03-30T13:09:49.938814+0000","last_clean":"2024-03-30T13:09:49.938814+0000","last_became_active":"2024-03-30T13:09:47.155884+0000","last_became_peered":"2024-03-30T13:09:47.155884+0000","last_unstale":"2024-03-30T13:09:49.938814+0000","last_undegraded":"2024-03-30T13:09:49.938814+0000","last_fullsized":"2024-03-30T13:09:49.938814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:28:31.146025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939042+0000","last_change":"2024-03-30T13:09:43.126029+0000","last_active":"2024-03-30T13:09:49.939042+0000","last_peered":"2024-03-30T13:09:49.939042+0000","last_clean":"2024-03-30T13:09:49.939042+0000","last_became_active":"2024-03-30T13:09:43.125724+0000","last_became_peered":"2024-03-30T13:09:43.125724+0000","last_unstale":"2024-03-30T13:09:49.939042+0000","last_undegraded":"2024-03-30T13:09:49.939042+0000","last_fullsized":"2024-03-30T13:09:49.939042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:44.840967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821402+0000","last_change":"2024-03-30T13:09:42.125370+0000","last_active":"2024-03-30T13:09:49.821402+0000","last_peered":"2024-03-30T13:09:49.821402+0000","last_clean":"2024-03-30T13:09:49.821402+0000","last_became_active":"2024-03-30T13:09:42.124450+0000","last_became_peered":"2024-03-30T13:09:42.124450+0000","last_unstale":"2024-03-30T13:09:49.821402+0000","last_undegraded":"2024-03-30T13:09:49.821402+0000","last_fullsized":"2024-03-30T13:09:49.821402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937709+0000","last_change":"2024-03-30T13:09:47.571108+0000","last_active":"2024-03-30T13:09:49.937709+0000","last_peered":"2024-03-30T13:09:49.937709+0000","last_clean":"2024-03-30T13:09:49.937709+0000","last_became_active":"2024-03-30T13:09:47.570725+0000","last_became_peered":"2024-03-30T13:09:47.570725+0000","last_unstale":"2024-03-30T13:09:49.937709+0000","last_undegraded":"2024-03-30T13:09:49.937709+0000","last_fullsized":"2024-03-30T13:09:49.937709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:48:47.108355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938937+0000","last_change":"2024-03-30T13:09:43.125121+0000","last_active":"2024-03-30T13:09:49.938937+0000","last_peered":"2024-03-30T13:09:49.938937+0000","last_clean":"2024-03-30T13:09:49.938937+0000","last_became_active":"2024-03-30T13:09:43.124795+0000","last_became_peered":"2024-03-30T13:09:43.124795+0000","last_unstale":"2024-03-30T13:09:49.938937+0000","last_undegraded":"2024-03-30T13:09:49.938937+0000","last_fullsized":"2024-03-30T13:09:49.938937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:55.220836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132420+0000","last_change":"2024-03-30T13:09:42.118818+0000","last_active":"2024-03-30T13:09:49.132420+0000","last_peered":"2024-03-30T13:09:49.132420+0000","last_clean":"2024-03-30T13:09:49.132420+0000","last_became_active":"2024-03-30T13:09:42.118529+0000","last_became_peered":"2024-03-30T13:09:42.118529+0000","last_unstale":"2024-03-30T13:09:49.132420+0000","last_undegraded":"2024-03-30T13:09:49.132420+0000","last_fullsized":"2024-03-30T13:09:49.132420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:40:03.615310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.131582+0000","last_change":"2024-03-30T13:09:47.149385+0000","last_active":"2024-03-30T13:09:49.131582+0000","last_peered":"2024-03-30T13:09:49.131582+0000","last_clean":"2024-03-30T13:09:49.131582+0000","last_became_active":"2024-03-30T13:09:47.149030+0000","last_became_peered":"2024-03-30T13:09:47.149030+0000","last_unstale":"2024-03-30T13:09:49.131582+0000","last_undegraded":"2024-03-30T13:09:49.131582+0000","last_fullsized":"2024-03-30T13:09:49.131582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:58:59.243607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.118404+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.117853+0000","last_became_peered":"2024-03-30T13:09:43.117853+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:15:39.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820864+0000","last_change":"2024-03-30T13:09:42.117594+0000","last_active":"2024-03-30T13:09:49.820864+0000","last_peered":"2024-03-30T13:09:49.820864+0000","last_clean":"2024-03-30T13:09:49.820864+0000","last_became_active":"2024-03-30T13:09:42.117276+0000","last_became_peered":"2024-03-30T13:09:42.117276+0000","last_unstale":"2024-03-30T13:09:49.820864+0000","last_undegraded":"2024-03-30T13:09:49.820864+0000","last_fullsized":"2024-03-30T13:09:49.820864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:41:43.020798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939158+0000","last_change":"2024-03-30T13:09:47.155090+0000","last_active":"2024-03-30T13:09:49.939158+0000","last_peered":"2024-03-30T13:09:49.939158+0000","last_clean":"2024-03-30T13:09:49.939158+0000","last_became_active":"2024-03-30T13:09:47.154070+0000","last_became_peered":"2024-03-30T13:09:47.154070+0000","last_unstale":"2024-03-30T13:09:49.939158+0000","last_undegraded":"2024-03-30T13:09:49.939158+0000","last_fullsized":"2024-03-30T13:09:49.939158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295287+0000","last_change":"2024-03-30T13:09:43.127827+0000","last_active":"2024-03-30T13:09:49.295287+0000","last_peered":"2024-03-30T13:09:49.295287+0000","last_clean":"2024-03-30T13:09:49.295287+0000","last_became_active":"2024-03-30T13:09:43.127217+0000","last_became_peered":"2024-03-30T13:09:43.127217+0000","last_unstale":"2024-03-30T13:09:49.295287+0000","last_undegraded":"2024-03-30T13:09:49.295287+0000","last_fullsized":"2024-03-30T13:09:49.295287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295274+0000","last_change":"2024-03-30T13:09:42.112507+0000","last_active":"2024-03-30T13:09:49.295274+0000","last_peered":"2024-03-30T13:09:49.295274+0000","last_clean":"2024-03-30T13:09:49.295274+0000","last_became_active":"2024-03-30T13:09:42.112227+0000","last_became_peered":"2024-03-30T13:09:42.112227+0000","last_unstale":"2024-03-30T13:09:49.295274+0000","last_undegraded":"2024-03-30T13:09:49.295274+0000","last_fullsized":"2024-03-30T13:09:49.295274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133062+0000","last_change":"2024-03-30T13:09:47.572882+0000","last_active":"2024-03-30T13:09:49.133062+0000","last_peered":"2024-03-30T13:09:49.133062+0000","last_clean":"2024-03-30T13:09:49.133062+0000","last_became_active":"2024-03-30T13:09:47.572527+0000","last_became_peered":"2024-03-30T13:09:47.572527+0000","last_unstale":"2024-03-30T13:09:49.133062+0000","last_undegraded":"2024-03-30T13:09:49.133062+0000","last_fullsized":"2024-03-30T13:09:49.133062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:36:18.356964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938472+0000","last_change":"2024-03-30T13:09:43.125989+0000","last_active":"2024-03-30T13:09:49.938472+0000","last_peered":"2024-03-30T13:09:49.938472+0000","last_clean":"2024-03-30T13:09:49.938472+0000","last_became_active":"2024-03-30T13:09:43.125685+0000","last_became_peered":"2024-03-30T13:09:43.125685+0000","last_unstale":"2024-03-30T13:09:49.938472+0000","last_undegraded":"2024-03-30T13:09:49.938472+0000","last_fullsized":"2024-03-30T13:09:49.938472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:23.535266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938231+0000","last_change":"2024-03-30T13:09:42.122605+0000","last_active":"2024-03-30T13:09:49.938231+0000","last_peered":"2024-03-30T13:09:49.938231+0000","last_clean":"2024-03-30T13:09:49.938231+0000","last_became_active":"2024-03-30T13:09:42.122338+0000","last_became_peered":"2024-03-30T13:09:42.122338+0000","last_unstale":"2024-03-30T13:09:49.938231+0000","last_undegraded":"2024-03-30T13:09:49.938231+0000","last_fullsized":"2024-03-30T13:09:49.938231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133046+0000","last_change":"2024-03-30T13:09:47.571300+0000","last_active":"2024-03-30T13:09:49.133046+0000","last_peered":"2024-03-30T13:09:49.133046+0000","last_clean":"2024-03-30T13:09:49.133046+0000","last_became_active":"2024-03-30T13:09:47.570892+0000","last_became_peered":"2024-03-30T13:09:47.570892+0000","last_unstale":"2024-03-30T13:09:49.133046+0000","last_undegraded":"2024-03-30T13:09:49.133046+0000","last_fullsized":"2024-03-30T13:09:49.133046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:09.284589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938281+0000","last_change":"2024-03-30T13:09:43.125239+0000","last_active":"2024-03-30T13:09:49.938281+0000","last_peered":"2024-03-30T13:09:49.938281+0000","last_clean":"2024-03-30T13:09:49.938281+0000","last_became_active":"2024-03-30T13:09:43.124815+0000","last_became_peered":"2024-03-30T13:09:43.124815+0000","last_unstale":"2024-03-30T13:09:49.938281+0000","last_undegraded":"2024-03-30T13:09:49.938281+0000","last_fullsized":"2024-03-30T13:09:49.938281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939006+0000","last_change":"2024-03-30T13:09:42.113307+0000","last_active":"2024-03-30T13:09:49.939006+0000","last_peered":"2024-03-30T13:09:49.939006+0000","last_clean":"2024-03-30T13:09:49.939006+0000","last_became_active":"2024-03-30T13:09:42.112968+0000","last_became_peered":"2024-03-30T13:09:42.112968+0000","last_unstale":"2024-03-30T13:09:49.939006+0000","last_undegraded":"2024-03-30T13:09:49.939006+0000","last_fullsized":"2024-03-30T13:09:49.939006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938380+0000","last_change":"2024-03-30T13:09:47.572011+0000","last_active":"2024-03-30T13:09:49.938380+0000","last_peered":"2024-03-30T13:09:49.938380+0000","last_clean":"2024-03-30T13:09:49.938380+0000","last_became_active":"2024-03-30T13:09:47.571632+0000","last_became_peered":"2024-03-30T13:09:47.571632+0000","last_unstale":"2024-03-30T13:09:49.938380+0000","last_undegraded":"2024-03-30T13:09:49.938380+0000","last_fullsized":"2024-03-30T13:09:49.938380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:20:18.911876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938714+0000","last_change":"2024-03-30T13:09:43.129993+0000","last_active":"2024-03-30T13:09:49.938714+0000","last_peered":"2024-03-30T13:09:49.938714+0000","last_clean":"2024-03-30T13:09:49.938714+0000","last_became_active":"2024-03-30T13:09:43.128265+0000","last_became_peered":"2024-03-30T13:09:43.128265+0000","last_unstale":"2024-03-30T13:09:49.938714+0000","last_undegraded":"2024-03-30T13:09:49.938714+0000","last_fullsized":"2024-03-30T13:09:49.938714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:28:32.691360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:52.955617+0000","last_change":"2024-03-30T13:09:42.120865+0000","last_active":"2024-03-30T13:09:52.955617+0000","last_peered":"2024-03-30T13:09:52.955617+0000","last_clean":"2024-03-30T13:09:52.955617+0000","last_became_active":"2024-03-30T13:09:42.120589+0000","last_became_peered":"2024-03-30T13:09:42.120589+0000","last_unstale":"2024-03-30T13:09:52.955617+0000","last_undegraded":"2024-03-30T13:09:52.955617+0000","last_fullsized":"2024-03-30T13:09:52.955617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:33:03.359370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938224+0000","last_change":"2024-03-30T13:09:47.155432+0000","last_active":"2024-03-30T13:09:49.938224+0000","last_peered":"2024-03-30T13:09:49.938224+0000","last_clean":"2024-03-30T13:09:49.938224+0000","last_became_active":"2024-03-30T13:09:47.155044+0000","last_became_peered":"2024-03-30T13:09:47.155044+0000","last_unstale":"2024-03-30T13:09:49.938224+0000","last_undegraded":"2024-03-30T13:09:49.938224+0000","last_fullsized":"2024-03-30T13:09:49.938224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:16.148467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132299+0000","last_change":"2024-03-30T13:09:43.115819+0000","last_active":"2024-03-30T13:09:49.132299+0000","last_peered":"2024-03-30T13:09:49.132299+0000","last_clean":"2024-03-30T13:09:49.132299+0000","last_became_active":"2024-03-30T13:09:43.115563+0000","last_became_peered":"2024-03-30T13:09:43.115563+0000","last_unstale":"2024-03-30T13:09:49.132299+0000","last_undegraded":"2024-03-30T13:09:49.132299+0000","last_fullsized":"2024-03-30T13:09:49.132299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:06:59.552427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938506+0000","last_change":"2024-03-30T13:09:42.121667+0000","last_active":"2024-03-30T13:09:49.938506+0000","last_peered":"2024-03-30T13:09:49.938506+0000","last_clean":"2024-03-30T13:09:49.938506+0000","last_became_active":"2024-03-30T13:09:42.121384+0000","last_became_peered":"2024-03-30T13:09:42.121384+0000","last_unstale":"2024-03-30T13:09:49.938506+0000","last_undegraded":"2024-03-30T13:09:49.938506+0000","last_fullsized":"2024-03-30T13:09:49.938506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:20:30.730301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132454+0000","last_change":"2024-03-30T13:09:47.147513+0000","last_active":"2024-03-30T13:09:49.132454+0000","last_peered":"2024-03-30T13:09:49.132454+0000","last_clean":"2024-03-30T13:09:49.132454+0000","last_became_active":"2024-03-30T13:09:47.147126+0000","last_became_peered":"2024-03-30T13:09:47.147126+0000","last_unstale":"2024-03-30T13:09:49.132454+0000","last_undegraded":"2024-03-30T13:09:49.132454+0000","last_fullsized":"2024-03-30T13:09:49.132454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:32.048989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939064+0000","last_change":"2024-03-30T13:09:43.130550+0000","last_active":"2024-03-30T13:09:49.939064+0000","last_peered":"2024-03-30T13:09:49.939064+0000","last_clean":"2024-03-30T13:09:49.939064+0000","last_became_active":"2024-03-30T13:09:43.130297+0000","last_became_peered":"2024-03-30T13:09:43.130297+0000","last_unstale":"2024-03-30T13:09:49.939064+0000","last_undegraded":"2024-03-30T13:09:49.939064+0000","last_fullsized":"2024-03-30T13:09:49.939064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938180+0000","last_change":"2024-03-30T13:09:42.122699+0000","last_active":"2024-03-30T13:09:49.938180+0000","last_peered":"2024-03-30T13:09:49.938180+0000","last_clean":"2024-03-30T13:09:49.938180+0000","last_became_active":"2024-03-30T13:09:42.122407+0000","last_became_peered":"2024-03-30T13:09:42.122407+0000","last_unstale":"2024-03-30T13:09:49.938180+0000","last_undegraded":"2024-03-30T13:09:49.938180+0000","last_fullsized":"2024-03-30T13:09:49.938180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:31:15.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132389+0000","last_change":"2024-03-30T13:09:47.160317+0000","last_active":"2024-03-30T13:09:49.132389+0000","last_peered":"2024-03-30T13:09:49.132389+0000","last_clean":"2024-03-30T13:09:49.132389+0000","last_became_active":"2024-03-30T13:09:47.159818+0000","last_became_peered":"2024-03-30T13:09:47.159818+0000","last_unstale":"2024-03-30T13:09:49.132389+0000","last_undegraded":"2024-03-30T13:09:49.132389+0000","last_fullsized":"2024-03-30T13:09:49.132389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:22:15.869872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938158+0000","last_change":"2024-03-30T13:09:43.128528+0000","last_active":"2024-03-30T13:09:49.938158+0000","last_peered":"2024-03-30T13:09:49.938158+0000","last_clean":"2024-03-30T13:09:49.938158+0000","last_became_active":"2024-03-30T13:09:43.128239+0000","last_became_peered":"2024-03-30T13:09:43.128239+0000","last_unstale":"2024-03-30T13:09:49.938158+0000","last_undegraded":"2024-03-30T13:09:49.938158+0000","last_fullsized":"2024-03-30T13:09:49.938158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:38:56.141275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295744+0000","last_change":"2024-03-30T13:09:42.118540+0000","last_active":"2024-03-30T13:09:49.295744+0000","last_peered":"2024-03-30T13:09:49.295744+0000","last_clean":"2024-03-30T13:09:49.295744+0000","last_became_active":"2024-03-30T13:09:42.118090+0000","last_became_peered":"2024-03-30T13:09:42.118090+0000","last_unstale":"2024-03-30T13:09:49.295744+0000","last_undegraded":"2024-03-30T13:09:49.295744+0000","last_fullsized":"2024-03-30T13:09:49.295744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:31.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132435+0000","last_change":"2024-03-30T13:09:47.571367+0000","last_active":"2024-03-30T13:09:49.132435+0000","last_peered":"2024-03-30T13:09:49.132435+0000","last_clean":"2024-03-30T13:09:49.132435+0000","last_became_active":"2024-03-30T13:09:47.570973+0000","last_became_peered":"2024-03-30T13:09:47.570973+0000","last_unstale":"2024-03-30T13:09:49.132435+0000","last_undegraded":"2024-03-30T13:09:49.132435+0000","last_fullsized":"2024-03-30T13:09:49.132435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:33:15.965915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939033+0000","last_change":"2024-03-30T13:09:47.572196+0000","last_active":"2024-03-30T13:09:49.939033+0000","last_peered":"2024-03-30T13:09:49.939033+0000","last_clean":"2024-03-30T13:09:49.939033+0000","last_became_active":"2024-03-30T13:09:47.571378+0000","last_became_peered":"2024-03-30T13:09:47.571378+0000","last_unstale":"2024-03-30T13:09:49.939033+0000","last_undegraded":"2024-03-30T13:09:49.939033+0000","last_fullsized":"2024-03-30T13:09:49.939033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:39.486465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938039+0000","last_change":"2024-03-30T13:09:43.125653+0000","last_active":"2024-03-30T13:09:49.938039+0000","last_peered":"2024-03-30T13:09:49.938039+0000","last_clean":"2024-03-30T13:09:49.938039+0000","last_became_active":"2024-03-30T13:09:43.125382+0000","last_became_peered":"2024-03-30T13:09:43.125382+0000","last_unstale":"2024-03-30T13:09:49.938039+0000","last_undegraded":"2024-03-30T13:09:49.938039+0000","last_fullsized":"2024-03-30T13:09:49.938039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:25:10.034049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937816+0000","last_change":"2024-03-30T13:09:42.115855+0000","last_active":"2024-03-30T13:09:49.937816+0000","last_peered":"2024-03-30T13:09:49.937816+0000","last_clean":"2024-03-30T13:09:49.937816+0000","last_became_active":"2024-03-30T13:09:42.115369+0000","last_became_peered":"2024-03-30T13:09:42.115369+0000","last_unstale":"2024-03-30T13:09:49.937816+0000","last_undegraded":"2024-03-30T13:09:49.937816+0000","last_fullsized":"2024-03-30T13:09:49.937816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:17:12.566587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295715+0000","last_change":"2024-03-30T13:09:47.156344+0000","last_active":"2024-03-30T13:09:49.295715+0000","last_peered":"2024-03-30T13:09:49.295715+0000","last_clean":"2024-03-30T13:09:49.295715+0000","last_became_active":"2024-03-30T13:09:47.155917+0000","last_became_peered":"2024-03-30T13:09:47.155917+0000","last_unstale":"2024-03-30T13:09:49.295715+0000","last_undegraded":"2024-03-30T13:09:49.295715+0000","last_fullsized":"2024-03-30T13:09:49.295715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:53.993664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132295+0000","last_change":"2024-03-30T13:09:43.128241+0000","last_active":"2024-03-30T13:09:49.132295+0000","last_peered":"2024-03-30T13:09:49.132295+0000","last_clean":"2024-03-30T13:09:49.132295+0000","last_became_active":"2024-03-30T13:09:43.127971+0000","last_became_peered":"2024-03-30T13:09:43.127971+0000","last_unstale":"2024-03-30T13:09:49.132295+0000","last_undegraded":"2024-03-30T13:09:49.132295+0000","last_fullsized":"2024-03-30T13:09:49.132295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132655+0000","last_change":"2024-03-30T13:09:42.122509+0000","last_active":"2024-03-30T13:09:49.132655+0000","last_peered":"2024-03-30T13:09:49.132655+0000","last_clean":"2024-03-30T13:09:49.132655+0000","last_became_active":"2024-03-30T13:09:42.122130+0000","last_became_peered":"2024-03-30T13:09:42.122130+0000","last_unstale":"2024-03-30T13:09:49.132655+0000","last_undegraded":"2024-03-30T13:09:49.132655+0000","last_fullsized":"2024-03-30T13:09:49.132655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:14:44.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295373+0000","last_change":"2024-03-30T13:09:42.115453+0000","last_active":"2024-03-30T13:09:49.295373+0000","last_peered":"2024-03-30T13:09:49.295373+0000","last_clean":"2024-03-30T13:09:49.295373+0000","last_became_active":"2024-03-30T13:09:42.115182+0000","last_became_peered":"2024-03-30T13:09:42.115182+0000","last_unstale":"2024-03-30T13:09:49.295373+0000","last_undegraded":"2024-03-30T13:09:49.295373+0000","last_fullsized":"2024-03-30T13:09:49.295373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:47:12.895083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938966+0000","last_change":"2024-03-30T13:09:43.127679+0000","last_active":"2024-03-30T13:09:49.938966+0000","last_peered":"2024-03-30T13:09:49.938966+0000","last_clean":"2024-03-30T13:09:49.938966+0000","last_became_active":"2024-03-30T13:09:43.127421+0000","last_became_peered":"2024-03-30T13:09:43.127421+0000","last_unstale":"2024-03-30T13:09:49.938966+0000","last_undegraded":"2024-03-30T13:09:49.938966+0000","last_fullsized":"2024-03-30T13:09:49.938966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:30:35.507683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821070+0000","last_change":"2024-03-30T13:09:47.154699+0000","last_active":"2024-03-30T13:09:49.821070+0000","last_peered":"2024-03-30T13:09:49.821070+0000","last_clean":"2024-03-30T13:09:49.821070+0000","last_became_active":"2024-03-30T13:09:47.154291+0000","last_became_peered":"2024-03-30T13:09:47.154291+0000","last_unstale":"2024-03-30T13:09:49.821070+0000","last_undegraded":"2024-03-30T13:09:49.821070+0000","last_fullsized":"2024-03-30T13:09:49.821070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:56:47.829620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133085+0000","last_change":"2024-03-30T13:09:42.124644+0000","last_active":"2024-03-30T13:09:49.133085+0000","last_peered":"2024-03-30T13:09:49.133085+0000","last_clean":"2024-03-30T13:09:49.133085+0000","last_became_active":"2024-03-30T13:09:42.124355+0000","last_became_peered":"2024-03-30T13:09:42.124355+0000","last_unstale":"2024-03-30T13:09:49.133085+0000","last_undegraded":"2024-03-30T13:09:49.133085+0000","last_fullsized":"2024-03-30T13:09:49.133085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:41.085835+0000","last_clean_scrub_stamp":"2024-03-30T13:09:41.085835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:15:50.714738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132040+0000","last_change":"2024-03-30T13:09:43.113299+0000","last_active":"2024-03-30T13:09:49.132040+0000","last_peered":"2024-03-30T13:09:49.132040+0000","last_clean":"2024-03-30T13:09:49.132040+0000","last_became_active":"2024-03-30T13:09:43.113031+0000","last_became_peered":"2024-03-30T13:09:43.113031+0000","last_unstale":"2024-03-30T13:09:49.132040+0000","last_undegraded":"2024-03-30T13:09:49.132040+0000","last_fullsized":"2024-03-30T13:09:49.132040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:57:32.715713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132575+0000","last_change":"2024-03-30T13:09:47.156912+0000","last_active":"2024-03-30T13:09:49.132575+0000","last_peered":"2024-03-30T13:09:49.132575+0000","last_clean":"2024-03-30T13:09:49.132575+0000","last_became_active":"2024-03-30T13:09:47.156492+0000","last_became_peered":"2024-03-30T13:09:47.156492+0000","last_unstale":"2024-03-30T13:09:49.132575+0000","last_undegraded":"2024-03-30T13:09:49.132575+0000","last_fullsized":"2024-03-30T13:09:49.132575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:46.115738+0000","last_clean_scrub_stamp":"2024-03-30T13:09:46.115738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:55:36.886194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295186+0000","last_change":"2024-03-30T13:09:43.127904+0000","last_active":"2024-03-30T13:09:49.295186+0000","last_peered":"2024-03-30T13:09:49.295186+0000","last_clean":"2024-03-30T13:09:49.295186+0000","last_became_active":"2024-03-30T13:09:43.127563+0000","last_became_peered":"2024-03-30T13:09:43.127563+0000","last_unstale":"2024-03-30T13:09:49.295186+0000","last_undegraded":"2024-03-30T13:09:49.295186+0000","last_fullsized":"2024-03-30T13:09:49.295186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.820892+0000","last_change":"2024-03-30T13:09:43.131561+0000","last_active":"2024-03-30T13:09:49.820892+0000","last_peered":"2024-03-30T13:09:49.820892+0000","last_clean":"2024-03-30T13:09:49.820892+0000","last_became_active":"2024-03-30T13:09:43.129703+0000","last_became_peered":"2024-03-30T13:09:43.129703+0000","last_unstale":"2024-03-30T13:09:49.820892+0000","last_undegraded":"2024-03-30T13:09:49.820892+0000","last_fullsized":"2024-03-30T13:09:49.820892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:46.613583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938131+0000","last_change":"2024-03-30T13:09:43.129678+0000","last_active":"2024-03-30T13:09:49.938131+0000","last_peered":"2024-03-30T13:09:49.938131+0000","last_clean":"2024-03-30T13:09:49.938131+0000","last_became_active":"2024-03-30T13:09:43.129405+0000","last_became_peered":"2024-03-30T13:09:43.129405+0000","last_unstale":"2024-03-30T13:09:49.938131+0000","last_undegraded":"2024-03-30T13:09:49.938131+0000","last_fullsized":"2024-03-30T13:09:49.938131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:13:29.545084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938913+0000","last_change":"2024-03-30T13:09:43.135202+0000","last_active":"2024-03-30T13:09:49.938913+0000","last_peered":"2024-03-30T13:09:49.938913+0000","last_clean":"2024-03-30T13:09:49.938913+0000","last_became_active":"2024-03-30T13:09:43.134895+0000","last_became_peered":"2024-03-30T13:09:43.134895+0000","last_unstale":"2024-03-30T13:09:49.938913+0000","last_undegraded":"2024-03-30T13:09:49.938913+0000","last_fullsized":"2024-03-30T13:09:49.938913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:11:20.840717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133173+0000","last_change":"2024-03-30T13:09:43.127414+0000","last_active":"2024-03-30T13:09:49.133173+0000","last_peered":"2024-03-30T13:09:49.133173+0000","last_clean":"2024-03-30T13:09:49.133173+0000","last_became_active":"2024-03-30T13:09:43.127077+0000","last_became_peered":"2024-03-30T13:09:43.127077+0000","last_unstale":"2024-03-30T13:09:49.133173+0000","last_undegraded":"2024-03-30T13:09:49.133173+0000","last_fullsized":"2024-03-30T13:09:49.133173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:17.251697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938324+0000","last_change":"2024-03-30T13:09:43.119060+0000","last_active":"2024-03-30T13:09:49.938324+0000","last_peered":"2024-03-30T13:09:49.938324+0000","last_clean":"2024-03-30T13:09:49.938324+0000","last_became_active":"2024-03-30T13:09:43.118585+0000","last_became_peered":"2024-03-30T13:09:43.118585+0000","last_unstale":"2024-03-30T13:09:49.938324+0000","last_undegraded":"2024-03-30T13:09:49.938324+0000","last_fullsized":"2024-03-30T13:09:49.938324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295833+0000","last_change":"2024-03-30T13:09:43.122310+0000","last_active":"2024-03-30T13:09:49.295833+0000","last_peered":"2024-03-30T13:09:49.295833+0000","last_clean":"2024-03-30T13:09:49.295833+0000","last_became_active":"2024-03-30T13:09:43.121999+0000","last_became_peered":"2024-03-30T13:09:43.121999+0000","last_unstale":"2024-03-30T13:09:49.295833+0000","last_undegraded":"2024-03-30T13:09:49.295833+0000","last_fullsized":"2024-03-30T13:09:49.295833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:12.837648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.133127+0000","last_change":"2024-03-30T13:09:43.122476+0000","last_active":"2024-03-30T13:09:49.133127+0000","last_peered":"2024-03-30T13:09:49.133127+0000","last_clean":"2024-03-30T13:09:49.133127+0000","last_became_active":"2024-03-30T13:09:43.120315+0000","last_became_peered":"2024-03-30T13:09:43.120315+0000","last_unstale":"2024-03-30T13:09:49.133127+0000","last_undegraded":"2024-03-30T13:09:49.133127+0000","last_fullsized":"2024-03-30T13:09:49.133127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937780+0000","last_change":"2024-03-30T13:09:43.128432+0000","last_active":"2024-03-30T13:09:49.937780+0000","last_peered":"2024-03-30T13:09:49.937780+0000","last_clean":"2024-03-30T13:09:49.937780+0000","last_became_active":"2024-03-30T13:09:43.128013+0000","last_became_peered":"2024-03-30T13:09:43.128013+0000","last_unstale":"2024-03-30T13:09:49.937780+0000","last_undegraded":"2024-03-30T13:09:49.937780+0000","last_fullsized":"2024-03-30T13:09:49.937780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:44:14.907897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132485+0000","last_change":"2024-03-30T13:09:43.120476+0000","last_active":"2024-03-30T13:09:49.132485+0000","last_peered":"2024-03-30T13:09:49.132485+0000","last_clean":"2024-03-30T13:09:49.132485+0000","last_became_active":"2024-03-30T13:09:43.120212+0000","last_became_peered":"2024-03-30T13:09:43.120212+0000","last_unstale":"2024-03-30T13:09:49.132485+0000","last_undegraded":"2024-03-30T13:09:49.132485+0000","last_fullsized":"2024-03-30T13:09:49.132485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:50:23.461880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938025+0000","last_change":"2024-03-30T13:09:43.137167+0000","last_active":"2024-03-30T13:09:49.938025+0000","last_peered":"2024-03-30T13:09:49.938025+0000","last_clean":"2024-03-30T13:09:49.938025+0000","last_became_active":"2024-03-30T13:09:43.136900+0000","last_became_peered":"2024-03-30T13:09:43.136900+0000","last_unstale":"2024-03-30T13:09:49.938025+0000","last_undegraded":"2024-03-30T13:09:49.938025+0000","last_fullsized":"2024-03-30T13:09:49.938025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:15:29.671082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124147+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.123860+0000","last_became_peered":"2024-03-30T13:09:43.123860+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:32:55.194498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295535+0000","last_change":"2024-03-30T13:09:43.127624+0000","last_active":"2024-03-30T13:09:49.295535+0000","last_peered":"2024-03-30T13:09:49.295535+0000","last_clean":"2024-03-30T13:09:49.295535+0000","last_became_active":"2024-03-30T13:09:43.127340+0000","last_became_peered":"2024-03-30T13:09:43.127340+0000","last_unstale":"2024-03-30T13:09:49.295535+0000","last_undegraded":"2024-03-30T13:09:49.295535+0000","last_fullsized":"2024-03-30T13:09:49.295535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938857+0000","last_change":"2024-03-30T13:09:43.122985+0000","last_active":"2024-03-30T13:09:49.938857+0000","last_peered":"2024-03-30T13:09:49.938857+0000","last_clean":"2024-03-30T13:09:49.938857+0000","last_became_active":"2024-03-30T13:09:43.122689+0000","last_became_peered":"2024-03-30T13:09:43.122689+0000","last_unstale":"2024-03-30T13:09:49.938857+0000","last_undegraded":"2024-03-30T13:09:49.938857+0000","last_fullsized":"2024-03-30T13:09:49.938857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:41.817012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.821341+0000","last_change":"2024-03-30T13:09:43.131062+0000","last_active":"2024-03-30T13:09:49.821341+0000","last_peered":"2024-03-30T13:09:49.821341+0000","last_clean":"2024-03-30T13:09:49.821341+0000","last_became_active":"2024-03-30T13:09:43.130727+0000","last_became_peered":"2024-03-30T13:09:43.130727+0000","last_unstale":"2024-03-30T13:09:49.821341+0000","last_undegraded":"2024-03-30T13:09:49.821341+0000","last_fullsized":"2024-03-30T13:09:49.821341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:03:31.612660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938468+0000","last_change":"2024-03-30T13:09:43.137027+0000","last_active":"2024-03-30T13:09:49.938468+0000","last_peered":"2024-03-30T13:09:49.938468+0000","last_clean":"2024-03-30T13:09:49.938468+0000","last_became_active":"2024-03-30T13:09:43.136755+0000","last_became_peered":"2024-03-30T13:09:43.136755+0000","last_unstale":"2024-03-30T13:09:49.938468+0000","last_undegraded":"2024-03-30T13:09:49.938468+0000","last_fullsized":"2024-03-30T13:09:49.938468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:38:26.087735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937690+0000","last_change":"2024-03-30T13:09:43.132197+0000","last_active":"2024-03-30T13:09:49.937690+0000","last_peered":"2024-03-30T13:09:49.937690+0000","last_clean":"2024-03-30T13:09:49.937690+0000","last_became_active":"2024-03-30T13:09:43.131927+0000","last_became_peered":"2024-03-30T13:09:43.131927+0000","last_unstale":"2024-03-30T13:09:49.937690+0000","last_undegraded":"2024-03-30T13:09:49.937690+0000","last_fullsized":"2024-03-30T13:09:49.937690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:27:01.999168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938139+0000","last_change":"2024-03-30T13:09:43.109275+0000","last_active":"2024-03-30T13:09:49.938139+0000","last_peered":"2024-03-30T13:09:49.938139+0000","last_clean":"2024-03-30T13:09:49.938139+0000","last_became_active":"2024-03-30T13:09:43.108964+0000","last_became_peered":"2024-03-30T13:09:43.108964+0000","last_unstale":"2024-03-30T13:09:49.938139+0000","last_undegraded":"2024-03-30T13:09:49.938139+0000","last_fullsized":"2024-03-30T13:09:49.938139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:48.847357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938228+0000","last_change":"2024-03-30T13:09:43.121915+0000","last_active":"2024-03-30T13:09:49.938228+0000","last_peered":"2024-03-30T13:09:49.938228+0000","last_clean":"2024-03-30T13:09:49.938228+0000","last_became_active":"2024-03-30T13:09:43.121625+0000","last_became_peered":"2024-03-30T13:09:43.121625+0000","last_unstale":"2024-03-30T13:09:49.938228+0000","last_undegraded":"2024-03-30T13:09:49.938228+0000","last_fullsized":"2024-03-30T13:09:49.938228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:12:44.483996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132890+0000","last_change":"2024-03-30T13:09:43.131187+0000","last_active":"2024-03-30T13:09:49.132890+0000","last_peered":"2024-03-30T13:09:49.132890+0000","last_clean":"2024-03-30T13:09:49.132890+0000","last_became_active":"2024-03-30T13:09:43.130682+0000","last_became_peered":"2024-03-30T13:09:43.130682+0000","last_unstale":"2024-03-30T13:09:49.132890+0000","last_undegraded":"2024-03-30T13:09:49.132890+0000","last_fullsized":"2024-03-30T13:09:49.132890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132240+0000","last_change":"2024-03-30T13:09:43.123005+0000","last_active":"2024-03-30T13:09:49.132240+0000","last_peered":"2024-03-30T13:09:49.132240+0000","last_clean":"2024-03-30T13:09:49.132240+0000","last_became_active":"2024-03-30T13:09:43.122761+0000","last_became_peered":"2024-03-30T13:09:43.122761+0000","last_unstale":"2024-03-30T13:09:49.132240+0000","last_undegraded":"2024-03-30T13:09:49.132240+0000","last_fullsized":"2024-03-30T13:09:49.132240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:48.062694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295578+0000","last_change":"2024-03-30T13:09:43.122715+0000","last_active":"2024-03-30T13:09:49.295578+0000","last_peered":"2024-03-30T13:09:49.295578+0000","last_clean":"2024-03-30T13:09:49.295578+0000","last_became_active":"2024-03-30T13:09:43.122445+0000","last_became_peered":"2024-03-30T13:09:43.122445+0000","last_unstale":"2024-03-30T13:09:49.295578+0000","last_undegraded":"2024-03-30T13:09:49.295578+0000","last_fullsized":"2024-03-30T13:09:49.295578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:38:12.076443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.295783+0000","last_change":"2024-03-30T13:09:43.130345+0000","last_active":"2024-03-30T13:09:49.295783+0000","last_peered":"2024-03-30T13:09:49.295783+0000","last_clean":"2024-03-30T13:09:49.295783+0000","last_became_active":"2024-03-30T13:09:43.130062+0000","last_became_peered":"2024-03-30T13:09:43.130062+0000","last_unstale":"2024-03-30T13:09:49.295783+0000","last_undegraded":"2024-03-30T13:09:49.295783+0000","last_fullsized":"2024-03-30T13:09:49.295783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:41:34.239774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.939037+0000","last_change":"2024-03-30T13:09:43.135018+0000","last_active":"2024-03-30T13:09:49.939037+0000","last_peered":"2024-03-30T13:09:49.939037+0000","last_clean":"2024-03-30T13:09:49.939037+0000","last_became_active":"2024-03-30T13:09:43.134354+0000","last_became_peered":"2024-03-30T13:09:43.134354+0000","last_unstale":"2024-03-30T13:09:49.939037+0000","last_undegraded":"2024-03-30T13:09:49.939037+0000","last_fullsized":"2024-03-30T13:09:49.939037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:05:04.849718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.132188+0000","last_change":"2024-03-30T13:09:43.123158+0000","last_active":"2024-03-30T13:09:49.132188+0000","last_peered":"2024-03-30T13:09:49.132188+0000","last_clean":"2024-03-30T13:09:49.132188+0000","last_became_active":"2024-03-30T13:09:43.122830+0000","last_became_peered":"2024-03-30T13:09:43.122830+0000","last_unstale":"2024-03-30T13:09:49.132188+0000","last_undegraded":"2024-03-30T13:09:49.132188+0000","last_fullsized":"2024-03-30T13:09:49.132188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938669+0000","last_change":"2024-03-30T13:09:43.124628+0000","last_active":"2024-03-30T13:09:49.938669+0000","last_peered":"2024-03-30T13:09:49.938669+0000","last_clean":"2024-03-30T13:09:49.938669+0000","last_became_active":"2024-03-30T13:09:43.124364+0000","last_became_peered":"2024-03-30T13:09:43.124364+0000","last_unstale":"2024-03-30T13:09:49.938669+0000","last_undegraded":"2024-03-30T13:09:49.938669+0000","last_fullsized":"2024-03-30T13:09:49.938669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:16:09.555508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.937937+0000","last_change":"2024-03-30T13:09:43.132387+0000","last_active":"2024-03-30T13:09:49.937937+0000","last_peered":"2024-03-30T13:09:49.937937+0000","last_clean":"2024-03-30T13:09:49.937937+0000","last_became_active":"2024-03-30T13:09:43.132110+0000","last_became_peered":"2024-03-30T13:09:43.132110+0000","last_unstale":"2024-03-30T13:09:49.937937+0000","last_undegraded":"2024-03-30T13:09:49.937937+0000","last_fullsized":"2024-03-30T13:09:49.937937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938308+0000","last_change":"2024-03-30T13:09:43.124528+0000","last_active":"2024-03-30T13:09:49.938308+0000","last_peered":"2024-03-30T13:09:49.938308+0000","last_clean":"2024-03-30T13:09:49.938308+0000","last_became_active":"2024-03-30T13:09:43.124023+0000","last_became_peered":"2024-03-30T13:09:43.124023+0000","last_unstale":"2024-03-30T13:09:49.938308+0000","last_undegraded":"2024-03-30T13:09:49.938308+0000","last_fullsized":"2024-03-30T13:09:49.938308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:09:10.450190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938061+0000","last_change":"2024-03-30T13:09:43.127663+0000","last_active":"2024-03-30T13:09:49.938061+0000","last_peered":"2024-03-30T13:09:49.938061+0000","last_clean":"2024-03-30T13:09:49.938061+0000","last_became_active":"2024-03-30T13:09:43.127180+0000","last_became_peered":"2024-03-30T13:09:43.127180+0000","last_unstale":"2024-03-30T13:09:49.938061+0000","last_undegraded":"2024-03-30T13:09:49.938061+0000","last_fullsized":"2024-03-30T13:09:49.938061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:36:33.418013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938739+0000","last_change":"2024-03-30T13:09:43.121807+0000","last_active":"2024-03-30T13:09:49.938739+0000","last_peered":"2024-03-30T13:09:49.938739+0000","last_clean":"2024-03-30T13:09:49.938739+0000","last_became_active":"2024-03-30T13:09:43.121498+0000","last_became_peered":"2024-03-30T13:09:43.121498+0000","last_unstale":"2024-03-30T13:09:49.938739+0000","last_undegraded":"2024-03-30T13:09:49.938739+0000","last_fullsized":"2024-03-30T13:09:49.938739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:29:03.981027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938554+0000","last_change":"2024-03-30T13:09:43.124024+0000","last_active":"2024-03-30T13:09:49.938554+0000","last_peered":"2024-03-30T13:09:49.938554+0000","last_clean":"2024-03-30T13:09:49.938554+0000","last_became_active":"2024-03-30T13:09:43.123748+0000","last_became_peered":"2024-03-30T13:09:43.123748+0000","last_unstale":"2024-03-30T13:09:49.938554+0000","last_undegraded":"2024-03-30T13:09:49.938554+0000","last_fullsized":"2024-03-30T13:09:49.938554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:18:30.126041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-30T13:09:49.938980+0000","last_change":"2024-03-30T13:09:43.122707+0000","last_active":"2024-03-30T13:09:49.938980+0000","last_peered":"2024-03-30T13:09:49.938980+0000","last_clean":"2024-03-30T13:09:49.938980+0000","last_became_active":"2024-03-30T13:09:43.122418+0000","last_became_peered":"2024-03-30T13:09:43.122418+0000","last_unstale":"2024-03-30T13:09:49.938980+0000","last_undegraded":"2024-03-30T13:09:49.938980+0000","last_fullsized":"2024-03-30T13:09:49.938980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:42.092966+0000","last_clean_scrub_stamp":"2024-03-30T13:09:42.092966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:44:49.970892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":700230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":109406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":876544,"data_stored":700286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":109280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":109440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":109370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:10:11.092 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T13:10:11.092 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-30T13:10:11.092 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-30T13:10:11.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T13:10:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.231+0000 7fd5c5724640 1 Processor -- start 2024-03-30T13:10:11.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.231+0000 7fd5c5724640 1 -- start start 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5c5724640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5c0162ce0 0x7fd5c00b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5c5724640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd5c00b05b0 0x7fd5c00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5c5724640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c00b5000 con 0x7fd5c015e450 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5c5724640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c00b0a20 con 0x7fd5c00b05b0 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5c5724640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c0162850 con 0x7fd5c0162ce0 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5bf7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd5c00b05b0 0x7fd5c00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5beffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd5c015e450 0x7fd5c00b41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:53998/0 (socket says 172.21.15.66:53998) 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5beffd640 1 -- 172.21.15.66:0/732000850 learned_addr learned my addr 172.21.15.66:0/732000850 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.232+0000 7fd5be7fc640 1 --2- 172.21.15.66:0/732000850 >> v2:172.21.15.117:3300/0 conn(0x7fd5c0162ce0 0x7fd5c00b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bdffb640 1 -- 172.21.15.66:0/732000850 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4259180635 0 0) 0x7fd5c00b5000 con 0x7fd5c015e450 2024-03-30T13:10:11.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bdffb640 1 -- 172.21.15.66:0/732000850 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5ac003520 con 0x7fd5c015e450 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/732000850 >> v2:172.21.15.117:3300/0 conn(0x7fd5c0162ce0 msgr2=0x7fd5c00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bf7fe640 1 --2- 172.21.15.66:0/732000850 >> v2:172.21.15.117:3300/0 conn(0x7fd5c0162ce0 0x7fd5c00b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/732000850 >> v1:172.21.15.66:6789/0 conn(0x7fd5c015e450 legacy=0x7fd5c00b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/732000850 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c00b4d20 con 0x7fd5c00b05b0 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bf7fe640 1 --2- 172.21.15.66:0/732000850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c00b05b0 0x7fd5c00d0e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5c01620c0 tx=0x7fd5b002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eb435531769ba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bdffb640 1 -- 172.21.15.66:0/732000850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0039070 con 0x7fd5c00b05b0 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.233+0000 7fd5bdffb640 1 -- 172.21.15.66:0/732000850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b002cc80 con 0x7fd5c00b05b0 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5bdffb640 1 -- 172.21.15.66:0/732000850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0034660 con 0x7fd5c00b05b0 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 -- 172.21.15.66:0/732000850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c00b05b0 msgr2=0x7fd5c00d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 --2- 172.21.15.66:0/732000850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c00b05b0 0x7fd5c00d0e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5c01620c0 tx=0x7fd5b002bed0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 -- 172.21.15.66:0/732000850 shutdown_connections 2024-03-30T13:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 --2- 172.21.15.66:0/732000850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c00b05b0 0x7fd5c00d0e40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 --2- 172.21.15.66:0/732000850 >> v2:172.21.15.117:3300/0 conn(0x7fd5c0162ce0 0x7fd5c00b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 -- 172.21.15.66:0/732000850 >> 172.21.15.66:0/732000850 conn(0x7fd5c006c690 msgr2=0x7fd5c006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 -- 172.21.15.66:0/732000850 shutdown_connections 2024-03-30T13:10:11.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.234+0000 7fd5c5724640 1 -- 172.21.15.66:0/732000850 wait complete. 2024-03-30T13:10:11.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.235+0000 7fd5c5724640 1 Processor -- start 2024-03-30T13:10:11.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.235+0000 7fd5c5724640 1 -- start start 2024-03-30T13:10:11.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5c5724640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00b05b0 0x7fd5c014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5c5724640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c015e450 0x7fd5c015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5c5724640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 0x7fd5c015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5c5724640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c006ed20 con 0x7fd5c0162ce0 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5c5724640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01624e0 con 0x7fd5c015e450 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5c5724640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c015e830 con 0x7fd5c00b05b0 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 0x7fd5c015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5be7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c015e450 0x7fd5c015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 0x7fd5c015fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34564/0 (socket says 172.21.15.66:34564) 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5beffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00b05b0 0x7fd5c014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5be7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c015e450 0x7fd5c015f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35256/0 (socket says 172.21.15.66:35256) 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.236+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/2813605794 learned_addr learned my addr 172.21.15.66:0/2813605794 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/2813605794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00b05b0 msgr2=0x7fd5c014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5bf7fe640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00b05b0 0x7fd5c014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/2813605794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c015e450 msgr2=0x7fd5c015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5bf7fe640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c015e450 0x7fd5c015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5bf7fe640 1 -- 172.21.15.66:0/2813605794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c014ef20 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5bf7fe640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 0x7fd5c015fc40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009e80 tx=0x7fd5b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0039050 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c0160460 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b0045070 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0009450 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.237+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c0160a20 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.238+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c0075a10 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.238+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd5b003e850 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.239+0000 7fd5b77fe640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5a006c530 0x7fd5a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.239+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd5b00ae710 con 0x7fd5c0162ce0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.239+0000 7fd5beffd640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5a006c530 0x7fd5a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.239+0000 7fd5beffd640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5a006c530 0x7fd5a006e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8005880 tx=0x7fd5b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.243+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd5b007c4d0 con 0x7fd5c0162ce0 2024-03-30T13:10:11.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.465+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd5c006e860 con 0x7fd5c0162ce0 2024-03-30T13:10:11.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.465+0000 7fd5b77fe640 1 -- 172.21.15.66:0/2813605794 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd5b00807d0 con 0x7fd5c0162ce0 2024-03-30T13:10:11.469 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:11.469 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T13:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5a006c530 msgr2=0x7fd5a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5a006c530 0x7fd5a006e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8005880 tx=0x7fd5b8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 msgr2=0x7fd5c015fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 0x7fd5c015fc40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009e80 tx=0x7fd5b0004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:11.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 shutdown_connections 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5a006c530 0x7fd5a006e9f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0162ce0 0x7fd5c015fc40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5c015e450 0x7fd5c015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 --2- 172.21.15.66:0/2813605794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5c00b05b0 0x7fd5c014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 >> 172.21.15.66:0/2813605794 conn(0x7fd5c006c690 msgr2=0x7fd5c00b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 shutdown_connections 2024-03-30T13:10:11.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.467+0000 7fd5c5724640 1 -- 172.21.15.66:0/2813605794 wait complete. 2024-03-30T13:10:11.482 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-30T13:10:11.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.587+0000 7f4d1dca5640 1 Processor -- start 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.587+0000 7f4d1dca5640 1 -- start start 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.587+0000 7f4d1dca5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4d180d1390 0x7f4d180d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d1dca5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d180b3570 0x7f4d180b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d1dca5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d180ba3d0 con 0x7f4d180b39d0 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d1dca5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4d180d1cb0 con 0x7f4d180d1390 2024-03-30T13:10:11.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d1dca5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f4d180cf6e0 con 0x7f4d180b3570 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4d180d1390 0x7f4d180d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4d180d1390 0x7f4d180d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34576/0 (socket says 172.21.15.66:34576) 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 -- 172.21.15.66:0/4274086745 learned_addr learned my addr 172.21.15.66:0/4274086745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d16ffd640 1 --2- 172.21.15.66:0/4274086745 >> v2:172.21.15.117:3300/0 conn(0x7f4d180b3570 0x7f4d180b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 -- 172.21.15.66:0/4274086745 >> v2:172.21.15.117:3300/0 conn(0x7f4d180b3570 msgr2=0x7f4d180b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 --2- 172.21.15.66:0/4274086745 >> v2:172.21.15.117:3300/0 conn(0x7f4d180b3570 0x7f4d180b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 -- 172.21.15.66:0/4274086745 >> v1:172.21.15.117:6789/0 conn(0x7f4d180b39d0 legacy=0x7f4d180b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.588+0000 7f4d177fe640 1 -- 172.21.15.66:0/4274086745 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d180b9050 con 0x7f4d180d1390 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d177fe640 1 --2- 172.21.15.66:0/4274086745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180d1390 0x7f4d180d1770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d180b63b0 tx=0x7f4d0802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce89632a8178ce21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d167fc640 1 -- 172.21.15.66:0/4274086745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0802f850 con 0x7f4d180d1390 2024-03-30T13:10:11.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d167fc640 1 -- 172.21.15.66:0/4274086745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0802f9f0 con 0x7f4d180d1390 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d167fc640 1 -- 172.21.15.66:0/4274086745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0802fdc0 con 0x7f4d180d1390 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4274086745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180d1390 msgr2=0x7f4d180d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4274086745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180d1390 0x7f4d180d1770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d180b63b0 tx=0x7f4d0802ba30 comp rx=0 tx=0).stop 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.589+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4274086745 shutdown_connections 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.590+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4274086745 >> v2:172.21.15.117:3300/0 conn(0x7f4d180b3570 0x7f4d180b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.590+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4274086745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180d1390 0x7f4d180d1770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.590+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4274086745 >> 172.21.15.66:0/4274086745 conn(0x7f4d180676e0 msgr2=0x7f4d18067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.590+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4274086745 shutdown_connections 2024-03-30T13:10:11.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.590+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4274086745 wait complete. 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.591+0000 7f4d1dca5640 1 Processor -- start 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.591+0000 7f4d1dca5640 1 -- start start 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d180b3570 0x7f4d18149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 0x7f4d18149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d180d1390 0x7f4d180c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180b98f0 con 0x7f4d180b39d0 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180b9610 con 0x7f4d180d1390 2024-03-30T13:10:11.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4d180b9bd0 con 0x7f4d180b3570 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 0x7f4d18149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 0x7f4d18149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34588/0 (socket says 172.21.15.66:34588) 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 -- 172.21.15.66:0/4199534567 learned_addr learned my addr 172.21.15.66:0/4199534567 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d177fe640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d180b3570 0x7f4d18149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d17fff640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d180d1390 0x7f4d180c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 -- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d180b3570 msgr2=0x7f4d18149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d180b3570 0x7f4d18149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 -- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d180d1390 msgr2=0x7f4d180c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d180d1390 0x7f4d180c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 -- 172.21.15.66:0/4199534567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d180cbd10 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d177fe640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d180b3570 0x7f4d18149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d17fff640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d180d1390 0x7f4d180c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d16ffd640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 0x7f4d18149870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00e9b0 tx=0x7f4d0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c017070 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00c9a0 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d180ca1e0 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00cd90 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.592+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d180ca7a0 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.593+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d18075a10 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.594+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4d0c01a070 con 0x7f4d180b39d0 2024-03-30T13:10:11.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.594+0000 7f4cfffff640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4d0406c530 0x7f4d0406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.594+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4d0c08b950 con 0x7f4d180b39d0 2024-03-30T13:10:11.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.594+0000 7f4d177fe640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4d0406c530 0x7f4d0406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.595+0000 7f4d177fe640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4d0406c530 0x7f4d0406e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d08036000 tx=0x7f4d08008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.599+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4d0c0595e0 con 0x7f4d180b39d0 2024-03-30T13:10:11.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.785+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d180b9ea0 con 0x7f4d180b39d0 2024-03-30T13:10:11.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.786+0000 7f4cfffff640 1 -- 172.21.15.66:0/4199534567 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f4d0c05d8e0 con 0x7f4d180b39d0 2024-03-30T13:10:11.790 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:11.790 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:11.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4d0406c530 msgr2=0x7f4d0406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4d0406c530 0x7f4d0406e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d08036000 tx=0x7f4d08008030 comp rx=0 tx=0).stop 2024-03-30T13:10:11.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 msgr2=0x7f4d18149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 0x7f4d18149870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00e9b0 tx=0x7f4d0c00ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 shutdown_connections 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4d0406c530 0x7f4d0406e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d180d1390 0x7f4d180c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d180b39d0 0x7f4d18149870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 --2- 172.21.15.66:0/4199534567 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d180b3570 0x7f4d18149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.788+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 >> 172.21.15.66:0/4199534567 conn(0x7f4d180676e0 msgr2=0x7f4d180d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.789+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 shutdown_connections 2024-03-30T13:10:11.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.789+0000 7f4d1dca5640 1 -- 172.21.15.66:0/4199534567 wait complete. 2024-03-30T13:10:11.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:11.804 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:11.913 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:11.915 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.911+0000 7fbfcf6f2640 1 Processor -- start 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.911+0000 7fbfcf6f2640 1 -- start start 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcf6f2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfc8155b10 0x7fbfc80b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcf6f2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc80d03c0 0x7fbfc80d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcf6f2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc80b5ff0 con 0x7fbfc815fae0 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcf6f2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc8155ef0 con 0x7fbfc8155b10 2024-03-30T13:10:11.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcf6f2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80b41b0 con 0x7fbfc80d03c0 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcd48e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfc8155b10 0x7fbfc80b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfccc8d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fbfc815fae0 0x7fbfc80b57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:40752/0 (socket says 172.21.15.66:40752) 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfccc8d640 1 -- 172.21.15.66:0/1001256492 learned_addr learned my addr 172.21.15.66:0/1001256492 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.912+0000 7fbfcdc8f640 1 --2- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc80d03c0 0x7fbfc80d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.913+0000 7fbfcd48e640 1 -- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc80d03c0 msgr2=0x7fbfc80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.913+0000 7fbfcd48e640 1 --2- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc80d03c0 0x7fbfc80d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.913+0000 7fbfcd48e640 1 -- 172.21.15.66:0/1001256492 >> v1:172.21.15.117:6789/0 conn(0x7fbfc815fae0 legacy=0x7fbfc80b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.913+0000 7fbfbffff640 1 -- 172.21.15.66:0/1001256492 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710303796 0 0) 0x7fbfc80b5ff0 con 0x7fbfc815fae0 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.913+0000 7fbfcd48e640 1 -- 172.21.15.66:0/1001256492 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc80b4490 con 0x7fbfc8155b10 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.913+0000 7fbfcd48e640 1 --2- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc8155b10 0x7fbfc80b4c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009d90 tx=0x7fbfc402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b135d28bc475377 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfbffff640 1 -- 172.21.15.66:0/1001256492 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4039070 con 0x7fbfc8155b10 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfbffff640 1 -- 172.21.15.66:0/1001256492 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc402cc50 con 0x7fbfc8155b10 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfbffff640 1 -- 172.21.15.66:0/1001256492 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4034440 con 0x7fbfc8155b10 2024-03-30T13:10:11.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc8155b10 msgr2=0x7fbfc80b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc8155b10 0x7fbfc80b4c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009d90 tx=0x7fbfc402bea0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/1001256492 shutdown_connections 2024-03-30T13:10:11.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc80d03c0 0x7fbfc80d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/1001256492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc8155b10 0x7fbfc80b4c90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/1001256492 >> 172.21.15.66:0/1001256492 conn(0x7fbfc80d1160 msgr2=0x7fbfc80d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:11.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/1001256492 shutdown_connections 2024-03-30T13:10:11.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.914+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/1001256492 wait complete. 2024-03-30T13:10:11.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 Processor -- start 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 -- start start 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80d03c0 0x7fbfc80b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc8155b10 0x7fbfc814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 0x7fbfc8145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc815b9d0 con 0x7fbfc815fae0 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80b62d0 con 0x7fbfc80d03c0 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.916+0000 7fbfcf6f2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc815bb50 con 0x7fbfc8155b10 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcdc8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 0x7fbfc8145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc8155b10 0x7fbfc814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcd48e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80d03c0 0x7fbfc80b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcdc8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 0x7fbfc8145bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49914/0 (socket says 172.21.15.66:49914) 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcdc8f640 1 -- 172.21.15.66:0/3182406246 learned_addr learned my addr 172.21.15.66:0/3182406246 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:11.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 -- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80d03c0 msgr2=0x7fbfc80b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80d03c0 0x7fbfc80b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 -- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 msgr2=0x7fbfc8145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 0x7fbfc8145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 -- 172.21.15.66:0/3182406246 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc814cdc0 con 0x7fbfc8155b10 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcd48e640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80d03c0 0x7fbfc80b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcdc8f640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 0x7fbfc8145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfccc8d640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc8155b10 0x7fbfc814e630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800c440 tx=0x7fbfb800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80074e0 con 0x7fbfc8155b10 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb80047e0 con 0x7fbfc8155b10 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8004bd0 con 0x7fbfc8155b10 2024-03-30T13:10:11.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.917+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc80b1bf0 con 0x7fbfc8155b10 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.918+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc80b2690 con 0x7fbfc8155b10 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.919+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc8075890 con 0x7fbfc8155b10 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.919+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbfb8011070 con 0x7fbfc8155b10 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.919+0000 7fbfbdffb640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbfac06c530 0x7fbfac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.920+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbfb808b490 con 0x7fbfc8155b10 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.920+0000 7fbfcd48e640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbfac06c530 0x7fbfac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:11.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.920+0000 7fbfcd48e640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbfac06c530 0x7fbfac06e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4036000 tx=0x7fbfc4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:11.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:11.928+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbfc814cdc0 con 0x7fbfc8155b10 2024-03-30T13:10:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.114+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfc806f640 con 0x7fbfc8155b10 2024-03-30T13:10:12.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.115+0000 7fbfbdffb640 1 -- 172.21.15.66:0/3182406246 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fbfb80591d0 con 0x7fbfc8155b10 2024-03-30T13:10:12.118 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:12.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbfac06c530 msgr2=0x7fbfac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbfac06c530 0x7fbfac06e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4036000 tx=0x7fbfc4030000 comp rx=0 tx=0).stop 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc8155b10 msgr2=0x7fbfc814e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc8155b10 0x7fbfc814e630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800c440 tx=0x7fbfb800c900 comp rx=0 tx=0).stop 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 shutdown_connections 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbfac06c530 0x7fbfac06e9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc815fae0 0x7fbfc8145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc8155b10 0x7fbfc814e630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 --2- 172.21.15.66:0/3182406246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80d03c0 0x7fbfc80b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 >> 172.21.15.66:0/3182406246 conn(0x7fbfc80d1160 msgr2=0x7fbfc806e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 shutdown_connections 2024-03-30T13:10:12.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.117+0000 7fbfcf6f2640 1 -- 172.21.15.66:0/3182406246 wait complete. 2024-03-30T13:10:12.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:12.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.239+0000 7fbda7b22640 1 Processor -- start 2024-03-30T13:10:12.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.239+0000 7fbda7b22640 1 -- start start 2024-03-30T13:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda7b22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda015a710 0x7fbda00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda7b22640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbda015aaf0 0x7fbda00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda7b22640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbda00b4ba0 0x7fbda006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda7b22640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbda00b41d0 con 0x7fbda015aaf0 2024-03-30T13:10:12.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda7b22640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fbda00b35c0 con 0x7fbda00b4ba0 2024-03-30T13:10:12.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda7b22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbda00b38a0 con 0x7fbda015a710 2024-03-30T13:10:12.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda50bd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbda015aaf0 0x7fbda00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda58be640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda015a710 0x7fbda00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda50bd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbda015aaf0 0x7fbda00b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49916/0 (socket says 172.21.15.66:49916) 2024-03-30T13:10:12.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda50bd640 1 -- 172.21.15.66:0/1241992327 learned_addr learned my addr 172.21.15.66:0/1241992327 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.240+0000 7fbda60bf640 1 --2- 172.21.15.66:0/1241992327 >> v2:172.21.15.117:3300/0 conn(0x7fbda00b4ba0 0x7fbda006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda50bd640 1 -- 172.21.15.66:0/1241992327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda015a710 msgr2=0x7fbda00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda50bd640 1 --2- 172.21.15.66:0/1241992327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda015a710 0x7fbda00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda50bd640 1 -- 172.21.15.66:0/1241992327 >> v2:172.21.15.117:3300/0 conn(0x7fbda00b4ba0 msgr2=0x7fbda006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda50bd640 1 --2- 172.21.15.66:0/1241992327 >> v2:172.21.15.117:3300/0 conn(0x7fbda00b4ba0 0x7fbda006d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda50bd640 1 -- 172.21.15.66:0/1241992327 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda00b3b80 con 0x7fbda015aaf0 2024-03-30T13:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda58be640 1 --2- 172.21.15.66:0/1241992327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda015a710 0x7fbda00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda60bf640 1 --2- 172.21.15.66:0/1241992327 >> v2:172.21.15.117:3300/0 conn(0x7fbda00b4ba0 0x7fbda006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.241+0000 7fbda50bd640 1 --2- 172.21.15.66:0/1241992327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda00b4660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbda00ccee0 tx=0x7fbd9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475353a4c1e4aacf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda48bc640 1 -- 172.21.15.66:0/1241992327 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94035020 con 0x7fbda015aaf0 2024-03-30T13:10:12.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda48bc640 1 -- 172.21.15.66:0/1241992327 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94031870 con 0x7fbda015aaf0 2024-03-30T13:10:12.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda48bc640 1 -- 172.21.15.66:0/1241992327 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94031c40 con 0x7fbda015aaf0 2024-03-30T13:10:12.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 -- 172.21.15.66:0/1241992327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 msgr2=0x7fbda00b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 --2- 172.21.15.66:0/1241992327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda00b4660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbda00ccee0 tx=0x7fbd9402b910 comp rx=0 tx=0).stop 2024-03-30T13:10:12.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 -- 172.21.15.66:0/1241992327 shutdown_connections 2024-03-30T13:10:12.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 --2- 172.21.15.66:0/1241992327 >> v2:172.21.15.117:3300/0 conn(0x7fbda00b4ba0 0x7fbda006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 --2- 172.21.15.66:0/1241992327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda00b4660 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 --2- 172.21.15.66:0/1241992327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda015a710 0x7fbda00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 -- 172.21.15.66:0/1241992327 >> 172.21.15.66:0/1241992327 conn(0x7fbda00b5b00 msgr2=0x7fbda006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 -- 172.21.15.66:0/1241992327 shutdown_connections 2024-03-30T13:10:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.242+0000 7fbda7b22640 1 -- 172.21.15.66:0/1241992327 wait complete. 2024-03-30T13:10:12.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 Processor -- start 2024-03-30T13:10:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 -- start start 2024-03-30T13:10:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbda015a710 0x7fbda0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda0158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda0156fc0 0x7fbda0159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00718c0 con 0x7fbda015aaf0 2024-03-30T13:10:12.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbda0072c50 con 0x7fbda015a710 2024-03-30T13:10:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda7b22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbda0071a40 con 0x7fbda0156fc0 2024-03-30T13:10:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda50bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda0158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda50bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda0158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49926/0 (socket says 172.21.15.66:49926) 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.244+0000 7fbda50bd640 1 -- 172.21.15.66:0/3449172634 learned_addr learned my addr 172.21.15.66:0/3449172634 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda60bf640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda0156fc0 0x7fbda0159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda58be640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbda015a710 0x7fbda0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda50bd640 1 -- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda0156fc0 msgr2=0x7fbda0159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda50bd640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda0156fc0 0x7fbda0159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda50bd640 1 -- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbda015a710 msgr2=0x7fbda0152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda50bd640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbda015a710 0x7fbda0152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda50bd640 1 -- 172.21.15.66:0/3449172634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda0157440 con 0x7fbda015aaf0 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda50bd640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda0158760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd9402b950 tx=0x7fbd940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94041070 con 0x7fbda015aaf0 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94031e10 con 0x7fbda015aaf0 2024-03-30T13:10:12.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda01576c0 con 0x7fbda015aaf0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94035040 con 0x7fbda015aaf0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.245+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda0157c80 con 0x7fbda015aaf0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.246+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda00af990 con 0x7fbda015aaf0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.247+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbd94038020 con 0x7fbda015aaf0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.247+0000 7fbd927fc640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbd8406c560 0x7fbd8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.247+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbd940ad560 con 0x7fbda015aaf0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.247+0000 7fbda58be640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbd8406c560 0x7fbd8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.248+0000 7fbda58be640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbd8406c560 0x7fbd8406ea20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009560 tx=0x7fbd9c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.252+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbda00af990 con 0x7fbda015aaf0 2024-03-30T13:10:12.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.436+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbda0075890 con 0x7fbda015aaf0 2024-03-30T13:10:12.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.437+0000 7fbd927fc640 1 -- 172.21.15.66:0/3449172634 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbda0157440 con 0x7fbda015aaf0 2024-03-30T13:10:12.440 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:12.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbd8406c560 msgr2=0x7fbd8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbd8406c560 0x7fbd8406ea20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009560 tx=0x7fbd9c009190 comp rx=0 tx=0).stop 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 msgr2=0x7fbda0158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda0158760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd9402b950 tx=0x7fbd940046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 shutdown_connections 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbd8406c560 0x7fbd8406ea20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbda0156fc0 0x7fbda0159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda015aaf0 0x7fbda0158760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 --2- 172.21.15.66:0/3449172634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbda015a710 0x7fbda0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 >> 172.21.15.66:0/3449172634 conn(0x7fbda00b5b00 msgr2=0x7fbda00b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 shutdown_connections 2024-03-30T13:10:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.439+0000 7fbda7b22640 1 -- 172.21.15.66:0/3449172634 wait complete. 2024-03-30T13:10:12.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:12.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.564+0000 7f20c5493640 1 Processor -- start 2024-03-30T13:10:12.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.564+0000 7f20c5493640 1 -- start start 2024-03-30T13:10:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20c5493640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20c00b5dc0 0x7f20c00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20c5493640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c00b66e0 con 0x7f20c00ba090 2024-03-30T13:10:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20c5493640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c00d2ff0 con 0x7f20c00b73d0 2024-03-30T13:10:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20c5493640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f20c00d2d90 con 0x7f20c00b5dc0 2024-03-30T13:10:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20beffd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20c00b5dc0 0x7f20c00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20bf7fe640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f20c00b73d0 0x7f20c00b7800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:40756/0 (socket says 172.21.15.66:40756) 2024-03-30T13:10:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20bf7fe640 1 -- 172.21.15.66:0/600082156 learned_addr learned my addr 172.21.15.66:0/600082156 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20beffd640 1 -- 172.21.15.66:0/600082156 >> v1:172.21.15.117:6789/0 conn(0x7f20c00b73d0 legacy=0x7f20c00b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20beffd640 1 -- 172.21.15.66:0/600082156 >> v1:172.21.15.66:6789/0 conn(0x7f20c00ba090 legacy=0x7f20c00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20bdffb640 1 -- 172.21.15.66:0/600082156 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180509308 0 0) 0x7f20c00d2ff0 con 0x7f20c00b73d0 2024-03-30T13:10:12.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.565+0000 7f20beffd640 1 -- 172.21.15.66:0/600082156 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c00d32d0 con 0x7f20c00b5dc0 2024-03-30T13:10:12.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.566+0000 7f20beffd640 1 --2- 172.21.15.66:0/600082156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b5dc0 0x7f20c00b61a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f20b40097a0 tx=0x7f20b402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1c0c6ff4791ac00 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.566+0000 7f20bdffb640 1 -- 172.21.15.66:0/600082156 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4037070 con 0x7f20c00b5dc0 2024-03-30T13:10:12.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.566+0000 7f20bdffb640 1 -- 172.21.15.66:0/600082156 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4030710 con 0x7f20c00b5dc0 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.566+0000 7f20bdffb640 1 -- 172.21.15.66:0/600082156 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4030ae0 con 0x7f20c00b5dc0 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 -- 172.21.15.66:0/600082156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b5dc0 msgr2=0x7f20c00b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 --2- 172.21.15.66:0/600082156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b5dc0 0x7f20c00b61a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f20b40097a0 tx=0x7f20b402b790 comp rx=0 tx=0).stop 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 -- 172.21.15.66:0/600082156 shutdown_connections 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 --2- 172.21.15.66:0/600082156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b5dc0 0x7f20c00b61a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 -- 172.21.15.66:0/600082156 >> 172.21.15.66:0/600082156 conn(0x7f20c00678f0 msgr2=0x7f20c00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 -- 172.21.15.66:0/600082156 shutdown_connections 2024-03-30T13:10:12.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.567+0000 7f20c5493640 1 -- 172.21.15.66:0/600082156 wait complete. 2024-03-30T13:10:12.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 Processor -- start 2024-03-30T13:10:12.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 -- start start 2024-03-30T13:10:12.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 0x7f20c0149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b73d0 0x7f20c0156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c00ba090 0x7f20c0161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20c00d3b70 con 0x7f20c00b5dc0 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20c00d3890 con 0x7f20c00b73d0 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20c5493640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f20c00d4090 con 0x7f20c00ba090 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 0x7f20c0149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20beffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 0x7f20c0149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49932/0 (socket says 172.21.15.66:49932) 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20beffd640 1 -- 172.21.15.66:0/873862414 learned_addr learned my addr 172.21.15.66:0/873862414 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20bf7fe640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c00ba090 0x7f20c0161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.569+0000 7f20be7fc640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b73d0 0x7f20c0156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20beffd640 1 -- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c00ba090 msgr2=0x7f20c0161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20beffd640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c00ba090 0x7f20c0161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20beffd640 1 -- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b73d0 msgr2=0x7f20c0156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20beffd640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b73d0 0x7f20c0156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20beffd640 1 -- 172.21.15.66:0/873862414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c00cb850 con 0x7f20c00b5dc0 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20beffd640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 0x7f20c0149a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20b4000c00 tx=0x7f20b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4042070 con 0x7f20c00b5dc0 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4032a40 con 0x7f20c00b5dc0 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c00cbb30 con 0x7f20c00b5dc0 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4037040 con 0x7f20c00b5dc0 2024-03-30T13:10:12.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.570+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c00cc0f0 con 0x7f20c00b5dc0 2024-03-30T13:10:12.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.572+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c0075890 con 0x7f20c00b5dc0 2024-03-30T13:10:12.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.572+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f20b4045070 con 0x7f20c00b5dc0 2024-03-30T13:10:12.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.572+0000 7f20a77fe640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f20ac06c530 0x7f20ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.572+0000 7f20be7fc640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f20ac06c530 0x7f20ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.572+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f20b40ad7e0 con 0x7f20c00b5dc0 2024-03-30T13:10:12.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.573+0000 7f20be7fc640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f20ac06c530 0x7f20ac06e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f20c01f7af0 tx=0x7f20b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.576+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f20b407b590 con 0x7f20c00b5dc0 2024-03-30T13:10:12.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.762+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f20c00b9e70 con 0x7f20c00b5dc0 2024-03-30T13:10:12.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.762+0000 7f20a77fe640 1 -- 172.21.15.66:0/873862414 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f20b407f890 con 0x7f20c00b5dc0 2024-03-30T13:10:12.766 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:12.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.764+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f20ac06c530 msgr2=0x7f20ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.764+0000 7f20c5493640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f20ac06c530 0x7f20ac06e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f20c01f7af0 tx=0x7f20b0009040 comp rx=0 tx=0).stop 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.764+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 msgr2=0x7f20c0149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.764+0000 7f20c5493640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 0x7f20c0149a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20b4000c00 tx=0x7f20b40046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 shutdown_connections 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f20ac06c530 0x7f20ac06e9f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c00ba090 0x7f20c0161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00b73d0 0x7f20c0156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 --2- 172.21.15.66:0/873862414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20c00b5dc0 0x7f20c0149a90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 >> 172.21.15.66:0/873862414 conn(0x7f20c00678f0 msgr2=0x7f20c00b8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 shutdown_connections 2024-03-30T13:10:12.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.765+0000 7f20c5493640 1 -- 172.21.15.66:0/873862414 wait complete. 2024-03-30T13:10:12.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.889+0000 7f7c5bfbd640 1 Processor -- start 2024-03-30T13:10:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.889+0000 7f7c5bfbd640 1 -- start start 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c5bfbd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7c540ba090 0x7f7c540d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c5bfbd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7c540b73d0 0x7f7c540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c5bfbd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c540d2ff0 con 0x7f7c540b5dc0 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c5bfbd640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7c540d32d0 con 0x7f7c540ba090 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c5bfbd640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7c540d35b0 con 0x7f7c540b73d0 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59558640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7c540ba090 0x7f7c540d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59d59640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7c540b5dc0 0x7f7c540b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55238/0 (socket says 172.21.15.66:55238) 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59d59640 1 -- 172.21.15.66:0/514295650 learned_addr learned my addr 172.21.15.66:0/514295650 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c5a55a640 1 --2- 172.21.15.66:0/514295650 >> v2:172.21.15.117:3300/0 conn(0x7f7c540b73d0 0x7f7c540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59558640 1 -- 172.21.15.66:0/514295650 >> v2:172.21.15.117:3300/0 conn(0x7f7c540b73d0 msgr2=0x7f7c540b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59558640 1 --2- 172.21.15.66:0/514295650 >> v2:172.21.15.117:3300/0 conn(0x7f7c540b73d0 0x7f7c540b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59558640 1 -- 172.21.15.66:0/514295650 >> v1:172.21.15.66:6789/0 conn(0x7f7c540b5dc0 legacy=0x7f7c540b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c58d57640 1 -- 172.21.15.66:0/514295650 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283664842 0 0) 0x7f7c540d2ff0 con 0x7f7c540b5dc0 2024-03-30T13:10:12.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.890+0000 7f7c59558640 1 -- 172.21.15.66:0/514295650 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c540d3890 con 0x7f7c540ba090 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.891+0000 7f7c59558640 1 --2- 172.21.15.66:0/514295650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540d2680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c440097a0 tx=0x7f7c4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165f1fe8a4d38b4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.891+0000 7f7c58d57640 1 -- 172.21.15.66:0/514295650 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44035020 con 0x7f7c540ba090 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.891+0000 7f7c58d57640 1 -- 172.21.15.66:0/514295650 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c44031830 con 0x7f7c540ba090 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.891+0000 7f7c58d57640 1 -- 172.21.15.66:0/514295650 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44031c00 con 0x7f7c540ba090 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/514295650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 msgr2=0x7f7c540d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/514295650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540d2680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c440097a0 tx=0x7f7c4402b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/514295650 shutdown_connections 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/514295650 >> v2:172.21.15.117:3300/0 conn(0x7f7c540b73d0 0x7f7c540b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/514295650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540d2680 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/514295650 >> 172.21.15.66:0/514295650 conn(0x7f7c540678f0 msgr2=0x7f7c540d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:12.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/514295650 shutdown_connections 2024-03-30T13:10:12.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.892+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/514295650 wait complete. 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.893+0000 7f7c5bfbd640 1 Processor -- start 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.893+0000 7f7c5bfbd640 1 -- start start 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5bfbd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c540b5dc0 0x7f7c540b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5bfbd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c540b73d0 0x7f7c540c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5bfbd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5bfbd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5415e880 con 0x7f7c540ba090 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5bfbd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540d4090 con 0x7f7c540b5dc0 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5bfbd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5415ea00 con 0x7f7c540b73d0 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49954/0 (socket says 172.21.15.66:49954) 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 -- 172.21.15.66:0/1596180650 learned_addr learned my addr 172.21.15.66:0/1596180650 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c59558640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c540b73d0 0x7f7c540c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c59d59640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c540b5dc0 0x7f7c540b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 -- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c540b73d0 msgr2=0x7f7c540c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c540b73d0 0x7f7c540c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 -- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c540b5dc0 msgr2=0x7f7c540b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c540b5dc0 0x7f7c540b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 -- 172.21.15.66:0/1596180650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c541610d0 con 0x7f7c540ba090 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.894+0000 7f7c5a55a640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540ca060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009dd0 tx=0x7f7c4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.895+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c018020 con 0x7f7c540ba090 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.895+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c014480 con 0x7f7c540ba090 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.895+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c541613b0 con 0x7f7c540ba090 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.895+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c014870 con 0x7f7c540ba090 2024-03-30T13:10:12.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.895+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c54161970 con 0x7f7c540ba090 2024-03-30T13:10:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.896+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c54075890 con 0x7f7c540ba090 2024-03-30T13:10:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.896+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7c4c005d10 con 0x7f7c540ba090 2024-03-30T13:10:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.896+0000 7f7c4a7fc640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7c3806c530 0x7f7c3806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.897+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7c4c08c3a0 con 0x7f7c540ba090 2024-03-30T13:10:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.897+0000 7f7c59d59640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7c3806c530 0x7f7c3806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:12.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.897+0000 7f7c59d59640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7c3806c530 0x7f7c3806e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c50000f60 tx=0x7f7c50007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:12.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:12.901+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7c4c022070 con 0x7f7c540ba090 2024-03-30T13:10:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.104+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7c5415df90 con 0x7f7c540ba090 2024-03-30T13:10:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.105+0000 7f7c4a7fc640 1 -- 172.21.15.66:0/1596180650 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f7c4c010040 con 0x7f7c540ba090 2024-03-30T13:10:13.108 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7c3806c530 msgr2=0x7f7c3806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7c3806c530 0x7f7c3806e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c50000f60 tx=0x7f7c50007d50 comp rx=0 tx=0).stop 2024-03-30T13:10:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 msgr2=0x7f7c540ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540ca060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009dd0 tx=0x7f7c4c00c490 comp rx=0 tx=0).stop 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 shutdown_connections 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7c3806c530 0x7f7c3806e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c540ba090 0x7f7c540ca060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c540b73d0 0x7f7c540c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 --2- 172.21.15.66:0/1596180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c540b5dc0 0x7f7c540b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 >> 172.21.15.66:0/1596180650 conn(0x7f7c540678f0 msgr2=0x7f7c540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 shutdown_connections 2024-03-30T13:10:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.107+0000 7f7c5bfbd640 1 -- 172.21.15.66:0/1596180650 wait complete. 2024-03-30T13:10:13.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.226+0000 7fbb3b006640 1 Processor -- start 2024-03-30T13:10:13.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.226+0000 7fbb3b006640 1 -- start start 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.227+0000 7fbb3b006640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbb3415e950 0x7fbb340d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.227+0000 7fbb3b006640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d48c0 0x7fbb340d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb3b006640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb340d36f0 con 0x7fbb340d2f00 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb3b006640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fbb340d3cb0 con 0x7fbb3415e950 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb3b006640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbb340d4d00 con 0x7fbb340d48c0 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb395a3640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbb340d2f00 0x7fbb340cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55240/0 (socket says 172.21.15.66:55240) 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb395a3640 1 -- 172.21.15.66:0/1362649338 learned_addr learned my addr 172.21.15.66:0/1362649338 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb38da2640 1 --2- 172.21.15.66:0/1362649338 >> v2:172.21.15.117:3300/0 conn(0x7fbb3415e950 0x7fbb340d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb33fff640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d48c0 0x7fbb340d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb337fe640 1 -- 172.21.15.66:0/1362649338 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20085263 0 0) 0x7fbb340d36f0 con 0x7fbb340d2f00 2024-03-30T13:10:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb337fe640 1 -- 172.21.15.66:0/1362649338 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb24003520 con 0x7fbb340d2f00 2024-03-30T13:10:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb38da2640 1 -- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d48c0 msgr2=0x7fbb340d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb38da2640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d48c0 0x7fbb340d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb38da2640 1 -- 172.21.15.66:0/1362649338 >> v1:172.21.15.66:6789/0 conn(0x7fbb340d2f00 legacy=0x7fbb340cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb38da2640 1 -- 172.21.15.66:0/1362649338 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb340d39d0 con 0x7fbb3415e950 2024-03-30T13:10:13.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.228+0000 7fbb33fff640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d48c0 0x7fbb340d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.229+0000 7fbb38da2640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb3415e950 0x7fbb340d4380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb280097a0 tx=0x7fbb2802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce27d8d47b7ccee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.229+0000 7fbb337fe640 1 -- 172.21.15.66:0/1362649338 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28034070 con 0x7fbb3415e950 2024-03-30T13:10:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.229+0000 7fbb337fe640 1 -- 172.21.15.66:0/1362649338 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb28030610 con 0x7fbb3415e950 2024-03-30T13:10:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.229+0000 7fbb337fe640 1 -- 172.21.15.66:0/1362649338 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280309e0 con 0x7fbb3415e950 2024-03-30T13:10:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 -- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb3415e950 msgr2=0x7fbb340d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb3415e950 0x7fbb340d4380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb280097a0 tx=0x7fbb2802b960 comp rx=0 tx=0).stop 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 -- 172.21.15.66:0/1362649338 shutdown_connections 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d48c0 0x7fbb340d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 --2- 172.21.15.66:0/1362649338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb3415e950 0x7fbb340d4380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 -- 172.21.15.66:0/1362649338 >> 172.21.15.66:0/1362649338 conn(0x7fbb340676e0 msgr2=0x7fbb34067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 -- 172.21.15.66:0/1362649338 shutdown_connections 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.230+0000 7fbb3b006640 1 -- 172.21.15.66:0/1362649338 wait complete. 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.232+0000 7fbb3b006640 1 Processor -- start 2024-03-30T13:10:13.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.232+0000 7fbb3b006640 1 -- start start 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb3b006640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d2f00 0x7fbb340b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb3b006640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb340d48c0 0x7fbb341f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb3b006640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 0x7fbb341e8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb3b006640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340d3390 con 0x7fbb3415e950 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb3b006640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340b7da0 con 0x7fbb340d48c0 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb3b006640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3415ed30 con 0x7fbb340d2f00 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb395a3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 0x7fbb341e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb395a3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 0x7fbb341e8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49958/0 (socket says 172.21.15.66:49958) 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb395a3640 1 -- 172.21.15.66:0/693064725 learned_addr learned my addr 172.21.15.66:0/693064725 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb33fff640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb340d48c0 0x7fbb341f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.233+0000 7fbb38da2640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d2f00 0x7fbb340b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb395a3640 1 -- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d2f00 msgr2=0x7fbb340b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb395a3640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d2f00 0x7fbb340b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb395a3640 1 -- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb340d48c0 msgr2=0x7fbb341f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb395a3640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb340d48c0 0x7fbb341f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb395a3640 1 -- 172.21.15.66:0/693064725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb341f4960 con 0x7fbb3415e950 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb33fff640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb340d48c0 0x7fbb341f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb395a3640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 0x7fbb341e8420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00cb40 tx=0x7fbb2c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c017070 con 0x7fbb3415e950 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb341f4bf0 con 0x7fbb3415e950 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00aca0 con 0x7fbb3415e950 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c012620 con 0x7fbb3415e950 2024-03-30T13:10:13.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.234+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb341f51b0 con 0x7fbb3415e950 2024-03-30T13:10:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.235+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb34075a10 con 0x7fbb3415e950 2024-03-30T13:10:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.235+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbb2c01a070 con 0x7fbb3415e950 2024-03-30T13:10:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.236+0000 7fbb317fa640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbb1806c530 0x7fbb1806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.236+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbb2c08b810 con 0x7fbb3415e950 2024-03-30T13:10:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.236+0000 7fbb38da2640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbb1806c530 0x7fbb1806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.236+0000 7fbb38da2640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbb1806c530 0x7fbb1806e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb28036000 tx=0x7fbb2803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.240+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbb2c0594b0 con 0x7fbb3415e950 2024-03-30T13:10:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.465+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb341f53f0 con 0x7fbb3415e950 2024-03-30T13:10:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.466+0000 7fbb317fa640 1 -- 172.21.15.66:0/693064725 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fbb2c089e20 con 0x7fbb3415e950 2024-03-30T13:10:13.470 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:13.470 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbb1806c530 msgr2=0x7fbb1806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbb1806c530 0x7fbb1806e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb28036000 tx=0x7fbb2803b040 comp rx=0 tx=0).stop 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 msgr2=0x7fbb341e8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 0x7fbb341e8420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00cb40 tx=0x7fbb2c00a060 comp rx=0 tx=0).stop 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 shutdown_connections 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbb1806c530 0x7fbb1806e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb3415e950 0x7fbb341e8420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb340d48c0 0x7fbb341f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 --2- 172.21.15.66:0/693064725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb340d2f00 0x7fbb340b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 >> 172.21.15.66:0/693064725 conn(0x7fbb340676e0 msgr2=0x7fbb340cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 shutdown_connections 2024-03-30T13:10:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.468+0000 7fbb3b006640 1 -- 172.21.15.66:0/693064725 wait complete. 2024-03-30T13:10:13.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.586+0000 7f782879d640 1 Processor -- start 2024-03-30T13:10:13.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.586+0000 7f782879d640 1 -- start start 2024-03-30T13:10:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f782879d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7820145090 0x7f78200b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f782879d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f78200d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f782879d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78200b5f80 con 0x7f78200b5aa0 2024-03-30T13:10:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f782879d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f78200b6260 con 0x7f7820145090 2024-03-30T13:10:13.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f782879d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f78200b6540 con 0x7f7820160540 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f7826539640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7820145090 0x7f78200b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f7826539640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7820145090 0x7f78200b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49964/0 (socket says 172.21.15.66:49964) 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f7826539640 1 -- 172.21.15.66:0/4063377201 learned_addr learned my addr 172.21.15.66:0/4063377201 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.587+0000 7f7826d3a640 1 --2- 172.21.15.66:0/4063377201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f78200d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.588+0000 7f7826539640 1 -- 172.21.15.66:0/4063377201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 msgr2=0x7f78200d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.588+0000 7f7826539640 1 --2- 172.21.15.66:0/4063377201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f78200d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.588+0000 7f7826539640 1 -- 172.21.15.66:0/4063377201 >> v1:172.21.15.117:6789/0 conn(0x7f78200b5aa0 legacy=0x7f782015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.588+0000 7f7825537640 1 -- 172.21.15.66:0/4063377201 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595932982 0 0) 0x7f78200b5f80 con 0x7f78200b5aa0 2024-03-30T13:10:13.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.588+0000 7f7826539640 1 -- 172.21.15.66:0/4063377201 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78200b6820 con 0x7f7820145090 2024-03-30T13:10:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.588+0000 7f7826539640 1 --2- 172.21.15.66:0/4063377201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f78200b1720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7818009870 tx=0x7f781802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85335e5eca25f6fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f7825537640 1 -- 172.21.15.66:0/4063377201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818035020 con 0x7f7820145090 2024-03-30T13:10:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f7825537640 1 -- 172.21.15.66:0/4063377201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818031640 con 0x7f7820145090 2024-03-30T13:10:13.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f7825537640 1 -- 172.21.15.66:0/4063377201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781802d040 con 0x7f7820145090 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 -- 172.21.15.66:0/4063377201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 msgr2=0x7f78200b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 --2- 172.21.15.66:0/4063377201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f78200b1720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7818009870 tx=0x7f781802b8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 -- 172.21.15.66:0/4063377201 shutdown_connections 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 --2- 172.21.15.66:0/4063377201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f78200d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 --2- 172.21.15.66:0/4063377201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f78200b1720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 -- 172.21.15.66:0/4063377201 >> 172.21.15.66:0/4063377201 conn(0x7f782006f8b0 msgr2=0x7f782006fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 -- 172.21.15.66:0/4063377201 shutdown_connections 2024-03-30T13:10:13.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.589+0000 7f782879d640 1 -- 172.21.15.66:0/4063377201 wait complete. 2024-03-30T13:10:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.590+0000 7f782879d640 1 Processor -- start 2024-03-30T13:10:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 -- start start 2024-03-30T13:10:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78200b5aa0 0x7f78200ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f782006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f782013b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7820145470 con 0x7f7820145090 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7820146000 con 0x7f78200b5aa0 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f782879d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f78201455f0 con 0x7f7820160540 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f7825d38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f782006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f7825d38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f782006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49976/0 (socket says 172.21.15.66:49976) 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f7825d38640 1 -- 172.21.15.66:0/2349977248 learned_addr learned my addr 172.21.15.66:0/2349977248 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f7826539640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78200b5aa0 0x7f78200ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.591+0000 7f7826d3a640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f782013b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7825d38640 1 -- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 msgr2=0x7f782013b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7825d38640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f782013b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7825d38640 1 -- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78200b5aa0 msgr2=0x7f78200ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7825d38640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78200b5aa0 0x7f78200ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7825d38640 1 -- 172.21.15.66:0/2349977248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78200aeed0 con 0x7f7820145090 2024-03-30T13:10:13.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7826539640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78200b5aa0 0x7f78200ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f7825d38640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f782006e0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f781400c520 tx=0x7f781400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814017070 con 0x7f7820145090 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78140035f0 con 0x7f7820145090 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78200af1b0 con 0x7f7820145090 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78140039e0 con 0x7f7820145090 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.592+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78200af770 con 0x7f7820145090 2024-03-30T13:10:13.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.593+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7820075a10 con 0x7f7820145090 2024-03-30T13:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.594+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f781401a070 con 0x7f7820145090 2024-03-30T13:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.594+0000 7f781effd640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f780c06c530 0x7f780c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.594+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f781408ad50 con 0x7f7820145090 2024-03-30T13:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.594+0000 7f7826539640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f780c06c530 0x7f780c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.595+0000 7f7826539640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f780c06c530 0x7f780c06e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7818036000 tx=0x7f781803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.598+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f781408e030 con 0x7f7820145090 2024-03-30T13:10:13.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.783+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78200b2120 con 0x7f7820145090 2024-03-30T13:10:13.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.785+0000 7f781effd640 1 -- 172.21.15.66:0/2349977248 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7f7814058a70 con 0x7f7820145090 2024-03-30T13:10:13.788 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:13.788 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.786+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f780c06c530 msgr2=0x7f780c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.786+0000 7f782879d640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f780c06c530 0x7f780c06e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7818036000 tx=0x7f781803c040 comp rx=0 tx=0).stop 2024-03-30T13:10:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.786+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 msgr2=0x7f782006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.786+0000 7f782879d640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f782006e0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f781400c520 tx=0x7f781400c9e0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 shutdown_connections 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f780c06c530 0x7f780c06e9f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7820160540 0x7f782013b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7820145090 0x7f782006e0b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 --2- 172.21.15.66:0/2349977248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78200b5aa0 0x7f78200ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 >> 172.21.15.66:0/2349977248 conn(0x7f782006f8b0 msgr2=0x7f782006fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 shutdown_connections 2024-03-30T13:10:13.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.787+0000 7f782879d640 1 -- 172.21.15.66:0/2349977248 wait complete. 2024-03-30T13:10:13.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:13.801 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:13.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.908+0000 7f38d5033640 1 Processor -- start 2024-03-30T13:10:13.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.908+0000 7f38d5033640 1 -- start start 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38d5033640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38d0071020 0x7f38d006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38d5033640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d009aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38d5033640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f38d0162120 0x7f38d01625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38d5033640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f38d00d0770 con 0x7f38d0162120 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38d5033640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f38d015f180 con 0x7f38d0071020 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38d5033640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f38d015f460 con 0x7f38d009aa30 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f38d0162120 0x7f38d01625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f38d0162120 0x7f38d01625b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49984/0 (socket says 172.21.15.66:49984) 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 -- 172.21.15.66:0/2498092025 learned_addr learned my addr 172.21.15.66:0/2498092025 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38ce59c640 1 --2- 172.21.15.66:0/2498092025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d009aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38ced9d640 1 --2- 172.21.15.66:0/2498092025 >> v2:172.21.15.117:3300/0 conn(0x7f38d0071020 0x7f38d006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 -- 172.21.15.66:0/2498092025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 msgr2=0x7f38d009aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:13.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 --2- 172.21.15.66:0/2498092025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d009aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 -- 172.21.15.66:0/2498092025 >> v2:172.21.15.117:3300/0 conn(0x7f38d0071020 msgr2=0x7f38d006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:13.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 --2- 172.21.15.66:0/2498092025 >> v2:172.21.15.117:3300/0 conn(0x7f38d0071020 0x7f38d006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.909+0000 7f38cf59e640 1 -- 172.21.15.66:0/2498092025 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d015f740 con 0x7f38d0162120 2024-03-30T13:10:13.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.910+0000 7f38cf59e640 1 --2- 172.21.15.66:0/2498092025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d01625b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38d00682a0 tx=0x7f38c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cce86a082997848 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.910+0000 7f38cdd9b640 1 -- 172.21.15.66:0/2498092025 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8035020 con 0x7f38d0162120 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.910+0000 7f38cdd9b640 1 -- 172.21.15.66:0/2498092025 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c8031870 con 0x7f38d0162120 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.910+0000 7f38cdd9b640 1 -- 172.21.15.66:0/2498092025 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8031c40 con 0x7f38d0162120 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 -- 172.21.15.66:0/2498092025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 msgr2=0x7f38d01625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 --2- 172.21.15.66:0/2498092025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d01625b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38d00682a0 tx=0x7f38c802b910 comp rx=0 tx=0).stop 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 -- 172.21.15.66:0/2498092025 shutdown_connections 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 --2- 172.21.15.66:0/2498092025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d01625b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 --2- 172.21.15.66:0/2498092025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d009aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 --2- 172.21.15.66:0/2498092025 >> v2:172.21.15.117:3300/0 conn(0x7f38d0071020 0x7f38d006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 -- 172.21.15.66:0/2498092025 >> 172.21.15.66:0/2498092025 conn(0x7f38d00b3ce0 msgr2=0x7f38d00b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:13.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 -- 172.21.15.66:0/2498092025 shutdown_connections 2024-03-30T13:10:13.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.911+0000 7f38d5033640 1 -- 172.21.15.66:0/2498092025 wait complete. 2024-03-30T13:10:13.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.912+0000 7f38d5033640 1 Processor -- start 2024-03-30T13:10:13.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.912+0000 7f38d5033640 1 -- start start 2024-03-30T13:10:13.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38d5033640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38d0071020 0x7f38d00b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38d5033640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d01541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38d5033640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d0154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38d5033640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38d006cd80 con 0x7f38d0162120 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38d5033640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0071400 con 0x7f38d0071020 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38d5033640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f38d006f620 con 0x7f38d009aa30 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d0154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38ced9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38d0071020 0x7f38d00b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d0154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49998/0 (socket says 172.21.15.66:49998) 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 -- 172.21.15.66:0/3788200131 learned_addr learned my addr 172.21.15.66:0/3788200131 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38ce59c640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d01541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 -- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 msgr2=0x7f38d01541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d01541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 -- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38d0071020 msgr2=0x7f38d00b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38d0071020 0x7f38d00b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38cf59e640 1 -- 172.21.15.66:0/3788200131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d00b10f0 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38ced9d640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38d0071020 0x7f38d00b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.913+0000 7f38ce59c640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d01541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.914+0000 7f38cf59e640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d0154730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38c802b950 tx=0x7f38c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.914+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8031e10 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.914+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c803ad60 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.914+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d00b0b30 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.914+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8044630 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.914+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d015bc00 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.915+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d0075890 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.915+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f38c8038020 con 0x7f38d0162120 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.916+0000 7f38b77fe640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38bc06c530 0x7f38bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:13.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.916+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f38c80ad9d0 con 0x7f38d0162120 2024-03-30T13:10:13.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.916+0000 7f38ced9d640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38bc06c530 0x7f38bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:13.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.916+0000 7f38ced9d640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38bc06c530 0x7f38bc06e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009fd0 tx=0x7f38c000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:13.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:13.920+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f38c803f070 con 0x7f38d0162120 2024-03-30T13:10:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.103+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38d00b0820 con 0x7f38d0162120 2024-03-30T13:10:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.104+0000 7f38b77fe640 1 -- 172.21.15.66:0/3788200131 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f38c807b660 con 0x7f38d0162120 2024-03-30T13:10:14.107 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38bc06c530 msgr2=0x7f38bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38bc06c530 0x7f38bc06e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009fd0 tx=0x7f38c000d040 comp rx=0 tx=0).stop 2024-03-30T13:10:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 msgr2=0x7f38d0154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d0154730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38c802b950 tx=0x7f38c80046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 shutdown_connections 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38bc06c530 0x7f38bc06e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38d0162120 0x7f38d0154730 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38d009aa30 0x7f38d01541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 --2- 172.21.15.66:0/3788200131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38d0071020 0x7f38d00b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.105+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 >> 172.21.15.66:0/3788200131 conn(0x7f38d00b3ce0 msgr2=0x7f38d015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.106+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 shutdown_connections 2024-03-30T13:10:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.106+0000 7f38d5033640 1 -- 172.21.15.66:0/3788200131 wait complete. 2024-03-30T13:10:14.120 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:14.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.227+0000 7f8272e7b640 1 Processor -- start 2024-03-30T13:10:14.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.227+0000 7f8272e7b640 1 -- start start 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8272e7b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f826c0d27a0 0x7f826c0d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8272e7b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d48a0 0x7f826c15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8272e7b640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826c0b3c30 con 0x7f826c15c000 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8272e7b640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f826c0d2c00 con 0x7f826c0d27a0 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8272e7b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0d4cf0 con 0x7f826c0d48a0 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f826bfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f826c0d27a0 0x7f826c0d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8270c17640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f826c15c000 0x7f826c0b9cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55252/0 (socket says 172.21.15.66:55252) 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8270c17640 1 -- 172.21.15.66:0/827495228 learned_addr learned my addr 172.21.15.66:0/827495228 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f8271418640 1 --2- 172.21.15.66:0/827495228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d48a0 0x7f826c15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f826bfff640 1 -- 172.21.15.66:0/827495228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d48a0 msgr2=0x7f826c15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f826bfff640 1 --2- 172.21.15.66:0/827495228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d48a0 0x7f826c15edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f826bfff640 1 -- 172.21.15.66:0/827495228 >> v1:172.21.15.66:6789/0 conn(0x7f826c15c000 legacy=0x7f826c0b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.228+0000 7f826bfff640 1 -- 172.21.15.66:0/827495228 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c162110 con 0x7f826c0d27a0 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.229+0000 7f826bfff640 1 --2- 172.21.15.66:0/827495228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d27a0 0x7f826c0d4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6054bc293501b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.229+0000 7f826b7fe640 1 -- 172.21.15.66:0/827495228 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580315a0 con 0x7f826c0d27a0 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.229+0000 7f826b7fe640 1 -- 172.21.15.66:0/827495228 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8258037070 con 0x7f826c0d27a0 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.229+0000 7f826b7fe640 1 -- 172.21.15.66:0/827495228 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258031960 con 0x7f826c0d27a0 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.229+0000 7f8272e7b640 1 -- 172.21.15.66:0/827495228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d27a0 msgr2=0x7f826c0d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.229+0000 7f8272e7b640 1 --2- 172.21.15.66:0/827495228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d27a0 0x7f826c0d4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:14.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.230+0000 7f8272e7b640 1 -- 172.21.15.66:0/827495228 shutdown_connections 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.230+0000 7f8272e7b640 1 --2- 172.21.15.66:0/827495228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d48a0 0x7f826c15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.230+0000 7f8272e7b640 1 --2- 172.21.15.66:0/827495228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d27a0 0x7f826c0d4360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.230+0000 7f8272e7b640 1 -- 172.21.15.66:0/827495228 >> 172.21.15.66:0/827495228 conn(0x7f826c0676e0 msgr2=0x7f826c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.230+0000 7f8272e7b640 1 -- 172.21.15.66:0/827495228 shutdown_connections 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.230+0000 7f8272e7b640 1 -- 172.21.15.66:0/827495228 wait complete. 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.231+0000 7f8272e7b640 1 Processor -- start 2024-03-30T13:10:14.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.231+0000 7f8272e7b640 1 -- start start 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8272e7b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d27a0 0x7f826c14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8272e7b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 0x7f826c1e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8272e7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c15c000 0x7f826c0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8272e7b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f826c162990 con 0x7f826c0d48a0 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8272e7b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1626b0 con 0x7f826c15c000 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8272e7b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f826c162c70 con 0x7f826c0d27a0 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 0x7f826c1e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 0x7f826c1e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50010/0 (socket says 172.21.15.66:50010) 2024-03-30T13:10:14.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 -- 172.21.15.66:0/120546724 learned_addr learned my addr 172.21.15.66:0/120546724 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8270c17640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d27a0 0x7f826c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8271418640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c15c000 0x7f826c0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 -- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d27a0 msgr2=0x7f826c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d27a0 0x7f826c14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 -- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c15c000 msgr2=0x7f826c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c15c000 0x7f826c0b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 -- 172.21.15.66:0/120546724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0c8df0 con 0x7f826c0d48a0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f8271418640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c15c000 0x7f826c0b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.232+0000 7f826bfff640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 0x7f826c1e9350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8258034040 tx=0x7f82580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.233+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258032630 con 0x7f826c0d48a0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.233+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82580327f0 con 0x7f826c0d48a0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.233+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c0c90d0 con 0x7f826c0d48a0 2024-03-30T13:10:14.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.233+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258032be0 con 0x7f826c0d48a0 2024-03-30T13:10:14.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.233+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c14c3f0 con 0x7f826c0d48a0 2024-03-30T13:10:14.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.234+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c075890 con 0x7f826c0d48a0 2024-03-30T13:10:14.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.234+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f825804e430 con 0x7f826c0d48a0 2024-03-30T13:10:14.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.234+0000 7f82697fa640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f825006c530 0x7f825006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.235+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f825802f030 con 0x7f826c0d48a0 2024-03-30T13:10:14.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.235+0000 7f8270c17640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f825006c530 0x7f825006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.235+0000 7f8270c17640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f825006c530 0x7f825006e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8260000f60 tx=0x7f8260007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.239+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8258040070 con 0x7f826c0d48a0 2024-03-30T13:10:14.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.425+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f826c0b3080 con 0x7f826c0d48a0 2024-03-30T13:10:14.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.426+0000 7f82697fa640 1 -- 172.21.15.66:0/120546724 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f825807b3d0 con 0x7f826c0d48a0 2024-03-30T13:10:14.429 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:14.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.427+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f825006c530 msgr2=0x7f825006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.427+0000 7f8272e7b640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f825006c530 0x7f825006e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8260000f60 tx=0x7f8260007e80 comp rx=0 tx=0).stop 2024-03-30T13:10:14.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.427+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 msgr2=0x7f826c1e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.427+0000 7f8272e7b640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 0x7f826c1e9350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8258034040 tx=0x7f82580046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 shutdown_connections 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f825006c530 0x7f825006e9f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f826c15c000 0x7f826c0b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826c0d48a0 0x7f826c1e9350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 --2- 172.21.15.66:0/120546724 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f826c0d27a0 0x7f826c14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 >> 172.21.15.66:0/120546724 conn(0x7f826c0676e0 msgr2=0x7f826c15c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 shutdown_connections 2024-03-30T13:10:14.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.428+0000 7f8272e7b640 1 -- 172.21.15.66:0/120546724 wait complete. 2024-03-30T13:10:14.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:14.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.548+0000 7f3bbcf22640 1 Processor -- start 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.548+0000 7f3bbcf22640 1 -- start start 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.548+0000 7f3bbcf22640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bb815e5b0 0x7f3bb8162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bbcf22640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bb8073310 0x7f3bb806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bbcf22640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb81630b0 con 0x7f3bb80d1be0 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bbcf22640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb8073880 con 0x7f3bb8073310 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bbcf22640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb80d0e20 con 0x7f3bb815e5b0 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bb659c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bb815e5b0 0x7f3bb8162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3badd9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bb8073310 0x7f3bb806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bb6d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3bb80d1be0 0x7f3bb80b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55264/0 (socket says 172.21.15.66:55264) 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bb6d9d640 1 -- 172.21.15.66:0/3229830565 learned_addr learned my addr 172.21.15.66:0/3229830565 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3badd9b640 1 -- 172.21.15.66:0/3229830565 >> v2:172.21.15.117:3300/0 conn(0x7f3bb815e5b0 msgr2=0x7f3bb8162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:14.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3badd9b640 1 --2- 172.21.15.66:0/3229830565 >> v2:172.21.15.117:3300/0 conn(0x7f3bb815e5b0 0x7f3bb8162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3badd9b640 1 -- 172.21.15.66:0/3229830565 >> v1:172.21.15.66:6789/0 conn(0x7f3bb80d1be0 legacy=0x7f3bb80b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3badd9b640 1 -- 172.21.15.66:0/3229830565 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80d1100 con 0x7f3bb8073310 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.549+0000 7f3bb659c640 1 --2- 172.21.15.66:0/3229830565 >> v2:172.21.15.117:3300/0 conn(0x7f3bb815e5b0 0x7f3bb8162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.550+0000 7f3badd9b640 1 --2- 172.21.15.66:0/3229830565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb806b580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80720e0 tx=0x7f3ba002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38baa9a56ac577d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.550+0000 7f3bb5d9b640 1 -- 172.21.15.66:0/3229830565 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00315a0 con 0x7f3bb8073310 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.550+0000 7f3bb5d9b640 1 -- 172.21.15.66:0/3229830565 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0037070 con 0x7f3bb8073310 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.550+0000 7f3bb5d9b640 1 -- 172.21.15.66:0/3229830565 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0031960 con 0x7f3bb8073310 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.550+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3229830565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 msgr2=0x7f3bb806b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.550+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3229830565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb806b580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80720e0 tx=0x7f3ba002b840 comp rx=0 tx=0).stop 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.551+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3229830565 shutdown_connections 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.551+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3229830565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb806b580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.551+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3229830565 >> v2:172.21.15.117:3300/0 conn(0x7f3bb815e5b0 0x7f3bb8162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.551+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3229830565 >> 172.21.15.66:0/3229830565 conn(0x7f3bb806e720 msgr2=0x7f3bb806eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:14.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.551+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3229830565 shutdown_connections 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.551+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3229830565 wait complete. 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.552+0000 7f3bbcf22640 1 Processor -- start 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.552+0000 7f3bbcf22640 1 -- start start 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bbcf22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb81f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bbcf22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bbcf22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb815e5b0 0x7f3bb8146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bbcf22640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb815e990 con 0x7f3bb8073310 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bbcf22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80d16c0 con 0x7f3bb80d1be0 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bbcf22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb806bac0 con 0x7f3bb815e5b0 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb81f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb81f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50030/0 (socket says 172.21.15.66:50030) 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 -- 172.21.15.66:0/3034295511 learned_addr learned my addr 172.21.15.66:0/3034295511 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb6d9d640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb815e5b0 0x7f3bb8146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3badd9b640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 -- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb815e5b0 msgr2=0x7f3bb8146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb815e5b0 0x7f3bb8146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 -- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb80d1be0 msgr2=0x7f3bb8161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 -- 172.21.15.66:0/3034295511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb815f0d0 con 0x7f3bb8073310 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.553+0000 7f3bb659c640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb81f8a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800e990 tx=0x7f3ba800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.554+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8017070 con 0x7f3bb8073310 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.554+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba800c9e0 con 0x7f3bb8073310 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.554+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb8160a30 con 0x7f3bb8073310 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.554+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800cdb0 con 0x7f3bb8073310 2024-03-30T13:10:14.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.554+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb8160ff0 con 0x7f3bb8073310 2024-03-30T13:10:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.555+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb815b900 con 0x7f3bb8073310 2024-03-30T13:10:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.555+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3ba801a070 con 0x7f3bb8073310 2024-03-30T13:10:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.555+0000 7f3baf7fe640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b9806c530 0x7f3b9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.556+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3ba808bba0 con 0x7f3bb8073310 2024-03-30T13:10:14.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.556+0000 7f3badd9b640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b9806c530 0x7f3b9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.556+0000 7f3badd9b640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b9806c530 0x7f3b9806e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0036000 tx=0x7f3ba003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.560+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3bb815b900 con 0x7f3bb8073310 2024-03-30T13:10:14.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.743+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3bb8075890 con 0x7f3bb8073310 2024-03-30T13:10:14.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.744+0000 7f3baf7fe640 1 -- 172.21.15.66:0/3034295511 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f3ba808b020 con 0x7f3bb8073310 2024-03-30T13:10:14.748 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b9806c530 msgr2=0x7f3b9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b9806c530 0x7f3b9806e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0036000 tx=0x7f3ba003d040 comp rx=0 tx=0).stop 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 msgr2=0x7f3bb81f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb81f8a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800e990 tx=0x7f3ba800ee50 comp rx=0 tx=0).stop 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 shutdown_connections 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3b9806c530 0x7f3b9806e9f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb815e5b0 0x7f3bb8146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb80d1be0 0x7f3bb8161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 --2- 172.21.15.66:0/3034295511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8073310 0x7f3bb81f8a70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 >> 172.21.15.66:0/3034295511 conn(0x7f3bb806e720 msgr2=0x7f3bb80b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:14.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 shutdown_connections 2024-03-30T13:10:14.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.746+0000 7f3bbcf22640 1 -- 172.21.15.66:0/3034295511 wait complete. 2024-03-30T13:10:14.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.868+0000 7f9ceaa8f640 1 Processor -- start 2024-03-30T13:10:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.868+0000 7f9ceaa8f640 1 -- start start 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ceaa8f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ce40d27a0 0x7f9ce40d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ceaa8f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ce40d48a0 0x7f9ce415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ceaa8f640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce40b3c30 con 0x7f9ce415c000 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ceaa8f640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce40d2c00 con 0x7f9ce40d27a0 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ceaa8f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce40d4cf0 con 0x7f9ce40d48a0 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce3fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ce40d27a0 0x7f9ce40d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce882b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9ce415c000 0x7f9ce40b9cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55272/0 (socket says 172.21.15.66:55272) 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce882b640 1 -- 172.21.15.66:0/265738893 learned_addr learned my addr 172.21.15.66:0/265738893 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce902c640 1 --2- 172.21.15.66:0/265738893 >> v2:172.21.15.117:3300/0 conn(0x7f9ce40d48a0 0x7f9ce415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce3fff640 1 -- 172.21.15.66:0/265738893 >> v2:172.21.15.117:3300/0 conn(0x7f9ce40d48a0 msgr2=0x7f9ce415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:14.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce3fff640 1 --2- 172.21.15.66:0/265738893 >> v2:172.21.15.117:3300/0 conn(0x7f9ce40d48a0 0x7f9ce415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce3fff640 1 -- 172.21.15.66:0/265738893 >> v1:172.21.15.66:6789/0 conn(0x7f9ce415c000 legacy=0x7f9ce40b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.869+0000 7f9ce3fff640 1 -- 172.21.15.66:0/265738893 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce4162110 con 0x7f9ce40d27a0 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.870+0000 7f9ce3fff640 1 --2- 172.21.15.66:0/265738893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40d4360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40cb420 tx=0x7f9cd802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa0f39dcf3418e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.870+0000 7f9ce37fe640 1 -- 172.21.15.66:0/265738893 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd80315a0 con 0x7f9ce40d27a0 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.870+0000 7f9ce37fe640 1 -- 172.21.15.66:0/265738893 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd8037070 con 0x7f9ce40d27a0 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.870+0000 7f9ce37fe640 1 -- 172.21.15.66:0/265738893 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8031960 con 0x7f9ce40d27a0 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/265738893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d27a0 msgr2=0x7f9ce40d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/265738893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40d4360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40cb420 tx=0x7f9cd802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/265738893 shutdown_connections 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/265738893 >> v2:172.21.15.117:3300/0 conn(0x7f9ce40d48a0 0x7f9ce415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/265738893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40d4360 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/265738893 >> 172.21.15.66:0/265738893 conn(0x7f9ce40676e0 msgr2=0x7f9ce4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:14.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/265738893 shutdown_connections 2024-03-30T13:10:14.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.871+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/265738893 wait complete. 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.872+0000 7f9ceaa8f640 1 Processor -- start 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.872+0000 7f9ceaa8f640 1 -- start start 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ceaa8f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ceaa8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 0x7f9ce414c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ceaa8f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ce415c000 0x7f9ce40c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ceaa8f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4162990 con 0x7f9ce40d48a0 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ceaa8f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41626b0 con 0x7f9ce40d27a0 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ceaa8f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce4162c70 con 0x7f9ce415c000 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 0x7f9ce414c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 0x7f9ce414c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50048/0 (socket says 172.21.15.66:50048) 2024-03-30T13:10:14.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 -- 172.21.15.66:0/1221715606 learned_addr learned my addr 172.21.15.66:0/1221715606 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce902c640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ce415c000 0x7f9ce40c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce882b640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 -- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ce415c000 msgr2=0x7f9ce40c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ce415c000 0x7f9ce40c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 -- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce40d27a0 msgr2=0x7f9ce40b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce3fff640 1 -- 172.21.15.66:0/1221715606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce40c90e0 con 0x7f9ce40d48a0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.873+0000 7f9ce902c640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ce415c000 0x7f9ce40c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.874+0000 7f9ce3fff640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 0x7f9ce414c630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009890 tx=0x7f9cd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.874+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8032630 con 0x7f9ce40d48a0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.874+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd80327f0 con 0x7f9ce40d48a0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.874+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce40b4530 con 0x7f9ce40d48a0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.874+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8032be0 con 0x7f9ce40d48a0 2024-03-30T13:10:14.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.874+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce40b4af0 con 0x7f9ce40d48a0 2024-03-30T13:10:14.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.875+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce4075890 con 0x7f9ce40d48a0 2024-03-30T13:10:14.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.876+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9cd804c020 con 0x7f9ce40d48a0 2024-03-30T13:10:14.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.876+0000 7f9ce17fa640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9cd006c530 0x7f9cd006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:14.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.876+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9cd802f020 con 0x7f9ce40d48a0 2024-03-30T13:10:14.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.876+0000 7f9ce882b640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9cd006c530 0x7f9cd006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:14.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.876+0000 7f9ce882b640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9cd006c530 0x7f9cd006e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0012e0 tx=0x7f9cdc001a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:14.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:14.880+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9cd8040070 con 0x7f9ce40d48a0 2024-03-30T13:10:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.068+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9ce40d4570 con 0x7f9ce40d48a0 2024-03-30T13:10:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.068+0000 7f9ce17fa640 1 -- 172.21.15.66:0/1221715606 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f9cd807b290 con 0x7f9ce40d48a0 2024-03-30T13:10:15.072 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9cd006c530 msgr2=0x7f9cd006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9cd006c530 0x7f9cd006e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0012e0 tx=0x7f9cdc001a30 comp rx=0 tx=0).stop 2024-03-30T13:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 msgr2=0x7f9ce414c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 0x7f9ce414c630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009890 tx=0x7f9cd80046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 shutdown_connections 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9cd006c530 0x7f9cd006e9f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ce415c000 0x7f9ce40c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ce40d48a0 0x7f9ce414c630 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 --2- 172.21.15.66:0/1221715606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce40d27a0 0x7f9ce40b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 >> 172.21.15.66:0/1221715606 conn(0x7f9ce40676e0 msgr2=0x7f9ce415c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.070+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 shutdown_connections 2024-03-30T13:10:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.071+0000 7f9ceaa8f640 1 -- 172.21.15.66:0/1221715606 wait complete. 2024-03-30T13:10:15.085 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:15.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.190+0000 7fc27cfa8640 1 Processor -- start 2024-03-30T13:10:15.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.190+0000 7fc27cfa8640 1 -- start start 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27cfa8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc27815e4f0 0x7fc2780cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27cfa8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2780b79c0 0x7fc2780d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27cfa8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780b9d20 con 0x7fc2780b7530 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27cfa8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc2780ba000 con 0x7fc27815e4f0 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27cfa8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc2780ba2e0 con 0x7fc2780b79c0 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27659c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc27815e4f0 0x7fc2780cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc276d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2780b79c0 0x7fc2780d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc275d9b640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc2780b7530 0x7fc2780d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:40776/0 (socket says 172.21.15.66:40776) 2024-03-30T13:10:15.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc275d9b640 1 -- 172.21.15.66:0/644286879 learned_addr learned my addr 172.21.15.66:0/644286879 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:15.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27659c640 1 -- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2780b79c0 msgr2=0x7fc2780d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:15.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27659c640 1 --2- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2780b79c0 0x7fc2780d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27659c640 1 -- 172.21.15.66:0/644286879 >> v1:172.21.15.117:6789/0 conn(0x7fc2780b7530 legacy=0x7fc2780d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.192+0000 7fc27659c640 1 -- 172.21.15.66:0/644286879 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2780ceaf0 con 0x7fc27815e4f0 2024-03-30T13:10:15.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.191+0000 7fc27559a640 1 -- 172.21.15.66:0/644286879 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538526708 0 0) 0x7fc2780b9d20 con 0x7fc2780b7530 2024-03-30T13:10:15.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.192+0000 7fc27659c640 1 --2- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27815e4f0 0x7fc2780cf460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72407e2d87377f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.192+0000 7fc27559a640 1 -- 172.21.15.66:0/644286879 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268039070 con 0x7fc27815e4f0 2024-03-30T13:10:15.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.192+0000 7fc27559a640 1 -- 172.21.15.66:0/644286879 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26802cbc0 con 0x7fc27815e4f0 2024-03-30T13:10:15.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.192+0000 7fc27559a640 1 -- 172.21.15.66:0/644286879 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268034440 con 0x7fc27815e4f0 2024-03-30T13:10:15.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 -- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27815e4f0 msgr2=0x7fc2780cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27815e4f0 0x7fc2780cf460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26802be10 comp rx=0 tx=0).stop 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 -- 172.21.15.66:0/644286879 shutdown_connections 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2780b79c0 0x7fc2780d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/644286879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27815e4f0 0x7fc2780cf460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 -- 172.21.15.66:0/644286879 >> 172.21.15.66:0/644286879 conn(0x7fc2780676e0 msgr2=0x7fc278067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 -- 172.21.15.66:0/644286879 shutdown_connections 2024-03-30T13:10:15.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.193+0000 7fc27cfa8640 1 -- 172.21.15.66:0/644286879 wait complete. 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.194+0000 7fc27cfa8640 1 Processor -- start 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.194+0000 7fc27cfa8640 1 -- start start 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27cfa8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 0x7fc2780b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27cfa8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2780b79c0 0x7fc2780b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27cfa8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27815e4f0 0x7fc2780c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27cfa8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc27815ea50 con 0x7fc2780b7530 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27cfa8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc27815e8d0 con 0x7fc2780b79c0 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27cfa8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc2780d30c0 con 0x7fc27815e4f0 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 0x7fc2780b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 0x7fc2780b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50054/0 (socket says 172.21.15.66:50054) 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc27659c640 1 -- 172.21.15.66:0/1033302232 learned_addr learned my addr 172.21.15.66:0/1033302232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc275d9b640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2780b79c0 0x7fc2780b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.195+0000 7fc276d9d640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27815e4f0 0x7fc2780c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.196+0000 7fc27659c640 1 -- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27815e4f0 msgr2=0x7fc2780c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.196+0000 7fc27659c640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27815e4f0 0x7fc2780c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.196+0000 7fc27659c640 1 -- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2780b79c0 msgr2=0x7fc2780b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.196+0000 7fc27659c640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2780b79c0 0x7fc2780b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.196+0000 7fc27659c640 1 -- 172.21.15.66:0/1033302232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2781f4040 con 0x7fc2780b7530 2024-03-30T13:10:15.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.196+0000 7fc27659c640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 0x7fc2780b5690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc268037040 tx=0x7fc268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.198+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268039050 con 0x7fc2780b7530 2024-03-30T13:10:15.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.199+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2780c7f30 con 0x7fc2780b7530 2024-03-30T13:10:15.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.199+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26803a040 con 0x7fc2780b7530 2024-03-30T13:10:15.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.199+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268009450 con 0x7fc2780b7530 2024-03-30T13:10:15.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.199+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2780b6640 con 0x7fc2780b7530 2024-03-30T13:10:15.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.199+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc278075a10 con 0x7fc2780b7530 2024-03-30T13:10:15.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.200+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc268047020 con 0x7fc2780b7530 2024-03-30T13:10:15.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.200+0000 7fc26effd640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc25806c530 0x7fc25806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.201+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc2680adbe0 con 0x7fc2780b7530 2024-03-30T13:10:15.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.201+0000 7fc275d9b640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc25806c530 0x7fc25806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.201+0000 7fc275d9b640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc25806c530 0x7fc25806e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc278161bd0 tx=0x7fc260009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.205+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc26804e020 con 0x7fc2780b7530 2024-03-30T13:10:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.427+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2780ce810 con 0x7fc2780b7530 2024-03-30T13:10:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.428+0000 7fc26effd640 1 -- 172.21.15.66:0/1033302232 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fc26807b7f0 con 0x7fc2780b7530 2024-03-30T13:10:15.432 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:15.432 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:15.432 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc25806c530 msgr2=0x7fc25806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc25806c530 0x7fc25806e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc278161bd0 tx=0x7fc260009040 comp rx=0 tx=0).stop 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 msgr2=0x7fc2780b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 0x7fc2780b5690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc268037040 tx=0x7fc268004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 shutdown_connections 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc25806c530 0x7fc25806e9f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27815e4f0 0x7fc2780c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2780b79c0 0x7fc2780b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 --2- 172.21.15.66:0/1033302232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2780b7530 0x7fc2780b5690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 >> 172.21.15.66:0/1033302232 conn(0x7fc2780676e0 msgr2=0x7fc2780d3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 shutdown_connections 2024-03-30T13:10:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.430+0000 7fc27cfa8640 1 -- 172.21.15.66:0/1033302232 wait complete. 2024-03-30T13:10:15.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:15.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.548+0000 7fdbbe1db640 1 Processor -- start 2024-03-30T13:10:15.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.548+0000 7fdbbe1db640 1 -- start start 2024-03-30T13:10:15.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbbe1db640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdbb80d1390 0x7fdbb80d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbbe1db640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80ba3d0 con 0x7fdbb80b3570 2024-03-30T13:10:15.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbbe1db640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80d1cb0 con 0x7fdbb80b39d0 2024-03-30T13:10:15.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbbe1db640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb80cf6e0 con 0x7fdbb80d1390 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbb6ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdbb80b3570 0x7fdbb80b7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55276/0 (socket says 172.21.15.66:55276) 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbb6ffd640 1 -- 172.21.15.66:0/2705132583 learned_addr learned my addr 172.21.15.66:0/2705132583 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbb77fe640 1 --2- 172.21.15.66:0/2705132583 >> v2:172.21.15.117:3300/0 conn(0x7fdbb80d1390 0x7fdbb80d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbb67fc640 1 -- 172.21.15.66:0/2705132583 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821609870 0 0) 0x7fdbb80ba3d0 con 0x7fdbb80b3570 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.549+0000 7fdbb67fc640 1 -- 172.21.15.66:0/2705132583 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba0003520 con 0x7fdbb80b3570 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.550+0000 7fdbb77fe640 1 -- 172.21.15.66:0/2705132583 >> v1:172.21.15.117:6789/0 conn(0x7fdbb80b39d0 legacy=0x7fdbb80b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.550+0000 7fdbb67fc640 1 -- 172.21.15.66:0/2705132583 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101703681 0 0) 0x7fdbb80d1cb0 con 0x7fdbb80b39d0 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.550+0000 7fdbb77fe640 1 -- 172.21.15.66:0/2705132583 >> v1:172.21.15.66:6789/0 conn(0x7fdbb80b3570 legacy=0x7fdbb80b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.550+0000 7fdbb77fe640 1 -- 172.21.15.66:0/2705132583 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb80b9050 con 0x7fdbb80d1390 2024-03-30T13:10:15.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.550+0000 7fdbb77fe640 1 --2- 172.21.15.66:0/2705132583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80d1390 0x7fdbb80d1770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdba80097a0 tx=0x7fdba802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c841d78a94578b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbb67fc640 1 -- 172.21.15.66:0/2705132583 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba802f810 con 0x7fdbb80d1390 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbb67fc640 1 -- 172.21.15.66:0/2705132583 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba802f9b0 con 0x7fdbb80d1390 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbb67fc640 1 -- 172.21.15.66:0/2705132583 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba802fd80 con 0x7fdbb80d1390 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 -- 172.21.15.66:0/2705132583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80d1390 msgr2=0x7fdbb80d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/2705132583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80d1390 0x7fdbb80d1770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdba80097a0 tx=0x7fdba802b9f0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 -- 172.21.15.66:0/2705132583 shutdown_connections 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/2705132583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80d1390 0x7fdbb80d1770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 -- 172.21.15.66:0/2705132583 >> 172.21.15.66:0/2705132583 conn(0x7fdbb80676e0 msgr2=0x7fdbb8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 -- 172.21.15.66:0/2705132583 shutdown_connections 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.551+0000 7fdbbe1db640 1 -- 172.21.15.66:0/2705132583 wait complete. 2024-03-30T13:10:15.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.552+0000 7fdbbe1db640 1 Processor -- start 2024-03-30T13:10:15.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.552+0000 7fdbbe1db640 1 -- start start 2024-03-30T13:10:15.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbbe1db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80b3570 0x7fdbb80ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbbe1db640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbb80b39d0 0x7fdbb81597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbbe1db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 0x7fdbb815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbbe1db640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb80b98f0 con 0x7fdbb80d1390 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbbe1db640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb80b9610 con 0x7fdbb80b3570 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbbe1db640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb80b9bd0 con 0x7fdbb80b39d0 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 0x7fdbb815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbb7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 0x7fdbb815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50062/0 (socket says 172.21.15.66:50062) 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbb7fff640 1 -- 172.21.15.66:0/1484300971 learned_addr learned my addr 172.21.15.66:0/1484300971 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbb77fe640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80b3570 0x7fdbb80ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.553+0000 7fdbb6ffd640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbb80b39d0 0x7fdbb81597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbb7fff640 1 -- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbb80b39d0 msgr2=0x7fdbb81597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbb7fff640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbb80b39d0 0x7fdbb81597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbb7fff640 1 -- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80b3570 msgr2=0x7fdbb80ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbb7fff640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80b3570 0x7fdbb80ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbb7fff640 1 -- 172.21.15.66:0/1484300971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb81618c0 con 0x7fdbb80d1390 2024-03-30T13:10:15.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbb7fff640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 0x7fdbb815e840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000c760 tx=0x7fdbb000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0003da0 con 0x7fdbb80d1390 2024-03-30T13:10:15.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb0005070 con 0x7fdbb80d1390 2024-03-30T13:10:15.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb81615e0 con 0x7fdbb80d1390 2024-03-30T13:10:15.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0005460 con 0x7fdbb80d1390 2024-03-30T13:10:15.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.554+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb80c9fe0 con 0x7fdbb80d1390 2024-03-30T13:10:15.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.555+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb8075a10 con 0x7fdbb80d1390 2024-03-30T13:10:15.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.555+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdbb0005600 con 0x7fdbb80d1390 2024-03-30T13:10:15.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.556+0000 7fdb9ffff640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdba406c530 0x7fdba406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.556+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdbb008b810 con 0x7fdbb80d1390 2024-03-30T13:10:15.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.556+0000 7fdbb77fe640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdba406c530 0x7fdba406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.556+0000 7fdbb77fe640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdba406c530 0x7fdba406e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdba8036000 tx=0x7fdba8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.560+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdbb00594a0 con 0x7fdbb80d1390 2024-03-30T13:10:15.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.749+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbb8161a90 con 0x7fdbb80d1390 2024-03-30T13:10:15.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.750+0000 7fdb9ffff640 1 -- 172.21.15.66:0/1484300971 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18502 (secure 0 0 0) 0x7fdbb0089e20 con 0x7fdbb80d1390 2024-03-30T13:10:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:15.754 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:09:49.128502+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdba406c530 msgr2=0x7fdba406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdba406c530 0x7fdba406e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdba8036000 tx=0x7fdba8008030 comp rx=0 tx=0).stop 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 msgr2=0x7fdbb815e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 0x7fdbb815e840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000c760 tx=0x7fdbb000cc20 comp rx=0 tx=0).stop 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 shutdown_connections 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdba406c530 0x7fdba406e9f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbb80d1390 0x7fdbb815e840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdbb80b39d0 0x7fdbb81597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 --2- 172.21.15.66:0/1484300971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbb80b3570 0x7fdbb80ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 >> 172.21.15.66:0/1484300971 conn(0x7fdbb80676e0 msgr2=0x7fdbb80d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:15.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 shutdown_connections 2024-03-30T13:10:15.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.752+0000 7fdbbe1db640 1 -- 172.21.15.66:0/1484300971 wait complete. 2024-03-30T13:10:15.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:15.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.870+0000 7f0b34196640 1 Processor -- start 2024-03-30T13:10:15.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.871+0000 7f0b34196640 1 -- start start 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.871+0000 7f0b34196640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b2c15e950 0x7f0b2c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.871+0000 7f0b34196640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0d30d0 0x7f0b2c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.871+0000 7f0b34196640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2c0d36f0 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.871+0000 7f0b34196640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f0b2c0d3cb0 con 0x7f0b2c15e950 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.871+0000 7f0b34196640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2c0d4d00 con 0x7f0b2c0d30d0 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b31731640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0b2c0d48c0 0x7f0b2c0d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55280/0 (socket says 172.21.15.66:55280) 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b31731640 1 -- 172.21.15.66:0/1294501975 learned_addr learned my addr 172.21.15.66:0/1294501975 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b32733640 1 --2- 172.21.15.66:0/1294501975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0d30d0 0x7f0b2c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b31f32640 1 --2- 172.21.15.66:0/1294501975 >> v2:172.21.15.117:3300/0 conn(0x7f0b2c15e950 0x7f0b2c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b30f30640 1 -- 172.21.15.66:0/1294501975 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3168536813 0 0) 0x7f0b2c0d36f0 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b30f30640 1 -- 172.21.15.66:0/1294501975 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b18003520 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b32733640 1 -- 172.21.15.66:0/1294501975 >> v2:172.21.15.117:3300/0 conn(0x7f0b2c15e950 msgr2=0x7f0b2c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b32733640 1 --2- 172.21.15.66:0/1294501975 >> v2:172.21.15.117:3300/0 conn(0x7f0b2c15e950 0x7f0b2c0d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b32733640 1 -- 172.21.15.66:0/1294501975 >> v1:172.21.15.66:6789/0 conn(0x7f0b2c0d48c0 legacy=0x7f0b2c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b32733640 1 -- 172.21.15.66:0/1294501975 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c0d39d0 con 0x7f0b2c0d30d0 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.872+0000 7f0b31f32640 1 --2- 172.21.15.66:0/1294501975 >> v2:172.21.15.117:3300/0 conn(0x7f0b2c15e950 0x7f0b2c0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:15.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.873+0000 7f0b32733640 1 --2- 172.21.15.66:0/1294501975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0d30d0 0x7f0b2c0b82c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009d90 tx=0x7f0b2402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1e05dcb1b04707 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.873+0000 7f0b30f30640 1 -- 172.21.15.66:0/1294501975 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24039070 con 0x7f0b2c0d30d0 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.873+0000 7f0b30f30640 1 -- 172.21.15.66:0/1294501975 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b2402cce0 con 0x7f0b2c0d30d0 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.873+0000 7f0b30f30640 1 -- 172.21.15.66:0/1294501975 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24034660 con 0x7f0b2c0d30d0 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 -- 172.21.15.66:0/1294501975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0d30d0 msgr2=0x7f0b2c0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 --2- 172.21.15.66:0/1294501975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0d30d0 0x7f0b2c0b82c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009d90 tx=0x7f0b2402bf30 comp rx=0 tx=0).stop 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 -- 172.21.15.66:0/1294501975 shutdown_connections 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 --2- 172.21.15.66:0/1294501975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0d30d0 0x7f0b2c0b82c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 --2- 172.21.15.66:0/1294501975 >> v2:172.21.15.117:3300/0 conn(0x7f0b2c15e950 0x7f0b2c0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 -- 172.21.15.66:0/1294501975 >> 172.21.15.66:0/1294501975 conn(0x7f0b2c0676e0 msgr2=0x7f0b2c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 -- 172.21.15.66:0/1294501975 shutdown_connections 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.874+0000 7f0b34196640 1 -- 172.21.15.66:0/1294501975 wait complete. 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.875+0000 7f0b34196640 1 Processor -- start 2024-03-30T13:10:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.875+0000 7f0b34196640 1 -- start start 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c0d30d0 0x7f0b2c1f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 0x7f0b2c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c15e950 0x7f0b2c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c15ed30 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c0b7da0 con 0x7f0b2c0d30d0 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2c15eeb0 con 0x7f0b2c15e950 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 0x7f0b2c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31f32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c0d30d0 0x7f0b2c1f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 0x7f0b2c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50064/0 (socket says 172.21.15.66:50064) 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 -- 172.21.15.66:0/2317376803 learned_addr learned my addr 172.21.15.66:0/2317376803 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b32733640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c15e950 0x7f0b2c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 -- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c15e950 msgr2=0x7f0b2c15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c15e950 0x7f0b2c15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 -- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c0d30d0 msgr2=0x7f0b2c1f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c0d30d0 0x7f0b2c1f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 -- 172.21.15.66:0/2317376803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c0c7f00 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31f32640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c0d30d0 0x7f0b2c1f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b31731640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 0x7f0b2c15bfc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c430 tx=0x7f0b2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28017070 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b28005ce0 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280072f0 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.876+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c1f7370 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.877+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c148350 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.877+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2c075a10 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.878+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b2801a070 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.878+0000 7f0b227fc640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b1006c530 0x7f0b1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.878+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0b2808aef0 con 0x7f0b2c0d48c0 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.878+0000 7f0b31f32640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b1006c530 0x7f0b1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.879+0000 7f0b31f32640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b1006c530 0x7f0b1006e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009560 tx=0x7f0b1c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:15.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:15.883+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0b2c0c7f00 con 0x7f0b2c0d48c0 2024-03-30T13:10:16.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.106+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b2c06b580 con 0x7f0b2c0d48c0 2024-03-30T13:10:16.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.106+0000 7f0b227fc640 1 -- 172.21.15.66:0/2317376803 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f0b28058b00 con 0x7f0b2c0d48c0 2024-03-30T13:10:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:16.110 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:16.110 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:16.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b1006c530 msgr2=0x7f0b1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b1006c530 0x7f0b1006e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009560 tx=0x7f0b1c009190 comp rx=0 tx=0).stop 2024-03-30T13:10:16.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 msgr2=0x7f0b2c15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 0x7f0b2c15bfc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c430 tx=0x7f0b2800c8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 shutdown_connections 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b1006c530 0x7f0b1006e9f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c15e950 0x7f0b2c15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b2c0d48c0 0x7f0b2c15bfc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 --2- 172.21.15.66:0/2317376803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c0d30d0 0x7f0b2c1f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.108+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 >> 172.21.15.66:0/2317376803 conn(0x7f0b2c0676e0 msgr2=0x7f0b2c0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.109+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 shutdown_connections 2024-03-30T13:10:16.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.109+0000 7f0b34196640 1 -- 172.21.15.66:0/2317376803 wait complete. 2024-03-30T13:10:16.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T13:09:42.480275+0000', 'modified': '2024-03-30T13:10:04.521329+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/4158016355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6833', 'nonce': 4158016355}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 4158016355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4290, 'qdb_cluster': [4290]} 2024-03-30T13:10:16.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T13:10:16.526 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-30T13:10:16.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.632+0000 7fa48d28f640 1 Processor -- start 2024-03-30T13:10:16.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.632+0000 7fa48d28f640 1 -- start start 2024-03-30T13:10:16.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa48d28f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa48815fe10 0x7fa4880b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa48d28f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880b4130 0x7fa48815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa48d28f640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4880cced0 con 0x7fa48815c260 2024-03-30T13:10:16.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa48d28f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa488071110 con 0x7fa48815fe10 2024-03-30T13:10:16.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa48d28f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa4880713f0 con 0x7fa4880b4130 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa486ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa48815c260 0x7fa4880cc7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55284/0 (socket says 172.21.15.66:55284) 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa486ffd640 1 -- 172.21.15.66:0/3870876142 learned_addr learned my addr 172.21.15.66:0/3870876142 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa4867fc640 1 --2- 172.21.15.66:0/3870876142 >> v2:172.21.15.117:3300/0 conn(0x7fa48815fe10 0x7fa4880b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa4877fe640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880b4130 0x7fa48815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa485ffb640 1 -- 172.21.15.66:0/3870876142 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619170221 0 0) 0x7fa4880cced0 con 0x7fa48815c260 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa485ffb640 1 -- 172.21.15.66:0/3870876142 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa474003520 con 0x7fa48815c260 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa4867fc640 1 -- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880b4130 msgr2=0x7fa48815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa4867fc640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880b4130 0x7fa48815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa4867fc640 1 -- 172.21.15.66:0/3870876142 >> v1:172.21.15.66:6789/0 conn(0x7fa48815c260 legacy=0x7fa4880cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.633+0000 7fa4867fc640 1 -- 172.21.15.66:0/3870876142 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4880716d0 con 0x7fa48815fe10 2024-03-30T13:10:16.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.634+0000 7fa4877fe640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880b4130 0x7fa48815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.634+0000 7fa4867fc640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48815fe10 0x7fa4880b3bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa4700097a0 tx=0x7fa47002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1fba87279f4b07a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.634+0000 7fa485ffb640 1 -- 172.21.15.66:0/3870876142 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470035020 con 0x7fa48815fe10 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.634+0000 7fa485ffb640 1 -- 172.21.15.66:0/3870876142 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470031830 con 0x7fa48815fe10 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.634+0000 7fa485ffb640 1 -- 172.21.15.66:0/3870876142 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470031c00 con 0x7fa48815fe10 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 -- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48815fe10 msgr2=0x7fa4880b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48815fe10 0x7fa4880b3bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa4700097a0 tx=0x7fa47002b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 -- 172.21.15.66:0/3870876142 shutdown_connections 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880b4130 0x7fa48815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 --2- 172.21.15.66:0/3870876142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48815fe10 0x7fa4880b3bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 -- 172.21.15.66:0/3870876142 >> 172.21.15.66:0/3870876142 conn(0x7fa4880cebf0 msgr2=0x7fa4880cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 -- 172.21.15.66:0/3870876142 shutdown_connections 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.635+0000 7fa48d28f640 1 -- 172.21.15.66:0/3870876142 wait complete. 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.636+0000 7fa48d28f640 1 Processor -- start 2024-03-30T13:10:16.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.636+0000 7fa48d28f640 1 -- start start 2024-03-30T13:10:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa48d28f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4880b4130 0x7fa4880b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa48d28f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 0x7fa488156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa48d28f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48815fe10 0x7fa4880b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa48d28f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa48815c640 con 0x7fa48815c260 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa48d28f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa488071c90 con 0x7fa4880b4130 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa48d28f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa4880ce640 con 0x7fa48815fe10 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 0x7fa488156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4880b4130 0x7fa4880b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 0x7fa488156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50078/0 (socket says 172.21.15.66:50078) 2024-03-30T13:10:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa4867fc640 1 -- 172.21.15.66:0/1736551911 learned_addr learned my addr 172.21.15.66:0/1736551911 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.637+0000 7fa4877fe640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48815fe10 0x7fa4880b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa4867fc640 1 -- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48815fe10 msgr2=0x7fa4880b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa4867fc640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48815fe10 0x7fa4880b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa4867fc640 1 -- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4880b4130 msgr2=0x7fa4880b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa4867fc640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4880b4130 0x7fa4880b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa4867fc640 1 -- 172.21.15.66:0/1736551911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4880b7af0 con 0x7fa48815c260 2024-03-30T13:10:16.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa486ffd640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4880b4130 0x7fa4880b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:16.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa4867fc640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 0x7fa488156ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa470034040 tx=0x7fa4700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa47f7fe640 1 -- 172.21.15.66:0/1736551911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470040070 con 0x7fa48815c260 2024-03-30T13:10:16.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa47f7fe640 1 -- 172.21.15.66:0/1736551911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47003a850 con 0x7fa48815c260 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4881570e0 con 0x7fa48815c260 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa47f7fe640 1 -- 172.21.15.66:0/1736551911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470035040 con 0x7fa48815c260 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.638+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4880b6ad0 con 0x7fa48815c260 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.640+0000 7fa47f7fe640 1 -- 172.21.15.66:0/1736551911 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa470038020 con 0x7fa48815c260 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.640+0000 7fa47f7fe640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa46806c530 0x7fa46806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.640+0000 7fa486ffd640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa46806c530 0x7fa46806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.640+0000 7fa47f7fe640 1 -- 172.21.15.66:0/1736551911 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa4700ad040 con 0x7fa48815c260 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.641+0000 7fa486ffd640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa46806c530 0x7fa46806e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa480005880 tx=0x7fa480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.641+0000 7fa48fe7f740 1 Processor -- start 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.641+0000 7fa48fe7f740 1 -- start start 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335c860 0x55f4f332f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 0x55f4f335f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f332fd40 0x55f4f335fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4f3299020 con 0x55f4f335cc40 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4f32415f0 con 0x55f4f335c860 2024-03-30T13:10:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55f4f3294fd0 con 0x55f4f332fd40 2024-03-30T13:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 0x55f4f335f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 0x55f4f335f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50080/0 (socket says 172.21.15.66:50080) 2024-03-30T13:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 -- 172.21.15.66:0/722492073 learned_addr learned my addr 172.21.15.66:0/722492073 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335c860 0x55f4f332f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4877fe640 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f332fd40 0x55f4f335fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 -- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f332fd40 msgr2=0x55f4f335fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f332fd40 0x55f4f335fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 -- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335c860 msgr2=0x55f4f332f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335c860 0x55f4f332f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.642+0000 7fa4867fc640 1 -- 172.21.15.66:0/722492073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f2fad2d0 con 0x55f4f335cc40 2024-03-30T13:10:16.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa4867fc640 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 0x55f4f335f5f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4700ab2c0 tx=0x7fa47007a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bf630504ff7a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa47cff9640 1 -- 172.21.15.66:0/722492073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47007f060 con 0x55f4f335cc40 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa47cff9640 1 -- 172.21.15.66:0/722492073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470083890 con 0x55f4f335cc40 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa47cff9640 1 -- 172.21.15.66:0/722492073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470078fb0 con 0x55f4f335cc40 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa48fe7f740 1 -- 172.21.15.66:0/722492073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 msgr2=0x55f4f335f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 0x55f4f335f5f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4700ab2c0 tx=0x7fa47007a610 comp rx=0 tx=0).stop 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa48fe7f740 1 -- 172.21.15.66:0/722492073 shutdown_connections 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f332fd40 0x55f4f335fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335cc40 0x55f4f335f5f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.643+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/722492073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335c860 0x55f4f332f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.644+0000 7fa48fe7f740 1 -- 172.21.15.66:0/722492073 >> 172.21.15.66:0/722492073 conn(0x55f4f3331a70 msgr2=0x55f4f337a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.644+0000 7fa48fe7f740 1 -- 172.21.15.66:0/722492073 shutdown_connections 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.644+0000 7fa48fe7f740 1 -- 172.21.15.66:0/722492073 wait complete. 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.644+0000 7fa48fe7f740 1 -- start start 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.644+0000 7fa48fe7f740 1 Processor -- start 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f337a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4f32415f0 con 0x55f4f335c860 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4f32549c0 con 0x55f4f335cc40 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55f4f3299020 con 0x55f4f33580e0 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa48fe7f740 10 client.0 _ll_register_callbacks cb 0x55f4f3238fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4877fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f337a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33781a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50094/0 (socket says 172.21.15.66:50094) 2024-03-30T13:10:16.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:36422/0 (socket says 172.21.15.66:36422) 2024-03-30T13:10:16.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa486ffd640 1 -- 172.21.15.66:0/99791948 learned_addr learned my addr 172.21.15.66:0/99791948 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:16.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4877fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f337a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:49224/0 (socket says 172.21.15.66:49224) 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4877fe640 1 -- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 msgr2=0x55f4f33786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4877fe640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.645+0000 7fa4877fe640 1 -- 172.21.15.66:0/99791948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 msgr2=0x55f4f33781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa4877fe640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa4877fe640 1 -- 172.21.15.66:0/99791948 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f30776b0 con 0x55f4f33580e0 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa486ffd640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa4867fc640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:16.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa4877fe640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f337a3e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa478014cd0 tx=0x7fa478012670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac79e8383d1cbb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa465ffb640 1 -- 172.21.15.66:0/99791948 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478003b90 con 0x55f4f33580e0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa465ffb640 1 -- 172.21.15.66:0/99791948 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa478003d30 con 0x55f4f33580e0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa470083890 con 0x55f4f33580e0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa465ffb640 1 -- 172.21.15.66:0/99791948 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa478027050 con 0x55f4f33580e0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 10 client.4634 fetch_fsmap learned FSMap version 8 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55f4f3361570 con 0x55f4f33580e0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa465ffb640 1 -- 172.21.15.66:0/99791948 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470083890 con 0x55f4f33580e0 2024-03-30T13:10:16.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa465ffb640 1 -- 172.21.15.66:0/99791948 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fa47802a020 con 0x55f4f33580e0 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 10 client.4634 fetch_fsmap finished waiting for FSMap version 8 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 10 client.4634 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 20 client.4634 populate_metadata read hostname 'smithi066' 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f337a920 0x55f4f3362010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 4 client.4634 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.646+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 --> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f326d4e0 con 0x55f4f337a920 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.647+0000 7fa486ffd640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f337a920 0x55f4f3362010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.647+0000 7fa486ffd640 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f337a920 0x55f4f3362010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964ce8b3cd51c5bf server_cookie=ae56578f13c63c5a in_seq=0 out_seq=0 2024-03-30T13:10:16.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.647+0000 7fa465ffb640 10 client.4634 ms_handle_connect on v2:172.21.15.117:6833/4158016355 2024-03-30T13:10:16.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa465ffb640 1 -- 172.21.15.66:0/99791948 <== mds.0 v2:172.21.15.117:6833/4158016355 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55f4f326d4e0 con 0x55f4f337a920 2024-03-30T13:10:16.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa465ffb640 10 client.4634 handle_command_reply: tid=0 2024-03-30T13:10:16.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 client.4634 shutdown 2024-03-30T13:10:16.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 msgr2=0x55f4f337a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f337a3e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa478014cd0 tx=0x7fa478012670 comp rx=0 tx=0).stop 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 shutdown_connections 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f337a920 0x55f4f3362010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f337a3e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/99791948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa465ffb640 0 client.4634 ms_handle_reset on v2:172.21.15.117:6833/4158016355 2024-03-30T13:10:16.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 >> 172.21.15.66:0/99791948 conn(0x55f4f332fd40 msgr2=0x55f4f337db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 shutdown_connections 2024-03-30T13:10:16.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 1 -- 172.21.15.66:0/99791948 wait complete. 2024-03-30T13:10:16.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.648+0000 7fa48fe7f740 20 client.4634 trim_cache size 0 max 16384 2024-03-30T13:10:16.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.666+0000 7fa48fe7f740 1 Processor -- start 2024-03-30T13:10:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.666+0000 7fa48fe7f740 1 -- start start 2024-03-30T13:10:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f335fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f3377170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881feeb0 con 0x55f4f335c860 2024-03-30T13:10:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4881fe270 con 0x55f4f335cc40 2024-03-30T13:10:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa4881fe860 con 0x55f4f33580e0 2024-03-30T13:10:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f3377170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f3377170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50108/0 (socket says 172.21.15.66:50108) 2024-03-30T13:10:16.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 -- 172.21.15.66:0/2669286520 learned_addr learned my addr 172.21.15.66:0/2669286520 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:16.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4877fe640 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa486ffd640 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f335fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 -- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 msgr2=0x55f4f335fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f335fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 -- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 msgr2=0x55f4f33776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.667+0000 7fa4867fc640 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa4867fc640 1 -- 172.21.15.66:0/2669286520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f335bfd0 con 0x55f4f335c860 2024-03-30T13:10:16.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa4867fc640 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f3377170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa4700ab260 tx=0x7fa470091000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ef80b48619427b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa467fff640 1 -- 172.21.15.66:0/2669286520 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47009a120 con 0x55f4f335c860 2024-03-30T13:10:16.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa467fff640 1 -- 172.21.15.66:0/2669286520 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47007f060 con 0x55f4f335c860 2024-03-30T13:10:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa467fff640 1 -- 172.21.15.66:0/2669286520 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470083890 con 0x55f4f335c860 2024-03-30T13:10:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 -- 172.21.15.66:0/2669286520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 msgr2=0x55f4f3377170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f3377170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa4700ab260 tx=0x7fa470091000 comp rx=0 tx=0).stop 2024-03-30T13:10:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 -- 172.21.15.66:0/2669286520 shutdown_connections 2024-03-30T13:10:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f3377170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/2669286520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f335fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 -- 172.21.15.66:0/2669286520 >> 172.21.15.66:0/2669286520 conn(0x55f4f332fd40 msgr2=0x55f4f33605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 -- 172.21.15.66:0/2669286520 shutdown_connections 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.668+0000 7fa48fe7f740 1 -- 172.21.15.66:0/2669286520 wait complete. 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.669+0000 7fa48fe7f740 1 -- start start 2024-03-30T13:10:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.669+0000 7fa48fe7f740 1 Processor -- start 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f3361fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33defb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4f338aa60 con 0x55f4f335c860 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4f335e5f0 con 0x55f4f335cc40 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55f4f337ad70 con 0x55f4f33580e0 2024-03-30T13:10:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 10 client.0 _ll_register_callbacks cb 0x7fa478009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:10:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f3361fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33defb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50124/0 (socket says 172.21.15.66:50124) 2024-03-30T13:10:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f3361fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:49238/0 (socket says 172.21.15.66:49238) 2024-03-30T13:10:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 -- 172.21.15.66:0/3524834941 learned_addr learned my addr 172.21.15.66:0/3524834941 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4877fe640 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 -- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 msgr2=0x55f4f3361fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f3361fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 -- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 msgr2=0x55f4f33781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 -- 172.21.15.66:0/3524834941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f335de30 con 0x55f4f335c860 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa4867fc640 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33defb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa47007ac40 tx=0x7fa470031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5543e8e06608cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa465ffb640 1 -- 172.21.15.66:0/3524834941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47009a120 con 0x55f4f335c860 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa465ffb640 1 -- 172.21.15.66:0/3524834941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa47003ac40 con 0x55f4f335c860 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.670+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47007f060 con 0x55f4f335c860 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa465ffb640 1 -- 172.21.15.66:0/3524834941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa47003ae40 con 0x55f4f335c860 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 10 client.4854 fetch_fsmap learned FSMap version 8 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55f4f33f9d20 con 0x55f4f335c860 2024-03-30T13:10:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa465ffb640 1 -- 172.21.15.66:0/3524834941 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47007f060 con 0x55f4f335c860 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa465ffb640 1 -- 172.21.15.66:0/3524834941 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fa470083400 con 0x55f4f335c860 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 10 client.4854 fetch_fsmap finished waiting for FSMap version 8 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 10 client.4854 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 20 client.4854 populate_metadata read hostname 'smithi066' 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f335ac30 0x55f4f335b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 4 client.4854 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 --> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] -- command(tid 0: {"prefix": "status"}) -- 0x55f4f3379f50 con 0x55f4f335ac30 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa486ffd640 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f335ac30 0x55f4f335b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:16.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa486ffd640 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f335ac30 0x55f4f335b010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262252621614a0be server_cookie=c7d6ec20bef8405e in_seq=0 out_seq=0 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.671+0000 7fa465ffb640 10 client.4854 ms_handle_connect on v2:172.21.15.117:6833/4158016355 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa465ffb640 1 -- 172.21.15.66:0/3524834941 <== mds.0 v2:172.21.15.117:6833/4158016355 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55f4f3379f50 con 0x55f4f335ac30 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa465ffb640 10 client.4854 handle_command_reply: tid=0 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 client.4854 shutdown 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 msgr2=0x55f4f33defb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33defb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa47007ac40 tx=0x7fa470031f90 comp rx=0 tx=0).stop 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 shutdown_connections 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:6833/4158016355,v1:172.21.15.117:6835/4158016355] conn(0x55f4f335ac30 0x55f4f335b010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4f335cc40 0x55f4f33781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa465ffb640 0 client.4854 ms_handle_reset on v2:172.21.15.117:6833/4158016355 2024-03-30T13:10:16.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4f335c860 0x55f4f33defb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 --2- 172.21.15.66:0/3524834941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4f33580e0 0x55f4f3361fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 >> 172.21.15.66:0/3524834941 conn(0x55f4f332fd40 msgr2=0x55f4f33605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 shutdown_connections 2024-03-30T13:10:16.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 1 -- 172.21.15.66:0/3524834941 wait complete. 2024-03-30T13:10:16.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48fe7f740 20 client.4854 trim_cache size 0 max 16384 2024-03-30T13:10:16.699 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:10:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "03992909-781c-43b4-940b-d3d6d831de0b", 2024-03-30T13:10:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T13:10:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4290, 2024-03-30T13:10:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T13:10:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T13:10:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T13:10:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 33.561074177000002, 2024-03-30T13:10:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 8, 2024-03-30T13:10:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 26, 2024-03-30T13:10:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 26, 2024-03-30T13:10:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 40.456906422000003 2024-03-30T13:10:16.701 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa46806c530 msgr2=0x7fa46806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa46806c530 0x7fa46806e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa480005880 tx=0x7fa480009040 comp rx=0 tx=0).stop 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 msgr2=0x7fa488156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 0x7fa488156ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa470034040 tx=0x7fa4700046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 shutdown_connections 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa46806c530 0x7fa46806e9f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48815fe10 0x7fa4880b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa48815c260 0x7fa488156ba0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 --2- 172.21.15.66:0/1736551911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4880b4130 0x7fa4880b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 >> 172.21.15.66:0/1736551911 conn(0x7fa4880cebf0 msgr2=0x7fa4880cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 shutdown_connections 2024-03-30T13:10:16.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:16.672+0000 7fa48d28f640 1 -- 172.21.15.66:0/1736551911 wait complete. 2024-03-30T13:10:16.705 INFO:teuthology.run_tasks:Running task sequential... 2024-03-30T13:10:16.793 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-30T13:10:16.793 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-30T13:10:16.793 INFO:teuthology.task.print:Enabling mgr modules 2024-03-30T13:10:16.793 INFO:teuthology.run_tasks:Running task kclient... 2024-03-30T13:10:16.814 INFO:tasks.kclient:Mounting kernel clients... 2024-03-30T13:10:16.815 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-30T13:10:16.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:16.815 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:10:16.833 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-30T13:10:16.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:16.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:10:16.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:16.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-03-30T13:10:17.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.001 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:10:17.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.017 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-03-30T13:10:17.090 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-03-30T13:10:17.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:17.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.092 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:10:17.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.108 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-30T13:10:17.176 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-03-30T13:10:17.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:17.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.177 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:10:17.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.233 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-30T13:10:17.299 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-03-30T13:10:17.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:17.301 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-30T13:10:17.301 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:10:17.301 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T13:10:17.301 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:10:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.406+0000 7f74ce6b3640 1 Processor -- start 2024-03-30T13:10:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74ce6b3640 1 -- start start 2024-03-30T13:10:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74ce6b3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74c815b270 0x7f74c815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74ce6b3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c80ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74ce6b3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c815b650 con 0x7f74c8072280 2024-03-30T13:10:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74ce6b3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f74c8072ab0 con 0x7f74c815b270 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74ce6b3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74c8072d90 con 0x7f74c80cd0e0 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.407+0000 7f74c77fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f74c8072280 0x7f74c80cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55300/0 (socket says 172.21.15.66:55300) 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c77fe640 1 -- 172.21.15.66:0/3999067242 learned_addr learned my addr 172.21.15.66:0/3999067242 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74ccc50640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c80ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c7fff640 1 --2- 172.21.15.66:0/3999067242 >> v2:172.21.15.117:3300/0 conn(0x7f74c815b270 0x7f74c815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c6ffd640 1 -- 172.21.15.66:0/3999067242 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171579496 0 0) 0x7f74c815b650 con 0x7f74c8072280 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c6ffd640 1 -- 172.21.15.66:0/3999067242 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74b0003520 con 0x7f74c8072280 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c7fff640 1 -- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 msgr2=0x7f74c80ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c7fff640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c80ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c7fff640 1 -- 172.21.15.66:0/3999067242 >> v1:172.21.15.66:6789/0 conn(0x7f74c8072280 legacy=0x7f74c80cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74c7fff640 1 -- 172.21.15.66:0/3999067242 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c8073070 con 0x7f74c815b270 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.408+0000 7f74ccc50640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c80ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74c7fff640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c815f7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0097a0 tx=0x7f74bc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326bf80b53244cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74c6ffd640 1 -- 172.21.15.66:0/3999067242 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc035020 con 0x7f74c815b270 2024-03-30T13:10:17.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74c6ffd640 1 -- 172.21.15.66:0/3999067242 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74bc031830 con 0x7f74c815b270 2024-03-30T13:10:17.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74c6ffd640 1 -- 172.21.15.66:0/3999067242 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc031be0 con 0x7f74c815b270 2024-03-30T13:10:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 -- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 msgr2=0x7f74c815f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c815f7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0097a0 tx=0x7f74bc02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 -- 172.21.15.66:0/3999067242 shutdown_connections 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c80ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/3999067242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c815f7e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 -- 172.21.15.66:0/3999067242 >> 172.21.15.66:0/3999067242 conn(0x7f74c80b1c90 msgr2=0x7f74c80b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 -- 172.21.15.66:0/3999067242 shutdown_connections 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.409+0000 7f74ce6b3640 1 -- 172.21.15.66:0/3999067242 wait complete. 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 Processor -- start 2024-03-30T13:10:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 -- start start 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 0x7f74c8151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c8151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c8167170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80726e0 con 0x7f74c8072280 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8073630 con 0x7f74c815b270 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.411+0000 7f74ce6b3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74c8072860 con 0x7f74c80cd0e0 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 0x7f74c8151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 0x7f74c8151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50132/0 (socket says 172.21.15.66:50132) 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 -- 172.21.15.66:0/1201204372 learned_addr learned my addr 172.21.15.66:0/1201204372 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74ccc50640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c8167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c77fe640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c8151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 -- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 msgr2=0x7f74c8151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c8151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 -- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 msgr2=0x7f74c8167170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c8167170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 -- 172.21.15.66:0/1201204372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c80b4050 con 0x7f74c8072280 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c7fff640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 0x7f74c8151a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f74bc034040 tx=0x7f74bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc040070 con 0x7f74c8072280 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74bc03e070 con 0x7f74c8072280 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c80b42d0 con 0x7f74c8072280 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc035040 con 0x7f74c8072280 2024-03-30T13:10:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.412+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c81681f0 con 0x7f74c8072280 2024-03-30T13:10:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.413+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c8075a10 con 0x7f74c8072280 2024-03-30T13:10:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.414+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f74bc038020 con 0x7f74c8072280 2024-03-30T13:10:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.414+0000 7f74c4ff9640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74b406c530 0x7f74b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.414+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f74bc0ad070 con 0x7f74c8072280 2024-03-30T13:10:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.414+0000 7f74c77fe640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74b406c530 0x7f74b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.415+0000 7f74c77fe640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74b406c530 0x7f74b406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f74c8152960 tx=0x7f74b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:17.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.418+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f74c80b4050 con 0x7f74c8072280 2024-03-30T13:10:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.640+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74c81688d0 con 0x7f74c8072280 2024-03-30T13:10:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.640+0000 7f74c4ff9640 1 -- 172.21.15.66:0/1201204372 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f74bc07b350 con 0x7f74c8072280 2024-03-30T13:10:17.644 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74b406c530 msgr2=0x7f74b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74b406c530 0x7f74b406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f74c8152960 tx=0x7f74b8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 msgr2=0x7f74c8151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 0x7f74c8151a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f74bc034040 tx=0x7f74bc0046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 shutdown_connections 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74b406c530 0x7f74b406e9f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74c815b270 0x7f74c8167170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74c80cd0e0 0x7f74c8151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 --2- 172.21.15.66:0/1201204372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74c8072280 0x7f74c8151a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 >> 172.21.15.66:0/1201204372 conn(0x7f74c80b1c90 msgr2=0x7f74c8074b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 shutdown_connections 2024-03-30T13:10:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17.642+0000 7f74ce6b3640 1 -- 172.21.15.66:0/1201204372 wait complete. 2024-03-30T13:10:17.656 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:10:17.657 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:10:17.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.657 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42613sec preferred_lft 42613sec 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T13:10:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:17.674 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T13:10:17.675 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:17.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:17.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:17.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.837 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:10:17.867 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T13:10:17.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:17.868 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T13:10:17.925 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T13:10:17.925 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:10:17.925 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:10:17.925 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:10:17.925 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:10:17.925 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:17.925 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:17.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:10:17.926 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:10:17.926 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:10:17.926 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:18.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:18.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:18.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:18.073 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:10:18.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:18.089 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-03-30T13:10:18.143 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:18.144 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:18.144 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:10:18.144 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-30T13:10:18.144 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:18.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:18.273 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-30T13:10:18.274 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:18.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:18.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:18.419 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:18.419 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:18.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T13:10:18.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T13:10:18.419 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:18.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:18.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:18.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:18.509 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T13:10:18.524 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:10:18.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:18.524 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:10:18.579 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-30T13:10:18.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:18.580 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:18.781 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:18.781 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:18.781 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-30T13:10:18.781 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 0@03992909-781c-43b4-940b-d3d6d831de0b.cephfs=/ 2024-03-30T13:10:18.781 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.66:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-30T13:10:18.782 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-30T13:10:18.782 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-03-30T13:10:18.782 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-30T13:10:18.782 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-03-30T13:10:18.782 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-30T13:10:18.784 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T13:10:18.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:18.784 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:10:18.815 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:18.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:18.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:18.913 INFO:teuthology.orchestra.run.smithi066.stdout:client4846 2024-03-30T13:10:18.913 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:18.913 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:18.931 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:19.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:19.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:19.030 INFO:teuthology.orchestra.run.smithi066.stdout:client4846 2024-03-30T13:10:19.031 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:19.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:19.088 INFO:teuthology.orchestra.run.smithi066.stdout:client4846 2024-03-30T13:10:19.088 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/03992909-781c-43b4-940b-d3d6d831de0b.client4846/status') 2024-03-30T13:10:19.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:19.153 INFO:teuthology.orchestra.run.smithi066.stdout:instance: client.4846 (3)172.21.15.66:0/1660813538 2024-03-30T13:10:19.154 INFO:teuthology.orchestra.run.smithi066.stdout:blocklisted: false 2024-03-30T13:10:19.154 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records in 2024-03-30T13:10:19.154 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records out 2024-03-30T13:10:19.154 INFO:teuthology.orchestra.run.smithi066.stderr:70 bytes copied, 2.2852e-05 s, 3.1 MB/s 2024-03-30T13:10:19.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:19.162 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-30T13:10:19.162 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:10:19.162 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-30T13:10:19.162 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:10:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.266+0000 7f871c453640 1 Processor -- start 2024-03-30T13:10:19.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.266+0000 7f871c453640 1 -- start start 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.267+0000 7f871c453640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f871415e950 0x7f87140d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.267+0000 7f871c453640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87140d48c0 0x7f87140d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.267+0000 7f871c453640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87140d2f00 0x7f87140cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.267+0000 7f871c453640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f87140d3cb0 con 0x7f871415e950 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.267+0000 7f871c453640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f87140d4d00 con 0x7f87140d2f00 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.267+0000 7f871c453640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f87140d36f0 con 0x7f87140d48c0 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f871415e950 0x7f87140d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f871415e950 0x7f87140d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50144/0 (socket says 172.21.15.66:50144) 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 -- 172.21.15.66:0/199301738 learned_addr learned my addr 172.21.15.66:0/199301738 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:19.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f87199ee640 1 --2- 172.21.15.66:0/199301738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87140d48c0 0x7f87140d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a9f0640 1 --2- 172.21.15.66:0/199301738 >> v2:172.21.15.117:3300/0 conn(0x7f87140d2f00 0x7f87140cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 -- 172.21.15.66:0/199301738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87140d48c0 msgr2=0x7f87140d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:19.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 --2- 172.21.15.66:0/199301738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87140d48c0 0x7f87140d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 -- 172.21.15.66:0/199301738 >> v2:172.21.15.117:3300/0 conn(0x7f87140d2f00 msgr2=0x7f87140cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:19.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 --2- 172.21.15.66:0/199301738 >> v2:172.21.15.117:3300/0 conn(0x7f87140d2f00 0x7f87140cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.268+0000 7f871a1ef640 1 -- 172.21.15.66:0/199301738 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87140d39d0 con 0x7f871415e950 2024-03-30T13:10:19.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.269+0000 7f871a1ef640 1 --2- 172.21.15.66:0/199301738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871415e950 0x7f87140d4380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f87140ce9f0 tx=0x7f870c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a0b604079c7845 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.269+0000 7f87191ed640 1 -- 172.21.15.66:0/199301738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c034070 con 0x7f871415e950 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.269+0000 7f87191ed640 1 -- 172.21.15.66:0/199301738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c030860 con 0x7f871415e950 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.269+0000 7f87191ed640 1 -- 172.21.15.66:0/199301738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c030c30 con 0x7f871415e950 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.269+0000 7f871c453640 1 -- 172.21.15.66:0/199301738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871415e950 msgr2=0x7f87140d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.269+0000 7f871c453640 1 --2- 172.21.15.66:0/199301738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871415e950 0x7f87140d4380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f87140ce9f0 tx=0x7f870c02b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 -- 172.21.15.66:0/199301738 shutdown_connections 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 --2- 172.21.15.66:0/199301738 >> v2:172.21.15.117:3300/0 conn(0x7f87140d2f00 0x7f87140cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 --2- 172.21.15.66:0/199301738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87140d48c0 0x7f87140d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 --2- 172.21.15.66:0/199301738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871415e950 0x7f87140d4380 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 -- 172.21.15.66:0/199301738 >> 172.21.15.66:0/199301738 conn(0x7f87140676e0 msgr2=0x7f8714067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 -- 172.21.15.66:0/199301738 shutdown_connections 2024-03-30T13:10:19.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.270+0000 7f871c453640 1 -- 172.21.15.66:0/199301738 wait complete. 2024-03-30T13:10:19.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.271+0000 7f871c453640 1 Processor -- start 2024-03-30T13:10:19.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.271+0000 7f871c453640 1 -- start start 2024-03-30T13:10:19.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.271+0000 7f871c453640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87140d2f00 0x7f871415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 0x7f8714160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f871415e950 0x7f87140ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87140d3390 con 0x7f87140d48c0 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87140b7da0 con 0x7f87140d2f00 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f871415ed30 con 0x7f871415e950 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 0x7f8714160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 0x7f8714160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50150/0 (socket says 172.21.15.66:50150) 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 -- 172.21.15.66:0/381164368 learned_addr learned my addr 172.21.15.66:0/381164368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871a1ef640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87140d2f00 0x7f871415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871a9f0640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f871415e950 0x7f87140ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 -- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f871415e950 msgr2=0x7f87140ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f871415e950 0x7f87140ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 -- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87140d2f00 msgr2=0x7f871415c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87140d2f00 0x7f871415c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 -- 172.21.15.66:0/381164368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8714161a70 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871a9f0640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f871415e950 0x7f87140ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f87199ee640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 0x7f8714160ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f870800e9b0 tx=0x7f870800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708017070 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870800c9a0 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870800cd90 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871414d800 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.272+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871414ddc0 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.273+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8714075a10 con 0x7f87140d48c0 2024-03-30T13:10:19.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.274+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f870801a070 con 0x7f87140d48c0 2024-03-30T13:10:19.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.274+0000 7f8712ffd640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f870006c530 0x7f870006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:19.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.274+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f870808b890 con 0x7f87140d48c0 2024-03-30T13:10:19.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.274+0000 7f871a1ef640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f870006c530 0x7f870006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:19.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.275+0000 7f871a1ef640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f870006c530 0x7f870006e9f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f870c036000 tx=0x7f870c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:19.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.279+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f87080595b0 con 0x7f87140d48c0 2024-03-30T13:10:19.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.505+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8714067f30 con 0x7f87140d48c0 2024-03-30T13:10:19.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.506+0000 7f8712ffd640 1 -- 172.21.15.66:0/381164368 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f870805d8b0 con 0x7f87140d48c0 2024-03-30T13:10:19.509 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f870006c530 msgr2=0x7f870006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f870006c530 0x7f870006e9f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f870c036000 tx=0x7f870c008030 comp rx=0 tx=0).stop 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 msgr2=0x7f8714160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 0x7f8714160ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f870800e9b0 tx=0x7f870800ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 shutdown_connections 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f870006c530 0x7f870006e9f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f871415e950 0x7f87140ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87140d48c0 0x7f8714160ff0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 --2- 172.21.15.66:0/381164368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87140d2f00 0x7f871415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 >> 172.21.15.66:0/381164368 conn(0x7f87140676e0 msgr2=0x7f87140b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 shutdown_connections 2024-03-30T13:10:19.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19.508+0000 7f871c453640 1 -- 172.21.15.66:0/381164368 wait complete. 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:10:19.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:10:19.523 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-30T13:10:19.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:19.523 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T13:10:19.538 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:19.539 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42611sec preferred_lft 42611sec 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 62:09:dc:7a:25:75 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::b002:76ff:fe8e:b0d7/64 scope link 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 62:09:dc:7a:25:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:10:19.540 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::6009:dcff:fe7a:2575/64 scope link tentative 2024-03-30T13:10:19.541 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:19.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:19.541 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:10:19.609 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T13:10:19.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:19.611 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T13:10:19.667 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T13:10:19.667 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:10:19.667 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:10:19.667 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:10:19.667 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:10:19.668 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:19.668 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:19.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:10:19.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:10:19.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:10:19.668 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:19.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:19.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:19.814 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:10:19.830 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T13:10:19.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:19.832 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-03-30T13:10:19.885 INFO:teuthology.orchestra.run.smithi066.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-30T13:10:19.886 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:19.886 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:19.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:10:19.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-30T13:10:19.886 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:19.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:20.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:20.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-30T13:10:20.047 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:ec:8b:c8:0b:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4ec:8bff:fec8:b28/64 scope link 2024-03-30T13:10:20.048 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:20.050 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-30T13:10:20.120 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:20.121 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:20.121 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:20.121 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.121 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:20.121 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.122 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:10:20.122 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c6:ec:8b:c8:0b:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:10:20.122 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:10:20.122 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.122 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c4ec:8bff:fec8:b28/64 scope link 2024-03-30T13:10:20.122 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:20.124 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-30T13:10:20.125 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:20.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:20.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.310 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:20.310 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:10:20.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.1 up 2024-03-30T13:10:20.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.1 master ceph-brx 2024-03-30T13:10:20.310 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:10:20.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:20.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:20.401 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-30T13:10:20.417 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-30T13:10:20.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:20.418 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-30T13:10:20.472 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-30T13:10:20.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:20.473 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:20.562 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:20.562 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:20.570 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:20.570 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:20.570 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-30T13:10:20.570 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 2@03992909-781c-43b4-940b-d3d6d831de0b.cephfs=/ 2024-03-30T13:10:20.570 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.66:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-30T13:10:20.570 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-30T13:10:20.571 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-03-30T13:10:20.571 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-30T13:10:20.571 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-03-30T13:10:20.571 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-30T13:10:20.572 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T13:10:20.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:20.572 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-30T13:10:20.603 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:20.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:20.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.700 INFO:teuthology.orchestra.run.smithi066.stdout:client4855 2024-03-30T13:10:20.700 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:20.700 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:20.716 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:20.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:20.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.814 INFO:teuthology.orchestra.run.smithi066.stdout:client4855 2024-03-30T13:10:20.815 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:20.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:20.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.872 INFO:teuthology.orchestra.run.smithi066.stdout:client4855 2024-03-30T13:10:20.873 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/03992909-781c-43b4-940b-d3d6d831de0b.client4855/status') 2024-03-30T13:10:20.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:20.938 INFO:teuthology.orchestra.run.smithi066.stdout:instance: client.4855 (3)172.21.15.66:0/2237694629 2024-03-30T13:10:20.939 INFO:teuthology.orchestra.run.smithi066.stdout:blocklisted: false 2024-03-30T13:10:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records in 2024-03-30T13:10:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records out 2024-03-30T13:10:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:70 bytes copied, 2.2448e-05 s, 3.1 MB/s 2024-03-30T13:10:20.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:20.946 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-30T13:10:20.946 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:10:20.946 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T13:10:20.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:10:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.050+0000 7f6d61122640 1 Processor -- start 2024-03-30T13:10:21.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.050+0000 7f6d61122640 1 -- start start 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.050+0000 7f6d61122640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c0d4a90 0x7f6d5c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.050+0000 7f6d61122640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c0d36f0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.050+0000 7f6d61122640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c0d39d0 con 0x7f6d5c15e950 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.050+0000 7f6d61122640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c0d3cb0 con 0x7f6d5c0d4a90 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d5b59e640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6d5c0d2f00 0x7f6d5c0b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:55304/0 (socket says 172.21.15.66:55304) 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d5b59e640 1 -- 172.21.15.66:0/95583739 learned_addr learned my addr 172.21.15.66:0/95583739 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d5a59c640 1 --2- 172.21.15.66:0/95583739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c0d4a90 0x7f6d5c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314646625 0 0) 0x7f6d5c0d36f0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d48003520 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170575027 0 0) 0x7f6d5c0d39d0 con 0x7f6d5c15e950 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d5c0d36f0 con 0x7f6d5c15e950 2024-03-30T13:10:21.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d5a59c640 1 -- 172.21.15.66:0/95583739 >> v1:172.21.15.117:6789/0 conn(0x7f6d5c15e950 legacy=0x7f6d5c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d5a59c640 1 -- 172.21.15.66:0/95583739 >> v1:172.21.15.66:6789/0 conn(0x7f6d5c0d2f00 legacy=0x7f6d5c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.051+0000 7f6d5a59c640 1 -- 172.21.15.66:0/95583739 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c0b7ac0 con 0x7f6d5c0d4a90 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d5a59c640 1 --2- 172.21.15.66:0/95583739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c0d4a90 0x7f6d5c0d29c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097d0 tx=0x7f6d4402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47442df6fd3bfe19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44034070 con 0x7f6d5c0d4a90 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d44030880 con 0x7f6d5c0d4a90 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d59d9b640 1 -- 172.21.15.66:0/95583739 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44030c30 con 0x7f6d5c0d4a90 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 -- 172.21.15.66:0/95583739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c0d4a90 msgr2=0x7f6d5c0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 --2- 172.21.15.66:0/95583739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c0d4a90 0x7f6d5c0d29c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097d0 tx=0x7f6d4402b9c0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 -- 172.21.15.66:0/95583739 shutdown_connections 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 --2- 172.21.15.66:0/95583739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c0d4a90 0x7f6d5c0d29c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 -- 172.21.15.66:0/95583739 >> 172.21.15.66:0/95583739 conn(0x7f6d5c0676e0 msgr2=0x7f6d5c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 -- 172.21.15.66:0/95583739 shutdown_connections 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.052+0000 7f6d61122640 1 -- 172.21.15.66:0/95583739 wait complete. 2024-03-30T13:10:21.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.054+0000 7f6d61122640 1 Processor -- start 2024-03-30T13:10:21.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.054+0000 7f6d61122640 1 -- start start 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d61122640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 0x7f6d5c1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d61122640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d5c0d4a90 0x7f6d5c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d61122640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c15e950 0x7f6d5c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d61122640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c15ed30 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d61122640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0d33b0 con 0x7f6d5c0d4a90 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d61122640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c15eeb0 con 0x7f6d5c15e950 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 0x7f6d5c1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 0x7f6d5c1f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50160/0 (socket says 172.21.15.66:50160) 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 -- 172.21.15.66:0/698418499 learned_addr learned my addr 172.21.15.66:0/698418499 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5a59c640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d5c0d4a90 0x7f6d5c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5b59e640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c15e950 0x7f6d5c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 -- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c15e950 msgr2=0x7f6d5c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c15e950 0x7f6d5c0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 -- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d5c0d4a90 msgr2=0x7f6d5c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d5c0d4a90 0x7f6d5c0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 -- 172.21.15.66:0/698418499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c1f7f00 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d5ad9d640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 0x7f6d5c1f5010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c430 tx=0x7f6d4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.055+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c017070 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.056+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0052f0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.056+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c003830 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.056+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c1f81e0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.056+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1f87a0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.056+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c075a10 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.057+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6d4c01a070 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.057+0000 7f6d537fe640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6d3c06c530 0x7f6d3c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.058+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6d4c08ad60 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.058+0000 7f6d5a59c640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6d3c06c530 0x7f6d3c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.058+0000 7f6d5a59c640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6d3c06c530 0x7f6d3c06e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d44036000 tx=0x7f6d4403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.062+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6d4c0589f0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.287+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d5c0b70a0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.288+0000 7f6d537fe640 1 -- 172.21.15.66:0/698418499 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f6d4c05ccf0 con 0x7f6d5c0d2f00 2024-03-30T13:10:21.291 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:10:21.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6d3c06c530 msgr2=0x7f6d3c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6d3c06c530 0x7f6d3c06e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d44036000 tx=0x7f6d4403b040 comp rx=0 tx=0).stop 2024-03-30T13:10:21.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 msgr2=0x7f6d5c1f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:21.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 0x7f6d5c1f5010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c430 tx=0x7f6d4c00c8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 shutdown_connections 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6d3c06c530 0x7f6d3c06e9f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5c15e950 0x7f6d5c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d5c0d4a90 0x7f6d5c0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 --2- 172.21.15.66:0/698418499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d5c0d2f00 0x7f6d5c1f5010 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.289+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 >> 172.21.15.66:0/698418499 conn(0x7f6d5c0676e0 msgr2=0x7f6d5c0cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.290+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 shutdown_connections 2024-03-30T13:10:21.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:21.290+0000 7f6d61122640 1 -- 172.21.15.66:0/698418499 wait complete. 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:10:21.304 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:10:21.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:21.305 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:21.322 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42562sec preferred_lft 42562sec 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-03-30T13:10:21.323 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:21.323 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T13:10:21.324 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:21.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:21.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:21.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:21.535 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:10:21.563 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-30T13:10:21.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:21.565 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-03-30T13:10:21.636 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-03-30T13:10:21.636 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:10:21.636 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:10:21.636 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:10:21.636 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:10:21.637 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:21.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:21.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:21.942 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:10:21.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:21.969 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-03-30T13:10:22.031 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:22.031 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:22.031 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:10:22.031 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-30T13:10:22.031 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:22.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:22.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:22.169 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-30T13:10:22.170 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:22.170 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:22.170 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-30T13:10:22.171 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T13:10:22.171 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-30T13:10:22.171 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-30T13:10:22.171 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-30T13:10:22.171 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:22.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:22.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:22.350 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:22.350 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:22.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-03-30T13:10:22.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T13:10:22.350 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:22.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:22.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:22.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:22.433 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T13:10:22.448 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:10:22.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:22.449 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:10:22.505 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-30T13:10:22.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:22.505 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:22.592 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:22.592 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:22.701 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:22.701 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:22.701 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-30T13:10:22.701 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with new device syntax: 1@03992909-781c-43b4-940b-d3d6d831de0b.cephfs=/ 2024-03-30T13:10:22.701 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.66:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-30T13:10:22.702 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-30T13:10:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-03-30T13:10:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-30T13:10:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-03-30T13:10:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-30T13:10:22.704 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T13:10:22.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:22.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:10:22.732 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:22.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:22.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:22.830 INFO:teuthology.orchestra.run.smithi117.stdout:client4864 2024-03-30T13:10:22.830 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:22.830 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:22.846 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:22.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:22.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:22.947 INFO:teuthology.orchestra.run.smithi117.stdout:client4864 2024-03-30T13:10:22.948 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:22.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:22.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:client4864 2024-03-30T13:10:23.006 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/03992909-781c-43b4-940b-d3d6d831de0b.client4864/status') 2024-03-30T13:10:23.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:23.068 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.4864 (3)192.168.0.1:0/2819919227 2024-03-30T13:10:23.069 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-03-30T13:10:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-03-30T13:10:23.069 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-03-30T13:10:23.070 INFO:teuthology.orchestra.run.smithi117.stderr:69 bytes copied, 2.1881e-05 s, 3.2 MB/s 2024-03-30T13:10:23.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:23.077 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-30T13:10:23.077 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:10:23.077 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-30T13:10:23.078 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:10:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.199+0000 7f7567a9a640 1 Processor -- start 2024-03-30T13:10:23.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.199+0000 7f7567a9a640 1 -- start start 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7567a9a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f756015b270 0x7f756015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7567a9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f75600ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7567a9a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f756015b650 con 0x7f7560072280 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7567a9a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7560072ab0 con 0x7f756015b270 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7567a9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7560072d90 con 0x7f75600cd0e0 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7566037640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f75600ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7565836640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f756015b270 0x7f756015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7566037640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f75600ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:43228/0 (socket says 172.21.15.66:43228) 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7566037640 1 -- 172.21.15.66:0/1342930244 learned_addr learned my addr 172.21.15.66:0/1342930244 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7565035640 1 --1- 172.21.15.66:0/1342930244 >> v1:172.21.15.117:6789/0 conn(0x7f7560072280 0x7f75600cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45314/0 (socket says 172.21.15.66:45314) 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.200+0000 7f7566037640 1 -- 172.21.15.66:0/1342930244 >> v2:172.21.15.117:3300/0 conn(0x7f756015b270 msgr2=0x7f756015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.201+0000 7f7566037640 1 --2- 172.21.15.66:0/1342930244 >> v2:172.21.15.117:3300/0 conn(0x7f756015b270 0x7f756015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.201+0000 7f7566037640 1 -- 172.21.15.66:0/1342930244 >> v1:172.21.15.117:6789/0 conn(0x7f7560072280 legacy=0x7f75600cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.201+0000 7f7566037640 1 -- 172.21.15.66:0/1342930244 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7560073070 con 0x7f75600cd0e0 2024-03-30T13:10:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.201+0000 7f7566037640 1 --2- 172.21.15.66:0/1342930244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f75600ca9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7558009e50 tx=0x7f755802e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b40285cb561869 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.201+0000 7f7564834640 1 -- 172.21.15.66:0/1342930244 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755802ed10 con 0x7f75600cd0e0 2024-03-30T13:10:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7564834640 1 -- 172.21.15.66:0/1342930244 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75580363f0 con 0x7f75600cd0e0 2024-03-30T13:10:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7564834640 1 -- 172.21.15.66:0/1342930244 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580367c0 con 0x7f75600cd0e0 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 -- 172.21.15.66:0/1342930244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 msgr2=0x7f75600ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 --2- 172.21.15.66:0/1342930244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f75600ca9d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7558009e50 tx=0x7f755802e170 comp rx=0 tx=0).stop 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 -- 172.21.15.66:0/1342930244 shutdown_connections 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 --2- 172.21.15.66:0/1342930244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f75600ca9d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 --2- 172.21.15.66:0/1342930244 >> v2:172.21.15.117:3300/0 conn(0x7f756015b270 0x7f756015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 -- 172.21.15.66:0/1342930244 >> 172.21.15.66:0/1342930244 conn(0x7f75600b1c90 msgr2=0x7f75600b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 -- 172.21.15.66:0/1342930244 shutdown_connections 2024-03-30T13:10:23.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.202+0000 7f7567a9a640 1 -- 172.21.15.66:0/1342930244 wait complete. 2024-03-30T13:10:23.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.203+0000 7f7567a9a640 1 Processor -- start 2024-03-30T13:10:23.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.203+0000 7f7567a9a640 1 -- start start 2024-03-30T13:10:23.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7567a9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 0x7f75601546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:23.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7567a9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f7560157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:23.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7567a9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f756015b270 0x7f756015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7567a9a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75600726e0 con 0x7f7560072280 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7567a9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7560073630 con 0x7f756015b270 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7567a9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7560072860 con 0x7f75600cd0e0 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7565836640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 0x7f75601546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7565836640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 0x7f75601546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48098/0 (socket says 172.21.15.66:48098) 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7565836640 1 -- 172.21.15.66:0/2511358983 learned_addr learned my addr 172.21.15.66:0/2511358983 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7566037640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f756015b270 0x7f756015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.204+0000 7f7565035640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f7560157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7565836640 1 -- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 msgr2=0x7f7560157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7565836640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f7560157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7565836640 1 -- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f756015b270 msgr2=0x7f756015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7565836640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f756015b270 0x7f756015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7565836640 1 -- 172.21.15.66:0/2511358983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601f3150 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7565836640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 0x7f75601546e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f755c00e8a0 tx=0x7f755c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c017070 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755c00c940 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7560154c20 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c00cd30 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.205+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75600b5120 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.206+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7560075a10 con 0x7f7560072280 2024-03-30T13:10:23.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.207+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f755c01a070 con 0x7f7560072280 2024-03-30T13:10:23.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.207+0000 7f75567fc640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f754406c530 0x7f754406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:23.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.207+0000 7f7565035640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f754406c530 0x7f754406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:23.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.207+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f755c08b870 con 0x7f7560072280 2024-03-30T13:10:23.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.208+0000 7f7565035640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f754406c530 0x7f754406e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7560070070 tx=0x7f7550007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:23.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.212+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f755c059480 con 0x7f7560072280 2024-03-30T13:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.450+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f75600cd720 con 0x7f7560072280 2024-03-30T13:10:23.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.450+0000 7f75567fc640 1 -- 172.21.15.66:0/2511358983 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f755c05d780 con 0x7f7560072280 2024-03-30T13:10:23.454 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:10:23.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f754406c530 msgr2=0x7f754406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f754406c530 0x7f754406e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7560070070 tx=0x7f7550007e80 comp rx=0 tx=0).stop 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 msgr2=0x7f75601546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 0x7f75601546e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f755c00e8a0 tx=0x7f755c00ed60 comp rx=0 tx=0).stop 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 shutdown_connections 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f754406c530 0x7f754406e9f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f756015b270 0x7f756015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75600cd0e0 0x7f7560157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 --2- 172.21.15.66:0/2511358983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7560072280 0x7f75601546e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 >> 172.21.15.66:0/2511358983 conn(0x7f75600b1c90 msgr2=0x7f75600d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 shutdown_connections 2024-03-30T13:10:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:23.453+0000 7f7567a9a640 1 -- 172.21.15.66:0/2511358983 wait complete. 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:10:23.469 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-30T13:10:23.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.469 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-03-30T13:10:23.485 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-03-30T13:10:23.486 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42560sec preferred_lft 42560sec 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether e6:1e:e5:47:b7:28 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::3c4b:adff:fe12:ffe9/64 scope link 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:10:23.487 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether e6:1e:e5:47:b7:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:10:23.488 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::e41e:e5ff:fe47:b728/64 scope link tentative 2024-03-30T13:10:23.488 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.488 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:10:23.551 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-30T13:10:23.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.553 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-03-30T13:10:23.608 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-03-30T13:10:23.608 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:10:23.608 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:10:23.608 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:10:23.608 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:10:23.609 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:23.609 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:23.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:10:23.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:10:23.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:10:23.609 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:23.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:23.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:23.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.745 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:10:23.760 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T13:10:23.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.760 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-03-30T13:10:23.815 INFO:teuthology.orchestra.run.smithi117.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-30T13:10:23.816 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:23.816 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:23.816 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:10:23.816 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-30T13:10:23.816 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:23.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:23.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:23.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether de:3e:e3:5f:38:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:10:23.965 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.966 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::dc3e:e3ff:fe5f:3814/64 scope link tentative 2024-03-30T13:10:23.966 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:23.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:23.967 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-30T13:10:24.034 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:10:24.034 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:10:24.034 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:10:24.034 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether de:3e:e3:5f:38:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::dc3e:e3ff:fe5f:3814/64 scope link tentative 2024-03-30T13:10:24.035 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:10:24.036 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-30T13:10:24.036 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:24.036 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:24.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-30T13:10:24.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-30T13:10:24.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-30T13:10:24.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-30T13:10:24.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-30T13:10:24.037 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:24.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:24.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:24.203 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:10:24.204 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:10:24.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.1 up 2024-03-30T13:10:24.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.1 master ceph-brx 2024-03-30T13:10:24.204 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:10:24.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:24.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:24.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:24.290 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-30T13:10:24.305 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-30T13:10:24.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:10:24.305 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-30T13:10:24.360 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-30T13:10:24.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:24.361 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:24.446 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:24.446 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:24.456 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:10:24.456 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:10:24.456 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-30T13:10:24.456 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with new device syntax: 3@03992909-781c-43b4-940b-d3d6d831de0b.cephfs=/ 2024-03-30T13:10:24.456 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.66:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-30T13:10:24.457 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-30T13:10:24.457 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-03-30T13:10:24.457 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-30T13:10:24.457 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-03-30T13:10:24.457 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-30T13:10:24.459 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T13:10:24.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:24.459 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-30T13:10:24.488 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:24.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:24.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:24.585 INFO:teuthology.orchestra.run.smithi117.stdout:client4878 2024-03-30T13:10:24.585 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:24.585 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:24.601 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:24.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:24.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:24.695 INFO:teuthology.orchestra.run.smithi117.stdout:client4878 2024-03-30T13:10:24.696 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:10:24.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:10:24.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:24.751 INFO:teuthology.orchestra.run.smithi117.stdout:client4878 2024-03-30T13:10:24.751 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/03992909-781c-43b4-940b-d3d6d831de0b.client4878/status') 2024-03-30T13:10:24.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:10:24.812 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.4878 (3)172.21.15.117:0/2129586815 2024-03-30T13:10:24.812 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-03-30T13:10:24.813 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-03-30T13:10:24.813 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-03-30T13:10:24.813 INFO:teuthology.orchestra.run.smithi117.stderr:71 bytes copied, 2.1484e-05 s, 3.3 MB/s 2024-03-30T13:10:24.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:10:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:10:24.819 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-30T13:10:24.835 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:24.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.951+0000 7f390dc62640 1 Processor -- start 2024-03-30T13:10:24.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.951+0000 7f390dc62640 1 -- start start 2024-03-30T13:10:24.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f390dc62640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3908156a10 0x7f390806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:24.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f390dc62640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390815d970 con 0x7f390806da20 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f390dc62640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39080cc930 con 0x7f390815d300 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f390dc62640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f390806de80 con 0x7f3908156a10 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f39077fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3908156a10 0x7f390806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f3906ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f390806da20 0x7f390815cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57264/0 (socket says 172.21.15.66:57264) 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f3906ffd640 1 -- 172.21.15.66:0/3993685313 learned_addr learned my addr 172.21.15.66:0/3993685313 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f39077fe640 1 -- 172.21.15.66:0/3993685313 >> v1:172.21.15.117:6789/0 conn(0x7f390815d300 legacy=0x7f39080ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:24.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f39077fe640 1 -- 172.21.15.66:0/3993685313 >> v1:172.21.15.66:6789/0 conn(0x7f390806da20 legacy=0x7f390815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:24.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f39077fe640 1 -- 172.21.15.66:0/3993685313 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39080ccc10 con 0x7f3908156a10 2024-03-30T13:10:24.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.952+0000 7f39067fc640 1 -- 172.21.15.66:0/3993685313 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591392679 0 0) 0x7f390815d970 con 0x7f390806da20 2024-03-30T13:10:24.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.953+0000 7f39077fe640 1 --2- 172.21.15.66:0/3993685313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3908156a10 0x7f390806d4e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009d90 tx=0x7f38f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ab3363fecbbe08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:24.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.953+0000 7f39067fc640 1 -- 172.21.15.66:0/3993685313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8039070 con 0x7f3908156a10 2024-03-30T13:10:24.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.953+0000 7f39067fc640 1 -- 172.21.15.66:0/3993685313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f802cc60 con 0x7f3908156a10 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.953+0000 7f39067fc640 1 -- 172.21.15.66:0/3993685313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8034660 con 0x7f3908156a10 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 -- 172.21.15.66:0/3993685313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3908156a10 msgr2=0x7f390806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 --2- 172.21.15.66:0/3993685313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3908156a10 0x7f390806d4e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009d90 tx=0x7f38f802beb0 comp rx=0 tx=0).stop 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 -- 172.21.15.66:0/3993685313 shutdown_connections 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 --2- 172.21.15.66:0/3993685313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3908156a10 0x7f390806d4e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 -- 172.21.15.66:0/3993685313 >> 172.21.15.66:0/3993685313 conn(0x7f39080b1c90 msgr2=0x7f39080b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 -- 172.21.15.66:0/3993685313 shutdown_connections 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.954+0000 7f390dc62640 1 -- 172.21.15.66:0/3993685313 wait complete. 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.955+0000 7f390dc62640 1 Processor -- start 2024-03-30T13:10:24.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 -- start start 2024-03-30T13:10:24.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 0x7f3908159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:24.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3908156a10 0x7f39080b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:24.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f390815d300 0x7f39081f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:24.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3908156df0 con 0x7f390806da20 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f390815d730 con 0x7f390815d300 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f390dc62640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f39080cd6f0 con 0x7f3908156a10 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f39077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 0x7f3908159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f39077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 0x7f3908159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48110/0 (socket says 172.21.15.66:48110) 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f39077fe640 1 -- 172.21.15.66:0/2652206461 learned_addr learned my addr 172.21.15.66:0/2652206461 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f3906ffd640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3908156a10 0x7f39080b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.956+0000 7f3907fff640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f390815d300 0x7f39081f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f39077fe640 1 -- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3908156a10 msgr2=0x7f39080b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f39077fe640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3908156a10 0x7f39080b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:24.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f39077fe640 1 -- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f390815d300 msgr2=0x7f39081f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f39077fe640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f390815d300 0x7f39081f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f39077fe640 1 -- 172.21.15.66:0/2652206461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39081f50f0 con 0x7f390806da20 2024-03-30T13:10:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f39077fe640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 0x7f3908159a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f38f8037040 tx=0x7f38f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8039050 con 0x7f390806da20 2024-03-30T13:10:24.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f8045070 con 0x7f390806da20 2024-03-30T13:10:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39080b4650 con 0x7f390806da20 2024-03-30T13:10:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f803ea50 con 0x7f390806da20 2024-03-30T13:10:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.957+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39080b4c10 con 0x7f390806da20 2024-03-30T13:10:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.958+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3908075a10 con 0x7f390806da20 2024-03-30T13:10:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.959+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f38f802cb20 con 0x7f390806da20 2024-03-30T13:10:24.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.959+0000 7f38effff640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38f406c560 0x7f38f406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:24.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.959+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f38f80ae140 con 0x7f390806da20 2024-03-30T13:10:24.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.959+0000 7f3906ffd640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38f406c560 0x7f38f406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:24.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.960+0000 7f3906ffd640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38f406c560 0x7f38f406ea20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007f70 tx=0x7f38fc007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:24.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:24.963+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f38f804e020 con 0x7f390806da20 2024-03-30T13:10:25.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.151+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39081572b0 con 0x7f390806da20 2024-03-30T13:10:25.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.152+0000 7f38effff640 1 -- 172.21.15.66:0/2652206461 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f39081f50f0 con 0x7f390806da20 2024-03-30T13:10:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:25.157 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38f406c560 msgr2=0x7f38f406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38f406c560 0x7f38f406ea20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38fc007f70 tx=0x7f38fc007f00 comp rx=0 tx=0).stop 2024-03-30T13:10:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 msgr2=0x7f3908159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 0x7f3908159a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f38f8037040 tx=0x7f38f8004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 shutdown_connections 2024-03-30T13:10:25.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f38f406c560 0x7f38f406ea20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f390815d300 0x7f39081f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3908156a10 0x7f39080b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 --2- 172.21.15.66:0/2652206461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f390806da20 0x7f3908159a20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 >> 172.21.15.66:0/2652206461 conn(0x7f39080b1c90 msgr2=0x7f39080aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:25.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 shutdown_connections 2024-03-30T13:10:25.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.154+0000 7f390dc62640 1 -- 172.21.15.66:0/2652206461 wait complete. 2024-03-30T13:10:25.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:25.170 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.277+0000 7fe023cc1640 1 Processor -- start 2024-03-30T13:10:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.278+0000 7fe023cc1640 1 -- start start 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.278+0000 7fe023cc1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe01c167120 0x7fe01c169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.278+0000 7fe023cc1640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c0d2ff0 con 0x7fe01c0ba090 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.278+0000 7fe023cc1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c0d32d0 con 0x7fe01c0b5dc0 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.278+0000 7fe023cc1640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c0d2d90 con 0x7fe01c167120 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe02225e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe01c167120 0x7fe01c169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe02125c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe01c0ba090 0x7fe01c0d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57272/0 (socket says 172.21.15.66:57272) 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe02125c640 1 -- 172.21.15.66:0/2814383316 learned_addr learned my addr 172.21.15.66:0/2814383316 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe02225e640 1 -- 172.21.15.66:0/2814383316 >> v1:172.21.15.117:6789/0 conn(0x7fe01c0b5dc0 legacy=0x7fe01c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe02225e640 1 -- 172.21.15.66:0/2814383316 >> v1:172.21.15.66:6789/0 conn(0x7fe01c0ba090 legacy=0x7fe01c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe02225e640 1 -- 172.21.15.66:0/2814383316 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0d35b0 con 0x7fe01c167120 2024-03-30T13:10:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.279+0000 7fe020a5b640 1 -- 172.21.15.66:0/2814383316 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44402584 0 0) 0x7fe01c0d2ff0 con 0x7fe01c0ba090 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe02225e640 1 --2- 172.21.15.66:0/2814383316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c167120 0x7fe01c169510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe014009dc0 tx=0x7fe01402bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8887fb0d8cc8d1fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe020a5b640 1 -- 172.21.15.66:0/2814383316 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014039070 con 0x7fe01c167120 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe020a5b640 1 -- 172.21.15.66:0/2814383316 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01402ccd0 con 0x7fe01c167120 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe020a5b640 1 -- 172.21.15.66:0/2814383316 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014034660 con 0x7fe01c167120 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 -- 172.21.15.66:0/2814383316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c167120 msgr2=0x7fe01c169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 --2- 172.21.15.66:0/2814383316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c167120 0x7fe01c169510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe014009dc0 tx=0x7fe01402bf20 comp rx=0 tx=0).stop 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 -- 172.21.15.66:0/2814383316 shutdown_connections 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 --2- 172.21.15.66:0/2814383316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c167120 0x7fe01c169510 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 -- 172.21.15.66:0/2814383316 >> 172.21.15.66:0/2814383316 conn(0x7fe01c0678f0 msgr2=0x7fe01c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 -- 172.21.15.66:0/2814383316 shutdown_connections 2024-03-30T13:10:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.280+0000 7fe023cc1640 1 -- 172.21.15.66:0/2814383316 wait complete. 2024-03-30T13:10:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 Processor -- start 2024-03-30T13:10:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 -- start start 2024-03-30T13:10:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe01c0b5dc0 0x7fe01c161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 0x7fe01c0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe01c167120 0x7fe01c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0d4090 con 0x7fe01c0ba090 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0d3b70 con 0x7fe01c0b5dc0 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe023cc1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c0b83e0 con 0x7fe01c167120 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe02125c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 0x7fe01c0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe02125c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 0x7fe01c0cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48132/0 (socket says 172.21.15.66:48132) 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe02125c640 1 -- 172.21.15.66:0/3620482105 learned_addr learned my addr 172.21.15.66:0/3620482105 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe02225e640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe01c167120 0x7fe01c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.282+0000 7fe021a5d640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe01c0b5dc0 0x7fe01c161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe02125c640 1 -- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe01c167120 msgr2=0x7fe01c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe02125c640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe01c167120 0x7fe01c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe02125c640 1 -- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe01c0b5dc0 msgr2=0x7fe01c161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe02125c640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe01c0b5dc0 0x7fe01c161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe02125c640 1 -- 172.21.15.66:0/3620482105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0cc360 con 0x7fe01c0ba090 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe02125c640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 0x7fe01c0cbe20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c4a0 tx=0x7fe00c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c018070 con 0x7fe01c0ba090 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00c0074e0 con 0x7fe01c0ba090 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c0cc670 con 0x7fe01c0ba090 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c004a20 con 0x7fe01c0ba090 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.283+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0ccf10 con 0x7fe01c0ba090 2024-03-30T13:10:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.284+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c075890 con 0x7fe01c0ba090 2024-03-30T13:10:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.284+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe00c01b070 con 0x7fe01c0ba090 2024-03-30T13:10:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.285+0000 7fe0127fc640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe00006c530 0x7fe00006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.285+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fe00c08add0 con 0x7fe01c0ba090 2024-03-30T13:10:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.285+0000 7fe021a5d640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe00006c530 0x7fe00006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.285+0000 7fe021a5d640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe00006c530 0x7fe00006e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0b3270 tx=0x7fe018009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.289+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe00c058980 con 0x7fe01c0ba090 2024-03-30T13:10:25.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.472+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe01c0cfb90 con 0x7fe01c0ba090 2024-03-30T13:10:25.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.472+0000 7fe0127fc640 1 -- 172.21.15.66:0/3620482105 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fe00c08c020 con 0x7fe01c0ba090 2024-03-30T13:10:25.476 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:25.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe00006c530 msgr2=0x7fe00006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe00006c530 0x7fe00006e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0b3270 tx=0x7fe018009040 comp rx=0 tx=0).stop 2024-03-30T13:10:25.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 msgr2=0x7fe01c0cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 0x7fe01c0cbe20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c4a0 tx=0x7fe00c00c960 comp rx=0 tx=0).stop 2024-03-30T13:10:25.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 shutdown_connections 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe00006c530 0x7fe00006e9f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe01c167120 0x7fe01c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe01c0ba090 0x7fe01c0cbe20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 --2- 172.21.15.66:0/3620482105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe01c0b5dc0 0x7fe01c161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 >> 172.21.15.66:0/3620482105 conn(0x7fe01c0678f0 msgr2=0x7fe01c0ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 shutdown_connections 2024-03-30T13:10:25.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.474+0000 7fe023cc1640 1 -- 172.21.15.66:0/3620482105 wait complete. 2024-03-30T13:10:25.489 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:25.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.596+0000 7fafd8a38640 1 Processor -- start 2024-03-30T13:10:25.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.596+0000 7fafd8a38640 1 -- start start 2024-03-30T13:10:25.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd8a38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd4073310 0x7fafd406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd8a38640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fafd40d1be0 0x7fafd406bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd8a38640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd4073880 con 0x7fafd415e5b0 2024-03-30T13:10:25.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd8a38640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fafd40d0e20 con 0x7fafd40d1be0 2024-03-30T13:10:25.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd8a38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fafd40d1100 con 0x7fafd4073310 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd259c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fafd415e5b0 0x7fafd4162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45350/0 (socket says 172.21.15.66:45350) 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd259c640 1 -- 172.21.15.66:0/2884944070 learned_addr learned my addr 172.21.15.66:0/2884944070 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd2d9d640 1 --2- 172.21.15.66:0/2884944070 >> v2:172.21.15.117:3300/0 conn(0x7fafd40d1be0 0x7fafd406bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd1d9b640 1 --2- 172.21.15.66:0/2884944070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd4073310 0x7fafd406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd159a640 1 -- 172.21.15.66:0/2884944070 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769629823 0 0) 0x7fafd4073880 con 0x7fafd415e5b0 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.597+0000 7fafd159a640 1 -- 172.21.15.66:0/2884944070 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc0003520 con 0x7fafd415e5b0 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.598+0000 7fafd1d9b640 1 -- 172.21.15.66:0/2884944070 >> v2:172.21.15.117:3300/0 conn(0x7fafd40d1be0 msgr2=0x7fafd406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.598+0000 7fafd1d9b640 1 --2- 172.21.15.66:0/2884944070 >> v2:172.21.15.117:3300/0 conn(0x7fafd40d1be0 0x7fafd406bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.598+0000 7fafd1d9b640 1 -- 172.21.15.66:0/2884944070 >> v1:172.21.15.117:6789/0 conn(0x7fafd415e5b0 legacy=0x7fafd4162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.598+0000 7fafd1d9b640 1 -- 172.21.15.66:0/2884944070 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd40d13e0 con 0x7fafd4073310 2024-03-30T13:10:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.598+0000 7fafd2d9d640 1 --2- 172.21.15.66:0/2884944070 >> v2:172.21.15.117:3300/0 conn(0x7fafd40d1be0 0x7fafd406bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.598+0000 7fafd1d9b640 1 --2- 172.21.15.66:0/2884944070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd4073310 0x7fafd406b580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0097a0 tx=0x7fafbc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa521faf4200302 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd159a640 1 -- 172.21.15.66:0/2884944070 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0315a0 con 0x7fafd4073310 2024-03-30T13:10:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd159a640 1 -- 172.21.15.66:0/2884944070 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc037070 con 0x7fafd4073310 2024-03-30T13:10:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd159a640 1 -- 172.21.15.66:0/2884944070 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc031960 con 0x7fafd4073310 2024-03-30T13:10:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 -- 172.21.15.66:0/2884944070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd4073310 msgr2=0x7fafd406b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 --2- 172.21.15.66:0/2884944070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd4073310 0x7fafd406b580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0097a0 tx=0x7fafbc02b840 comp rx=0 tx=0).stop 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 -- 172.21.15.66:0/2884944070 shutdown_connections 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 --2- 172.21.15.66:0/2884944070 >> v2:172.21.15.117:3300/0 conn(0x7fafd40d1be0 0x7fafd406bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 --2- 172.21.15.66:0/2884944070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd4073310 0x7fafd406b580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 -- 172.21.15.66:0/2884944070 >> 172.21.15.66:0/2884944070 conn(0x7fafd406e720 msgr2=0x7fafd406eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 -- 172.21.15.66:0/2884944070 shutdown_connections 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.599+0000 7fafd8a38640 1 -- 172.21.15.66:0/2884944070 wait complete. 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 Processor -- start 2024-03-30T13:10:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 -- start start 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd4073310 0x7fafd415f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd40d1be0 0x7fafd41e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 0x7fafd41f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fafd406f9a0 con 0x7fafd415e5b0 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafd415e990 con 0x7fafd4073310 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd8a38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fafd406fb20 con 0x7fafd40d1be0 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 0x7fafd41f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd40d1be0 0x7fafd41e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 0x7fafd41f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48144/0 (socket says 172.21.15.66:48144) 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd1d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd40d1be0 0x7fafd41e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:43288/0 (socket says 172.21.15.66:43288) 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd4073310 0x7fafd415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.601+0000 7fafd2d9d640 1 -- 172.21.15.66:0/414293962 learned_addr learned my addr 172.21.15.66:0/414293962 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:25.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd2d9d640 1 -- 172.21.15.66:0/414293962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd40d1be0 msgr2=0x7fafd41e9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd2d9d640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd40d1be0 0x7fafd41e9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd2d9d640 1 -- 172.21.15.66:0/414293962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd4073310 msgr2=0x7fafd415f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd2d9d640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd4073310 0x7fafd415f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd2d9d640 1 -- 172.21.15.66:0/414293962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd415fde0 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd2d9d640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 0x7fafd41f3fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009f00 tx=0x7fafc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4018030 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc400ce50 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd41600c0 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003bb0 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.602+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd415fa60 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.603+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd41605e0 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.604+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fafc401b070 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.604+0000 7fafcaffd640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafb406c560 0x7fafb406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.604+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fafc408c500 con 0x7fafd415e5b0 2024-03-30T13:10:25.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.604+0000 7fafd259c640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafb406c560 0x7fafb406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.604+0000 7fafd259c640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafb406c560 0x7fafb406ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fafcc005880 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.608+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fafd41605e0 con 0x7fafd415e5b0 2024-03-30T13:10:25.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.791+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fafd41502b0 con 0x7fafd415e5b0 2024-03-30T13:10:25.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.791+0000 7fafcaffd640 1 -- 172.21.15.66:0/414293962 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fafc4010040 con 0x7fafd415e5b0 2024-03-30T13:10:25.795 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafb406c560 msgr2=0x7fafb406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafb406c560 0x7fafb406ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fafcc005880 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 msgr2=0x7fafd41f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 0x7fafd41f3fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009f00 tx=0x7fafc400c490 comp rx=0 tx=0).stop 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 shutdown_connections 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafb406c560 0x7fafb406ea20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafd415e5b0 0x7fafd41f3fd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafd40d1be0 0x7fafd41e9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 --2- 172.21.15.66:0/414293962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd4073310 0x7fafd415f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 >> 172.21.15.66:0/414293962 conn(0x7fafd406e720 msgr2=0x7fafd4158620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:25.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 shutdown_connections 2024-03-30T13:10:25.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.793+0000 7fafd8a38640 1 -- 172.21.15.66:0/414293962 wait complete. 2024-03-30T13:10:25.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 Processor -- start 2024-03-30T13:10:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 -- start start 2024-03-30T13:10:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44fc15e7c0 0x7f44fc06c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44fc15eba0 0x7f44fc06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0d1170 con 0x7f44fc0b4800 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f44fc0cc840 con 0x7f44fc15eba0 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.917+0000 7f4502b50640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f44fc0d1450 con 0x7f44fc15e7c0 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fbfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44fc15eba0 0x7f44fc06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f45010ed640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f44fc0b4800 0x7f44fc09aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57286/0 (socket says 172.21.15.66:57286) 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f45010ed640 1 -- 172.21.15.66:0/929640653 learned_addr learned my addr 172.21.15.66:0/929640653 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fbfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44fc15eba0 0x7f44fc06cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48156/0 (socket says 172.21.15.66:48156) 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f45008ec640 1 --2- 172.21.15.66:0/929640653 >> v2:172.21.15.117:3300/0 conn(0x7f44fc15e7c0 0x7f44fc06c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fbfff640 1 -- 172.21.15.66:0/929640653 >> v2:172.21.15.117:3300/0 conn(0x7f44fc15e7c0 msgr2=0x7f44fc06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fbfff640 1 --2- 172.21.15.66:0/929640653 >> v2:172.21.15.117:3300/0 conn(0x7f44fc15e7c0 0x7f44fc06c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fbfff640 1 -- 172.21.15.66:0/929640653 >> v1:172.21.15.66:6789/0 conn(0x7f44fc0b4800 legacy=0x7f44fc09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fb7fe640 1 -- 172.21.15.66:0/929640653 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593635447 0 0) 0x7f44fc0d1170 con 0x7f44fc0b4800 2024-03-30T13:10:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.918+0000 7f44fbfff640 1 -- 172.21.15.66:0/929640653 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc0d1730 con 0x7f44fc15eba0 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f44fbfff640 1 --2- 172.21.15.66:0/929640653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc06cda0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44fc09bc30 tx=0x7f44ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f09d59ac1501d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f44fb7fe640 1 -- 172.21.15.66:0/929640653 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec035020 con 0x7f44fc15eba0 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f44fb7fe640 1 -- 172.21.15.66:0/929640653 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec031830 con 0x7f44fc15eba0 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f44fb7fe640 1 -- 172.21.15.66:0/929640653 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec031c00 con 0x7f44fc15eba0 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 -- 172.21.15.66:0/929640653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 msgr2=0x7f44fc06cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 --2- 172.21.15.66:0/929640653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc06cda0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44fc09bc30 tx=0x7f44ec02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 -- 172.21.15.66:0/929640653 shutdown_connections 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 --2- 172.21.15.66:0/929640653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc06cda0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 --2- 172.21.15.66:0/929640653 >> v2:172.21.15.117:3300/0 conn(0x7f44fc15e7c0 0x7f44fc06c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 -- 172.21.15.66:0/929640653 >> 172.21.15.66:0/929640653 conn(0x7f44fc0b3ce0 msgr2=0x7f44fc0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 -- 172.21.15.66:0/929640653 shutdown_connections 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.919+0000 7f4502b50640 1 -- 172.21.15.66:0/929640653 wait complete. 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 Processor -- start 2024-03-30T13:10:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 -- start start 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc0b4800 0x7f44fc151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc15e7c0 0x7f44fc1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0cde60 con 0x7f44fc15eba0 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0cdce0 con 0x7f44fc0b4800 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.921+0000 7f4502b50640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc0cdfe0 con 0x7f44fc15e7c0 2024-03-30T13:10:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48172/0 (socket says 172.21.15.66:48172) 2024-03-30T13:10:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 -- 172.21.15.66:0/3917026373 learned_addr learned my addr 172.21.15.66:0/3917026373 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f44fbfff640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc15e7c0 0x7f44fc1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45008ec640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc0b4800 0x7f44fc151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 -- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc15e7c0 msgr2=0x7f44fc1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc15e7c0 0x7f44fc1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 -- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc0b4800 msgr2=0x7f44fc151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc0b4800 0x7f44fc151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 -- 172.21.15.66:0/3917026373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc1617a0 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f45010ed640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc160b30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c410 tx=0x7f44e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40074e0 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e40047e0 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc161a80 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004c10 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.922+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc0c6ef0 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.923+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc075890 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.924+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f44e4011070 con 0x7f44fc15eba0 2024-03-30T13:10:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.924+0000 7f44f97fa640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f44dc06c530 0x7f44dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:25.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.924+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f44e408b520 con 0x7f44fc15eba0 2024-03-30T13:10:25.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.924+0000 7f45008ec640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f44dc06c530 0x7f44dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:25.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.925+0000 7f45008ec640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f44dc06c530 0x7f44dc06e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f44f40113e0 tx=0x7f44f400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:25.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:25.929+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f44e4059160 con 0x7f44fc15eba0 2024-03-30T13:10:26.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.113+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44fc0d0df0 con 0x7f44fc15eba0 2024-03-30T13:10:26.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.113+0000 7f44f97fa640 1 -- 172.21.15.66:0/3917026373 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f44e405d460 con 0x7f44fc15eba0 2024-03-30T13:10:26.117 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:26.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f44dc06c530 msgr2=0x7f44dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f44dc06c530 0x7f44dc06e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f44f40113e0 tx=0x7f44f400b040 comp rx=0 tx=0).stop 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 msgr2=0x7f44fc160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc160b30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c410 tx=0x7f44e400c8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 shutdown_connections 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f44dc06c530 0x7f44dc06e9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44fc15eba0 0x7f44fc160b30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44fc15e7c0 0x7f44fc1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 --2- 172.21.15.66:0/3917026373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc0b4800 0x7f44fc151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 >> 172.21.15.66:0/3917026373 conn(0x7f44fc0b3ce0 msgr2=0x7f44fc0cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 shutdown_connections 2024-03-30T13:10:26.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.115+0000 7f4502b50640 1 -- 172.21.15.66:0/3917026373 wait complete. 2024-03-30T13:10:26.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.237+0000 7f0cde972640 1 Processor -- start 2024-03-30T13:10:26.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.237+0000 7f0cde972640 1 -- start start 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cde972640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd80b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cde972640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0cd80ba090 0x7f0cd80d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cde972640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd80b66e0 con 0x7f0cd806c620 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cde972640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd80d2ff0 con 0x7f0cd80ba090 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cde972640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd80d32d0 con 0x7f0cd80b5dc0 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cdcf0f640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0cd806c620 0x7f0cd806cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57300/0 (socket says 172.21.15.66:57300) 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cdcf0f640 1 -- 172.21.15.66:0/3226229462 learned_addr learned my addr 172.21.15.66:0/3226229462 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd7fff640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd80b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd77fe640 1 --2- 172.21.15.66:0/3226229462 >> v2:172.21.15.117:3300/0 conn(0x7f0cd80ba090 0x7f0cd80d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd6ffd640 1 -- 172.21.15.66:0/3226229462 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423693384 0 0) 0x7f0cd80b66e0 con 0x7f0cd806c620 2024-03-30T13:10:26.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd6ffd640 1 -- 172.21.15.66:0/3226229462 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cc0003520 con 0x7f0cd806c620 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd77fe640 1 -- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 msgr2=0x7f0cd80b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd77fe640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd80b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd77fe640 1 -- 172.21.15.66:0/3226229462 >> v1:172.21.15.66:6789/0 conn(0x7f0cd806c620 legacy=0x7f0cd806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd77fe640 1 -- 172.21.15.66:0/3226229462 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd80d35b0 con 0x7f0cd80ba090 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.238+0000 7f0cd7fff640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd80b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.239+0000 7f0cd77fe640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd80ba090 0x7f0cd80d2680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0097a0 tx=0x7f0ccc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5074bd627ea97f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.239+0000 7f0cd6ffd640 1 -- 172.21.15.66:0/3226229462 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc035020 con 0x7f0cd80ba090 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.239+0000 7f0cd6ffd640 1 -- 172.21.15.66:0/3226229462 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ccc031830 con 0x7f0cd80ba090 2024-03-30T13:10:26.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.239+0000 7f0cd6ffd640 1 -- 172.21.15.66:0/3226229462 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc031be0 con 0x7f0cd80ba090 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 -- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd80ba090 msgr2=0x7f0cd80d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd80ba090 0x7f0cd80d2680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0097a0 tx=0x7f0ccc02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 -- 172.21.15.66:0/3226229462 shutdown_connections 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd80ba090 0x7f0cd80d2680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 --2- 172.21.15.66:0/3226229462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd80b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 -- 172.21.15.66:0/3226229462 >> 172.21.15.66:0/3226229462 conn(0x7f0cd80678f0 msgr2=0x7f0cd80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 -- 172.21.15.66:0/3226229462 shutdown_connections 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.240+0000 7f0cde972640 1 -- 172.21.15.66:0/3226229462 wait complete. 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.241+0000 7f0cde972640 1 Processor -- start 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.241+0000 7f0cde972640 1 -- start start 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cde972640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd806c620 0x7f0cd8154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cde972640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd8158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cde972640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 0x7f0cd8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cde972640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd80d4090 con 0x7f0cd80ba090 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cde972640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd80d3b70 con 0x7f0cd806c620 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cde972640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd80b83e0 con 0x7f0cd80b5dc0 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 0x7f0cd8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd806c620 0x7f0cd8154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cd77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 0x7f0cd8160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48174/0 (socket says 172.21.15.66:48174) 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 -- 172.21.15.66:0/3288107136 learned_addr learned my addr 172.21.15.66:0/3288107136 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:26.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 -- 172.21.15.66:0/3288107136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 msgr2=0x7f0cd8158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd8158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 -- 172.21.15.66:0/3288107136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd806c620 msgr2=0x7f0cd8154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd806c620 0x7f0cd8154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.242+0000 7f0cdcf0f640 1 -- 172.21.15.66:0/3288107136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd81573a0 con 0x7f0cd80ba090 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.243+0000 7f0cdcf0f640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 0x7f0cd8160d00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000cb40 tx=0x7f0cd000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.243+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0017070 con 0x7f0cd80ba090 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.243+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd000aca0 con 0x7f0cd80ba090 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.243+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd8157680 con 0x7f0cd80ba090 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.243+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0012620 con 0x7f0cd80ba090 2024-03-30T13:10:26.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.243+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd8157c40 con 0x7f0cd80ba090 2024-03-30T13:10:26.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.244+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd8075890 con 0x7f0cd80ba090 2024-03-30T13:10:26.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.244+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0cd001a070 con 0x7f0cd80ba090 2024-03-30T13:10:26.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.245+0000 7f0cd4ff9640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0cc406c530 0x7f0cc406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.245+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f0cd008b950 con 0x7f0cd80ba090 2024-03-30T13:10:26.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.245+0000 7f0cd7fff640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0cc406c530 0x7f0cc406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.245+0000 7f0cd7fff640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0cc406c530 0x7f0cc406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009560 tx=0x7f0cc8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.249+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0cd0059480 con 0x7f0cd80ba090 2024-03-30T13:10:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.438+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0cd80b3350 con 0x7f0cd80ba090 2024-03-30T13:10:26.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.438+0000 7f0cd4ff9640 1 -- 172.21.15.66:0/3288107136 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f0cd005d780 con 0x7f0cd80ba090 2024-03-30T13:10:26.443 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0cc406c530 msgr2=0x7f0cc406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0cc406c530 0x7f0cc406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009560 tx=0x7f0cc8009190 comp rx=0 tx=0).stop 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 msgr2=0x7f0cd8160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 0x7f0cd8160d00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000cb40 tx=0x7f0cd000a060 comp rx=0 tx=0).stop 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 shutdown_connections 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0cc406c530 0x7f0cc406e9f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cd80ba090 0x7f0cd8160d00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cd80b5dc0 0x7f0cd8158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 --2- 172.21.15.66:0/3288107136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd806c620 0x7f0cd8154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 >> 172.21.15.66:0/3288107136 conn(0x7f0cd80678f0 msgr2=0x7f0cd80b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:26.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 shutdown_connections 2024-03-30T13:10:26.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.440+0000 7f0cde972640 1 -- 172.21.15.66:0/3288107136 wait complete. 2024-03-30T13:10:26.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:26.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 Processor -- start 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 -- start start 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d4a90 0x7f3db00d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3db00d2f00 0x7f3db00b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db00d36f0 con 0x7f3db015e950 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3db00d3cb0 con 0x7f3db00d2f00 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.559+0000 7f3db6990640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3db00d39d0 con 0x7f3db00d4a90 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3db4f2d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3db00d2f00 0x7f3db00b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3daffff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3db015e950 0x7f3db00d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45356/0 (socket says 172.21.15.66:45356) 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3daffff640 1 -- 172.21.15.66:0/864693361 learned_addr learned my addr 172.21.15.66:0/864693361 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3daf7fe640 1 --2- 172.21.15.66:0/864693361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d4a90 0x7f3db00d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3db4f2d640 1 -- 172.21.15.66:0/864693361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d4a90 msgr2=0x7f3db00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:26.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3db4f2d640 1 --2- 172.21.15.66:0/864693361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d4a90 0x7f3db00d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3db4f2d640 1 -- 172.21.15.66:0/864693361 >> v1:172.21.15.117:6789/0 conn(0x7f3db015e950 legacy=0x7f3db00d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3db4f2d640 1 -- 172.21.15.66:0/864693361 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db00b7ac0 con 0x7f3db00d2f00 2024-03-30T13:10:26.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.560+0000 7f3daeffd640 1 -- 172.21.15.66:0/864693361 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95284698 0 0) 0x7f3db00d36f0 con 0x7f3db015e950 2024-03-30T13:10:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db4f2d640 1 --2- 172.21.15.66:0/864693361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d2f00 0x7f3db00b82c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d90 tx=0x7f3da802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130b941fee258670 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3daeffd640 1 -- 172.21.15.66:0/864693361 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8039070 con 0x7f3db00d2f00 2024-03-30T13:10:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3daeffd640 1 -- 172.21.15.66:0/864693361 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da802cce0 con 0x7f3db00d2f00 2024-03-30T13:10:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3daeffd640 1 -- 172.21.15.66:0/864693361 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8034660 con 0x7f3db00d2f00 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 -- 172.21.15.66:0/864693361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d2f00 msgr2=0x7f3db00b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 --2- 172.21.15.66:0/864693361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d2f00 0x7f3db00b82c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d90 tx=0x7f3da802bf30 comp rx=0 tx=0).stop 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 -- 172.21.15.66:0/864693361 shutdown_connections 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 --2- 172.21.15.66:0/864693361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d2f00 0x7f3db00b82c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 --2- 172.21.15.66:0/864693361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d4a90 0x7f3db00d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 -- 172.21.15.66:0/864693361 >> 172.21.15.66:0/864693361 conn(0x7f3db00676e0 msgr2=0x7f3db0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 -- 172.21.15.66:0/864693361 shutdown_connections 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.561+0000 7f3db6990640 1 -- 172.21.15.66:0/864693361 wait complete. 2024-03-30T13:10:26.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 Processor -- start 2024-03-30T13:10:26.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 -- start start 2024-03-30T13:10:26.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d2f00 0x7f3db0159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 0x7f3db014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db015e950 0x7f3db015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3db015ed30 con 0x7f3db00d4a90 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3db00d33b0 con 0x7f3db015e950 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.563+0000 7f3db6990640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3db015eeb0 con 0x7f3db00d2f00 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 0x7f3db014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 0x7f3db014ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48198/0 (socket says 172.21.15.66:48198) 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 -- 172.21.15.66:0/2945263556 learned_addr learned my addr 172.21.15.66:0/2945263556 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daffff640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d2f00 0x7f3db0159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3db4f2d640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db015e950 0x7f3db015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 -- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d2f00 msgr2=0x7f3db0159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d2f00 0x7f3db0159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 -- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db015e950 msgr2=0x7f3db015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db015e950 0x7f3db015acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 -- 172.21.15.66:0/2945263556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db0152b00 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3daf7fe640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 0x7f3db014ec70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3da000e9b0 tx=0x7f3da000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0017070 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da000c9a0 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db014f1b0 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da000cd90 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.564+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db0152dc0 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.565+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db0075a10 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.566+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3da001a070 con 0x7f3db00d4a90 2024-03-30T13:10:26.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.566+0000 7f3dacff9640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3d9c06c530 0x7f3d9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.566+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f3da008ba50 con 0x7f3db00d4a90 2024-03-30T13:10:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.566+0000 7f3daffff640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3d9c06c530 0x7f3d9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.567+0000 7f3daffff640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3d9c06c530 0x7f3d9c06e9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3da4007fd0 tx=0x7f3da4007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.571+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3da0059690 con 0x7f3db00d4a90 2024-03-30T13:10:26.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.755+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3db00b6f20 con 0x7f3db00d4a90 2024-03-30T13:10:26.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.756+0000 7f3dacff9640 1 -- 172.21.15.66:0/2945263556 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f3da005d990 con 0x7f3db00d4a90 2024-03-30T13:10:26.760 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:26.760 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:26.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3d9c06c530 msgr2=0x7f3d9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3d9c06c530 0x7f3d9c06e9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3da4007fd0 tx=0x7f3da4007de0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 msgr2=0x7f3db014ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 0x7f3db014ec70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3da000e9b0 tx=0x7f3da000ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 shutdown_connections 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3d9c06c530 0x7f3d9c06e9f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db015e950 0x7f3db015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db00d4a90 0x7f3db014ec70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 --2- 172.21.15.66:0/2945263556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db00d2f00 0x7f3db0159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.758+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 >> 172.21.15.66:0/2945263556 conn(0x7f3db00676e0 msgr2=0x7f3db00cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.759+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 shutdown_connections 2024-03-30T13:10:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.759+0000 7f3db6990640 1 -- 172.21.15.66:0/2945263556 wait complete. 2024-03-30T13:10:26.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:26.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:26.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.881+0000 7f61b5b7c640 1 Processor -- start 2024-03-30T13:10:26.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.881+0000 7f61b5b7c640 1 -- start start 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.881+0000 7f61b5b7c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61b013f0c0 0x7f61b006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.881+0000 7f61b5b7c640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b00ce9c0 con 0x7f61b006d720 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.881+0000 7f61b5b7c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b0160b50 con 0x7f61b01605a0 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.881+0000 7f61b5b7c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f61b0160e30 con 0x7f61b013f0c0 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61aeffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f61b006d720 0x7f61b015fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57304/0 (socket says 172.21.15.66:57304) 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61aeffd640 1 -- 172.21.15.66:0/1349418856 learned_addr learned my addr 172.21.15.66:0/1349418856 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61af7fe640 1 --2- 172.21.15.66:0/1349418856 >> v2:172.21.15.117:3300/0 conn(0x7f61b013f0c0 0x7f61b006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2378599485 0 0) 0x7f61a0001110 con 0x7f61b006d720 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6198003520 con 0x7f61b006d720 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3025341099 0 0) 0x7f61b0160b50 con 0x7f61b01605a0 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61a0001110 con 0x7f61b01605a0 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61af7fe640 1 -- 172.21.15.66:0/1349418856 >> v1:172.21.15.117:6789/0 conn(0x7f61b01605a0 legacy=0x7f61b00cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61af7fe640 1 -- 172.21.15.66:0/1349418856 >> v1:172.21.15.66:6789/0 conn(0x7f61b006d720 legacy=0x7f61b015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.882+0000 7f61af7fe640 1 -- 172.21.15.66:0/1349418856 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b013fa40 con 0x7f61b013f0c0 2024-03-30T13:10:26.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.883+0000 7f61af7fe640 1 --2- 172.21.15.66:0/1349418856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b013f0c0 0x7f61b006d1e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f61a40097a0 tx=0x7f61a402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9c7420d4a7da22 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.883+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a40315a0 con 0x7f61b013f0c0 2024-03-30T13:10:26.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.883+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a4037070 con 0x7f61b013f0c0 2024-03-30T13:10:26.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.883+0000 7f61ae7fc640 1 -- 172.21.15.66:0/1349418856 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4031960 con 0x7f61b013f0c0 2024-03-30T13:10:26.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.883+0000 7f61b5b7c640 1 -- 172.21.15.66:0/1349418856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b013f0c0 msgr2=0x7f61b006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.883+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/1349418856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b013f0c0 0x7f61b006d1e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f61a40097a0 tx=0x7f61a402b840 comp rx=0 tx=0).stop 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.884+0000 7f61b5b7c640 1 -- 172.21.15.66:0/1349418856 shutdown_connections 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.884+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/1349418856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b013f0c0 0x7f61b006d1e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.884+0000 7f61b5b7c640 1 -- 172.21.15.66:0/1349418856 >> 172.21.15.66:0/1349418856 conn(0x7f61b00b5b00 msgr2=0x7f61b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.884+0000 7f61b5b7c640 1 -- 172.21.15.66:0/1349418856 shutdown_connections 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.884+0000 7f61b5b7c640 1 -- 172.21.15.66:0/1349418856 wait complete. 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.885+0000 7f61b5b7c640 1 Processor -- start 2024-03-30T13:10:26.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.885+0000 7f61b5b7c640 1 -- start start 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b006d720 0x7f61b0142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 0x7f61b013c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61b01605a0 0x7f61b00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61b013f4a0 con 0x7f61b013f0c0 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0140000 con 0x7f61b006d720 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f61b013f620 con 0x7f61b01605a0 2024-03-30T13:10:26.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 0x7f61b013c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 0x7f61b013c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48202/0 (socket says 172.21.15.66:48202) 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 -- 172.21.15.66:0/2056162588 learned_addr learned my addr 172.21.15.66:0/2056162588 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61af7fe640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b006d720 0x7f61b0142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61affff640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61b01605a0 0x7f61b00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 -- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61b01605a0 msgr2=0x7f61b00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61b01605a0 0x7f61b00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 -- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b006d720 msgr2=0x7f61b0142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b006d720 0x7f61b0142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 -- 172.21.15.66:0/2056162588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0141070 con 0x7f61b013f0c0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61aeffd640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 0x7f61b013c8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f61a000c730 tx=0x7f61a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0017070 con 0x7f61b013f0c0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a0005ab0 con 0x7f61b013f0c0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b0141350 con 0x7f61b013f0c0 2024-03-30T13:10:26.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0007070 con 0x7f61b013f0c0 2024-03-30T13:10:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.886+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b0141910 con 0x7f61b013f0c0 2024-03-30T13:10:26.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.887+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b0075890 con 0x7f61b013f0c0 2024-03-30T13:10:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.888+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f61a001a070 con 0x7f61b013f0c0 2024-03-30T13:10:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.888+0000 7f6197fff640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f619c06c530 0x7f619c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.888+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f61a008af10 con 0x7f61b013f0c0 2024-03-30T13:10:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.888+0000 7f61af7fe640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f619c06c530 0x7f619c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:26.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.889+0000 7f61af7fe640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f619c06c530 0x7f619c06e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f61a4036000 tx=0x7f61a403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:26.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:26.893+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f61b00ce9c0 con 0x7f61b013f0c0 2024-03-30T13:10:27.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.077+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61b006c2c0 con 0x7f61b013f0c0 2024-03-30T13:10:27.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.077+0000 7f6197fff640 1 -- 172.21.15.66:0/2056162588 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f61a0058b50 con 0x7f61b013f0c0 2024-03-30T13:10:27.081 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f619c06c530 msgr2=0x7f619c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f619c06c530 0x7f619c06e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f61a4036000 tx=0x7f61a403d040 comp rx=0 tx=0).stop 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 msgr2=0x7f61b013c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 0x7f61b013c8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f61a000c730 tx=0x7f61a000cbf0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 shutdown_connections 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f619c06c530 0x7f619c06e9f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61b01605a0 0x7f61b00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b013f0c0 0x7f61b013c8f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 --2- 172.21.15.66:0/2056162588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61b006d720 0x7f61b0142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 >> 172.21.15.66:0/2056162588 conn(0x7f61b00b5b00 msgr2=0x7f61b013dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:27.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 shutdown_connections 2024-03-30T13:10:27.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.079+0000 7f61b5b7c640 1 -- 172.21.15.66:0/2056162588 wait complete. 2024-03-30T13:10:27.094 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:27.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.202+0000 7f420c315640 1 Processor -- start 2024-03-30T13:10:27.206 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.202+0000 7f420c315640 1 -- start start 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f420c315640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4204144cb0 0x7f42040b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f420c315640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f420415f800 0x7f4204072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f420c315640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42040b35e0 con 0x7f4204072b80 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f420c315640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4204072fd0 con 0x7f420415f800 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f420c315640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f42040b3e80 con 0x7f4204144cb0 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f420415f800 0x7f4204072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f420415f800 0x7f4204072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48208/0 (socket says 172.21.15.66:48208) 2024-03-30T13:10:27.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 -- 172.21.15.66:0/1626241784 learned_addr learned my addr 172.21.15.66:0/1626241784 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f420a0b1640 1 --2- 172.21.15.66:0/1626241784 >> v2:172.21.15.117:3300/0 conn(0x7f4204144cb0 0x7f42040b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 -- 172.21.15.66:0/1626241784 >> v2:172.21.15.117:3300/0 conn(0x7f4204144cb0 msgr2=0x7f42040b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 --2- 172.21.15.66:0/1626241784 >> v2:172.21.15.117:3300/0 conn(0x7f4204144cb0 0x7f42040b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 -- 172.21.15.66:0/1626241784 >> v1:172.21.15.117:6789/0 conn(0x7f4204072b80 legacy=0x7f4204071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.203+0000 7f42098b0640 1 -- 172.21.15.66:0/1626241784 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040b38c0 con 0x7f420415f800 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.204+0000 7f42098b0640 1 --2- 172.21.15.66:0/1626241784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f420415f800 0x7f4204072640 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f420406e3e0 tx=0x7f41f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bb534ae3c942e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.204+0000 7f42090af640 1 -- 172.21.15.66:0/1626241784 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4034070 con 0x7f420415f800 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.204+0000 7f42090af640 1 -- 172.21.15.66:0/1626241784 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4030860 con 0x7f420415f800 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.204+0000 7f42090af640 1 -- 172.21.15.66:0/1626241784 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4030c30 con 0x7f420415f800 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 -- 172.21.15.66:0/1626241784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f420415f800 msgr2=0x7f4204072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 --2- 172.21.15.66:0/1626241784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f420415f800 0x7f4204072640 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f420406e3e0 tx=0x7f41f402b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 -- 172.21.15.66:0/1626241784 shutdown_connections 2024-03-30T13:10:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 --2- 172.21.15.66:0/1626241784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f420415f800 0x7f4204072640 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 --2- 172.21.15.66:0/1626241784 >> v2:172.21.15.117:3300/0 conn(0x7f4204144cb0 0x7f42040b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 -- 172.21.15.66:0/1626241784 >> 172.21.15.66:0/1626241784 conn(0x7f42040b5b00 msgr2=0x7f420406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 -- 172.21.15.66:0/1626241784 shutdown_connections 2024-03-30T13:10:27.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.205+0000 7f420c315640 1 -- 172.21.15.66:0/1626241784 wait complete. 2024-03-30T13:10:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.206+0000 7f420c315640 1 Processor -- start 2024-03-30T13:10:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.206+0000 7f420c315640 1 -- start start 2024-03-30T13:10:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4204072b80 0x7f420415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 0x7f42040af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420415f800 0x7f420413cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f420415fc40 con 0x7f4204144cb0 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42040cdb30 con 0x7f420415f800 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f42040cf060 con 0x7f4204072b80 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 0x7f42040af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 0x7f42040af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48210/0 (socket says 172.21.15.66:48210) 2024-03-30T13:10:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 -- 172.21.15.66:0/3458076624 learned_addr learned my addr 172.21.15.66:0/3458076624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420a0b1640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4204072b80 0x7f420415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420a8b2640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420415f800 0x7f420413cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 -- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4204072b80 msgr2=0x7f420415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4204072b80 0x7f420415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 -- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420415f800 msgr2=0x7f420413cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420415f800 0x7f420413cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 -- 172.21.15.66:0/3458076624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040af8a0 con 0x7f4204144cb0 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f42098b0640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 0x7f42040af360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41f402b9e0 tx=0x7f41f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4034040 con 0x7f4204144cb0 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4039d90 con 0x7f4204144cb0 2024-03-30T13:10:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420413f270 con 0x7f4204144cb0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4043610 con 0x7f4204144cb0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.207+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420413e030 con 0x7f4204144cb0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.208+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4204075890 con 0x7f4204144cb0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.209+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41f4037070 con 0x7f4204144cb0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.209+0000 7f41faffd640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41e806c530 0x7f41e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.209+0000 7f420a0b1640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41e806c530 0x7f41e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.210+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f41f40adad0 con 0x7f4204144cb0 2024-03-30T13:10:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.210+0000 7f420a0b1640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41e806c530 0x7f41e806e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4200009ad0 tx=0x7f4200008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.214+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41f403e070 con 0x7f4204144cb0 2024-03-30T13:10:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.401+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42040b4030 con 0x7f4204144cb0 2024-03-30T13:10:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.401+0000 7f41faffd640 1 -- 172.21.15.66:0/3458076624 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f41f407bd60 con 0x7f4204144cb0 2024-03-30T13:10:27.405 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41e806c530 msgr2=0x7f41e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41e806c530 0x7f41e806e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4200009ad0 tx=0x7f4200008040 comp rx=0 tx=0).stop 2024-03-30T13:10:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 msgr2=0x7f42040af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 0x7f42040af360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41f402b9e0 tx=0x7f41f40046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 shutdown_connections 2024-03-30T13:10:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41e806c530 0x7f41e806e9f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f420415f800 0x7f420413cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4204144cb0 0x7f42040af360 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 --2- 172.21.15.66:0/3458076624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4204072b80 0x7f420415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 >> 172.21.15.66:0/3458076624 conn(0x7f42040b5b00 msgr2=0x7f420406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 shutdown_connections 2024-03-30T13:10:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.403+0000 7f420c315640 1 -- 172.21.15.66:0/3458076624 wait complete. 2024-03-30T13:10:27.418 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:27.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.525+0000 7fb7455bd640 1 Processor -- start 2024-03-30T13:10:27.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.525+0000 7fb7455bd640 1 -- start start 2024-03-30T13:10:27.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.526+0000 7fb7455bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb74015e570 0x7fb7400b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb7455bd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb7400cf7f0 0x7fb7400d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb7455bd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb74015e950 con 0x7fb7400ba0b0 2024-03-30T13:10:27.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb7455bd640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fb7400b94a0 con 0x7fb7400cf7f0 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb7455bd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb7400b8890 con 0x7fb74015e570 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73e7fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb7400ba0b0 0x7fb7400cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57308/0 (socket says 172.21.15.66:57308) 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73effd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb74015e570 0x7fb7400b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73e7fc640 1 -- 172.21.15.66:0/3768891455 learned_addr learned my addr 172.21.15.66:0/3768891455 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73f7fe640 1 --2- 172.21.15.66:0/3768891455 >> v2:172.21.15.117:3300/0 conn(0x7fb7400cf7f0 0x7fb7400d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73dffb640 1 -- 172.21.15.66:0/3768891455 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711798845 0 0) 0x7fb74015e950 con 0x7fb7400ba0b0 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73dffb640 1 -- 172.21.15.66:0/3768891455 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb72c003520 con 0x7fb7400ba0b0 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73f7fe640 1 -- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb74015e570 msgr2=0x7fb7400b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73f7fe640 1 --2- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb74015e570 0x7fb7400b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73f7fe640 1 -- 172.21.15.66:0/3768891455 >> v1:172.21.15.66:6789/0 conn(0x7fb7400ba0b0 legacy=0x7fb7400cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73f7fe640 1 -- 172.21.15.66:0/3768891455 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7400b8b70 con 0x7fb7400cf7f0 2024-03-30T13:10:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.527+0000 7fb73effd640 1 --2- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb74015e570 0x7fb7400b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.528+0000 7fb73f7fe640 1 --2- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400cf7f0 0x7fb7400d2730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb730009d90 tx=0x7fb73002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5845efe04df8998a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.528+0000 7fb73dffb640 1 -- 172.21.15.66:0/3768891455 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730039070 con 0x7fb7400cf7f0 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.528+0000 7fb73dffb640 1 -- 172.21.15.66:0/3768891455 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73002cc50 con 0x7fb7400cf7f0 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.528+0000 7fb73dffb640 1 -- 172.21.15.66:0/3768891455 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730034440 con 0x7fb7400cf7f0 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 -- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400cf7f0 msgr2=0x7fb7400d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 --2- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400cf7f0 0x7fb7400d2730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb730009d90 tx=0x7fb73002bea0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 -- 172.21.15.66:0/3768891455 shutdown_connections 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 --2- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400cf7f0 0x7fb7400d2730 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 --2- 172.21.15.66:0/3768891455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb74015e570 0x7fb7400b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 -- 172.21.15.66:0/3768891455 >> 172.21.15.66:0/3768891455 conn(0x7fb7400676e0 msgr2=0x7fb740067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 -- 172.21.15.66:0/3768891455 shutdown_connections 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.529+0000 7fb7455bd640 1 -- 172.21.15.66:0/3768891455 wait complete. 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.530+0000 7fb7455bd640 1 Processor -- start 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.530+0000 7fb7455bd640 1 -- start start 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb7455bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400ba0b0 0x7fb7401f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb7455bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 0x7fb740152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb7455bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 0x7fb7401528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb7455bd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400b84d0 con 0x7fb74015e570 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb7455bd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400b9130 con 0x7fb7400ba0b0 2024-03-30T13:10:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb7455bd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb7400b8650 con 0x7fb7400cf7f0 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 0x7fb7401528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73e7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 0x7fb740152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400ba0b0 0x7fb7401f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73e7fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 0x7fb740152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:43356/0 (socket says 172.21.15.66:43356) 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 0x7fb7401528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48216/0 (socket says 172.21.15.66:48216) 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73e7fc640 1 -- 172.21.15.66:0/1325704697 learned_addr learned my addr 172.21.15.66:0/1325704697 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 -- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 msgr2=0x7fb740152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 0x7fb740152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 -- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400ba0b0 msgr2=0x7fb7401f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400ba0b0 0x7fb7401f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 -- 172.21.15.66:0/1325704697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7401565c0 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73e7fc640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 0x7fb740152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.531+0000 7fb73f7fe640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 0x7fb7401528a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb730033000 tx=0x7fb730004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.532+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730039050 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.532+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb730045070 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.532+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7401568a0 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.532+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730009450 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.532+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb740156e60 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.533+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb740075890 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.533+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb73003e610 con 0x7fb74015e570 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.533+0000 7fb7377fe640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb72006c530 0x7fb72006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.534+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fb7300ae400 con 0x7fb74015e570 2024-03-30T13:10:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.534+0000 7fb73effd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb72006c530 0x7fb72006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.534+0000 7fb73effd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb72006c530 0x7fb72006e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb738009560 tx=0x7fb738009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.538+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb73007bf30 con 0x7fb74015e570 2024-03-30T13:10:27.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.724+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7400d35e0 con 0x7fb74015e570 2024-03-30T13:10:27.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.724+0000 7fb7377fe640 1 -- 172.21.15.66:0/1325704697 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fb730080230 con 0x7fb74015e570 2024-03-30T13:10:27.728 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb72006c530 msgr2=0x7fb72006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb72006c530 0x7fb72006e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb738009560 tx=0x7fb738009190 comp rx=0 tx=0).stop 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 msgr2=0x7fb7401528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 0x7fb7401528a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb730033000 tx=0x7fb730004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 shutdown_connections 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb72006c530 0x7fb72006e9f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74015e570 0x7fb7401528a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7400cf7f0 0x7fb740152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 --2- 172.21.15.66:0/1325704697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7400ba0b0 0x7fb7401f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 >> 172.21.15.66:0/1325704697 conn(0x7fb7400676e0 msgr2=0x7fb7400d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:27.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 shutdown_connections 2024-03-30T13:10:27.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.726+0000 7fb7455bd640 1 -- 172.21.15.66:0/1325704697 wait complete. 2024-03-30T13:10:27.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:27.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.847+0000 7f30ce414640 1 Processor -- start 2024-03-30T13:10:27.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.847+0000 7f30ce414640 1 -- start start 2024-03-30T13:10:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30ce414640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f30c815e4b0 0x7f30c80cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30ce414640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c80ccf50 0x7f30c8162500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30ce414640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c8162c00 con 0x7f30c8162090 2024-03-30T13:10:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30ce414640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f30c8162ee0 con 0x7f30c815e4b0 2024-03-30T13:10:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30ce414640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f30c80cebb0 con 0x7f30c80ccf50 2024-03-30T13:10:27.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f30c815e4b0 0x7f30c80cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f30c815e4b0 0x7f30c80cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48220/0 (socket says 172.21.15.66:48220) 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 -- 172.21.15.66:0/3209184485 learned_addr learned my addr 172.21.15.66:0/3209184485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30cc9b1640 1 --2- 172.21.15.66:0/3209184485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c80ccf50 0x7f30c8162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 -- 172.21.15.66:0/3209184485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c80ccf50 msgr2=0x7f30c8162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 --2- 172.21.15.66:0/3209184485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c80ccf50 0x7f30c8162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 -- 172.21.15.66:0/3209184485 >> v1:172.21.15.117:6789/0 conn(0x7f30c8162090 legacy=0x7f30c80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c6ffd640 1 -- 172.21.15.66:0/3209184485 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070941038 0 0) 0x7f30c8162c00 con 0x7f30c8162090 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.848+0000 7f30c7fff640 1 -- 172.21.15.66:0/3209184485 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c80cee90 con 0x7f30c815e4b0 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.849+0000 7f30c7fff640 1 --2- 172.21.15.66:0/3209184485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c815e4b0 0x7f30c80cf3b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009870 tx=0x7f30bc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac114d9e97ce84b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.849+0000 7f30c6ffd640 1 -- 172.21.15.66:0/3209184485 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc034070 con 0x7f30c815e4b0 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.849+0000 7f30c6ffd640 1 -- 172.21.15.66:0/3209184485 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc030840 con 0x7f30c815e4b0 2024-03-30T13:10:27.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.849+0000 7f30c6ffd640 1 -- 172.21.15.66:0/3209184485 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc030c10 con 0x7f30c815e4b0 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 -- 172.21.15.66:0/3209184485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c815e4b0 msgr2=0x7f30c80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 --2- 172.21.15.66:0/3209184485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c815e4b0 0x7f30c80cf3b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009870 tx=0x7f30bc02b980 comp rx=0 tx=0).stop 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 -- 172.21.15.66:0/3209184485 shutdown_connections 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 --2- 172.21.15.66:0/3209184485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c80ccf50 0x7f30c8162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 --2- 172.21.15.66:0/3209184485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c815e4b0 0x7f30c80cf3b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 -- 172.21.15.66:0/3209184485 >> 172.21.15.66:0/3209184485 conn(0x7f30c80b3d20 msgr2=0x7f30c80b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 -- 172.21.15.66:0/3209184485 shutdown_connections 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.850+0000 7f30ce414640 1 -- 172.21.15.66:0/3209184485 wait complete. 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.851+0000 7f30ce414640 1 Processor -- start 2024-03-30T13:10:27.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.851+0000 7f30ce414640 1 -- start start 2024-03-30T13:10:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30ce414640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 0x7f30c815b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30ce414640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c815e4b0 0x7f30c80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30ce414640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c8162090 0x7f30c80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30ce414640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30c80ce7f0 con 0x7f30c80ccf50 2024-03-30T13:10:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30ce414640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30c815f8a0 con 0x7f30c815e4b0 2024-03-30T13:10:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30ce414640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f30c80ce970 con 0x7f30c8162090 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 0x7f30c815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 0x7f30c815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48236/0 (socket says 172.21.15.66:48236) 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30c7fff640 1 -- 172.21.15.66:0/463155437 learned_addr learned my addr 172.21.15.66:0/463155437 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30cc9b1640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c8162090 0x7f30c80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.852+0000 7f30c77fe640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c815e4b0 0x7f30c80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c7fff640 1 -- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c8162090 msgr2=0x7f30c80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c7fff640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c8162090 0x7f30c80b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c7fff640 1 -- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c815e4b0 msgr2=0x7f30c80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c7fff640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c815e4b0 0x7f30c80b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c7fff640 1 -- 172.21.15.66:0/463155437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c80b6490 con 0x7f30c80ccf50 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c7fff640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 0x7f30c815b700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30bc000c00 tx=0x7f30bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc048070 con 0x7f30c80ccf50 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc030640 con 0x7f30c80ccf50 2024-03-30T13:10:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc034040 con 0x7f30c80ccf50 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c80b6770 con 0x7f30c80ccf50 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.853+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c80b6d30 con 0x7f30c80ccf50 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.854+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c8075890 con 0x7f30c80ccf50 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.855+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f30bc04b070 con 0x7f30c80ccf50 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.855+0000 7f30c4ff9640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30b406c530 0x7f30b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.855+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f30bc0ad910 con 0x7f30c80ccf50 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.855+0000 7f30c77fe640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30b406c530 0x7f30b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:27.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.855+0000 7f30c77fe640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30b406c530 0x7f30b406e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30c80cbca0 tx=0x7f30b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:27.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:27.859+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f30bc030810 con 0x7f30c80ccf50 2024-03-30T13:10:28.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.043+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f30c815dad0 con 0x7f30c80ccf50 2024-03-30T13:10:28.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.043+0000 7f30c4ff9640 1 -- 172.21.15.66:0/463155437 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f30bc07b5e0 con 0x7f30c80ccf50 2024-03-30T13:10:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:28.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30b406c530 msgr2=0x7f30b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30b406c530 0x7f30b406e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30c80cbca0 tx=0x7f30b8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:28.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 msgr2=0x7f30c815b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 0x7f30c815b700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30bc000c00 tx=0x7f30bc004730 comp rx=0 tx=0).stop 2024-03-30T13:10:28.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 shutdown_connections 2024-03-30T13:10:28.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30b406c530 0x7f30b406e9f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30c8162090 0x7f30c80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c815e4b0 0x7f30c80b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 --2- 172.21.15.66:0/463155437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30c80ccf50 0x7f30c815b700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 >> 172.21.15.66:0/463155437 conn(0x7f30c80b3d20 msgr2=0x7f30c80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:28.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 shutdown_connections 2024-03-30T13:10:28.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.045+0000 7f30ce414640 1 -- 172.21.15.66:0/463155437 wait complete. 2024-03-30T13:10:28.060 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:28.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.164+0000 7f702486a640 1 Processor -- start 2024-03-30T13:10:28.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.164+0000 7f702486a640 1 -- start start 2024-03-30T13:10:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f702486a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f70200b7530 0x7f70200d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f702486a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b79c0 0x7f70200d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f702486a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70200b9d20 con 0x7f702015e4f0 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f702486a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f70200d3430 con 0x7f70200b7530 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f702486a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f70200ba000 con 0x7f70200b79c0 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f701e59c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f702015e4f0 0x7f70200cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45392/0 (socket says 172.21.15.66:45392) 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f701e59c640 1 -- 172.21.15.66:0/275088805 learned_addr learned my addr 172.21.15.66:0/275088805 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f701dd9b640 1 --2- 172.21.15.66:0/275088805 >> v2:172.21.15.117:3300/0 conn(0x7f70200b7530 0x7f70200d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.165+0000 7f701ed9d640 1 --2- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b79c0 0x7f70200d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701d59a640 1 -- 172.21.15.66:0/275088805 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837676664 0 0) 0x7f70200b9d20 con 0x7f702015e4f0 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701d59a640 1 -- 172.21.15.66:0/275088805 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f700c003520 con 0x7f702015e4f0 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701dd9b640 1 -- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b79c0 msgr2=0x7f70200d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701dd9b640 1 --2- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b79c0 0x7f70200d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701dd9b640 1 -- 172.21.15.66:0/275088805 >> v1:172.21.15.117:6789/0 conn(0x7f702015e4f0 legacy=0x7f70200cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701dd9b640 1 -- 172.21.15.66:0/275088805 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70200ba2e0 con 0x7f70200b7530 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.166+0000 7f701dd9b640 1 --2- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70200b7530 0x7f70200d29b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f70080097a0 tx=0x7f700802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da043937ae83a24f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f701d59a640 1 -- 172.21.15.66:0/275088805 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70080315a0 con 0x7f70200b7530 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f701d59a640 1 -- 172.21.15.66:0/275088805 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7008037070 con 0x7f70200b7530 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f701d59a640 1 -- 172.21.15.66:0/275088805 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008031960 con 0x7f70200b7530 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 -- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70200b7530 msgr2=0x7f70200d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 --2- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70200b7530 0x7f70200d29b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f70080097a0 tx=0x7f700802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 -- 172.21.15.66:0/275088805 shutdown_connections 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 --2- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b79c0 0x7f70200d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 --2- 172.21.15.66:0/275088805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70200b7530 0x7f70200d29b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 -- 172.21.15.66:0/275088805 >> 172.21.15.66:0/275088805 conn(0x7f70200676e0 msgr2=0x7f7020067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:28.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 -- 172.21.15.66:0/275088805 shutdown_connections 2024-03-30T13:10:28.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.167+0000 7f702486a640 1 -- 172.21.15.66:0/275088805 wait complete. 2024-03-30T13:10:28.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 Processor -- start 2024-03-30T13:10:28.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 -- start start 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b7530 0x7f702015fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 0x7f7020154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f702015e4f0 0x7f70200c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f702015ea50 con 0x7f70200b79c0 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f702015e8d0 con 0x7f702015e4f0 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f702486a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f70200d3d00 con 0x7f70200b7530 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f701dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 0x7f7020154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f701dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 0x7f7020154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48244/0 (socket says 172.21.15.66:48244) 2024-03-30T13:10:28.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f701dd9b640 1 -- 172.21.15.66:0/3437347657 learned_addr learned my addr 172.21.15.66:0/3437347657 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.169+0000 7f701e59c640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b7530 0x7f702015fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701ed9d640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f702015e4f0 0x7f70200c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701dd9b640 1 -- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b7530 msgr2=0x7f702015fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701dd9b640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b7530 0x7f702015fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701dd9b640 1 -- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f702015e4f0 msgr2=0x7f70200c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701dd9b640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f702015e4f0 0x7f70200c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701dd9b640 1 -- 172.21.15.66:0/3437347657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70200c81c0 con 0x7f70200b79c0 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701ed9d640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f702015e4f0 0x7f70200c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f701dd9b640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 0x7f7020154d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7008034040 tx=0x7f70080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008032630 con 0x7f70200b79c0 2024-03-30T13:10:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70080327f0 con 0x7f70200b79c0 2024-03-30T13:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008032be0 con 0x7f70200b79c0 2024-03-30T13:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7020160160 con 0x7f70200b79c0 2024-03-30T13:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.170+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70200c90d0 con 0x7f70200b79c0 2024-03-30T13:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.171+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7020075a10 con 0x7f70200b79c0 2024-03-30T13:10:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.172+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f700804e430 con 0x7f70200b79c0 2024-03-30T13:10:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.172+0000 7f7016ffd640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f700006c560 0x7f700006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.172+0000 7f701e59c640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f700006c560 0x7f700006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.172+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f700802f030 con 0x7f70200b79c0 2024-03-30T13:10:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.173+0000 7f701e59c640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f700006c560 0x7f700006ea20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7018001200 tx=0x7f7018000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.177+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7008040070 con 0x7f70200b79c0 2024-03-30T13:10:28.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.362+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70200d4870 con 0x7f70200b79c0 2024-03-30T13:10:28.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.364+0000 7f7016ffd640 1 -- 172.21.15.66:0/3437347657 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f700807b360 con 0x7f70200b79c0 2024-03-30T13:10:28.367 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:28.368 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f700006c560 msgr2=0x7f700006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f700006c560 0x7f700006ea20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7018001200 tx=0x7f7018000bc0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 msgr2=0x7f7020154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 0x7f7020154d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7008034040 tx=0x7f70080046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 shutdown_connections 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f700006c560 0x7f700006ea20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f702015e4f0 0x7f70200c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70200b79c0 0x7f7020154d80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 --2- 172.21.15.66:0/3437347657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70200b7530 0x7f702015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 >> 172.21.15.66:0/3437347657 conn(0x7f70200676e0 msgr2=0x7f70200cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 shutdown_connections 2024-03-30T13:10:28.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.366+0000 7f702486a640 1 -- 172.21.15.66:0/3437347657 wait complete. 2024-03-30T13:10:28.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:28.381 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.487+0000 7f432c8d9640 1 Processor -- start 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.488+0000 7f432c8d9640 1 -- start start 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.488+0000 7f432c8d9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f43280d13d0 0x7f43280cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.488+0000 7f432c8d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d3ba0 0x7f43280b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.488+0000 7f432c8d9640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43280cf860 con 0x7f43280b3760 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.488+0000 7f432c8d9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f43280cfb40 con 0x7f43280d13d0 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.488+0000 7f432c8d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f43280d33a0 con 0x7f43280d3ba0 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f4325d9b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f43280b3760 0x7f43280d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57312/0 (socket says 172.21.15.66:57312) 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f4325d9b640 1 -- 172.21.15.66:0/760050933 learned_addr learned my addr 172.21.15.66:0/760050933 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:28.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432659c640 1 --2- 172.21.15.66:0/760050933 >> v2:172.21.15.117:3300/0 conn(0x7f43280d13d0 0x7f43280cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f4326d9d640 1 --2- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d3ba0 0x7f43280b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432559a640 1 -- 172.21.15.66:0/760050933 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3954724680 0 0) 0x7f43280cf860 con 0x7f43280b3760 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432559a640 1 -- 172.21.15.66:0/760050933 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4310003520 con 0x7f43280b3760 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432659c640 1 -- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d3ba0 msgr2=0x7f43280b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432659c640 1 --2- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d3ba0 0x7f43280b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432659c640 1 -- 172.21.15.66:0/760050933 >> v1:172.21.15.66:6789/0 conn(0x7f43280b3760 legacy=0x7f43280d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.489+0000 7f432659c640 1 -- 172.21.15.66:0/760050933 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43280d3680 con 0x7f43280d13d0 2024-03-30T13:10:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.490+0000 7f432659c640 1 --2- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d13d0 0x7f43280cecc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f43200097a0 tx=0x7f432002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204600182648f357 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.490+0000 7f432559a640 1 -- 172.21.15.66:0/760050933 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320035020 con 0x7f43280d13d0 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.490+0000 7f432559a640 1 -- 172.21.15.66:0/760050933 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4320031830 con 0x7f43280d13d0 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.490+0000 7f432559a640 1 -- 172.21.15.66:0/760050933 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320031be0 con 0x7f43280d13d0 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.490+0000 7f432c8d9640 1 -- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d13d0 msgr2=0x7f43280cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.490+0000 7f432c8d9640 1 --2- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d13d0 0x7f43280cecc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f43200097a0 tx=0x7f432002b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.491+0000 7f432c8d9640 1 -- 172.21.15.66:0/760050933 shutdown_connections 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.491+0000 7f432c8d9640 1 --2- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d3ba0 0x7f43280b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.491+0000 7f432c8d9640 1 --2- 172.21.15.66:0/760050933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d13d0 0x7f43280cecc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.491+0000 7f432c8d9640 1 -- 172.21.15.66:0/760050933 >> 172.21.15.66:0/760050933 conn(0x7f4328067710 msgr2=0x7f4328067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.491+0000 7f432c8d9640 1 -- 172.21.15.66:0/760050933 shutdown_connections 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.491+0000 7f432c8d9640 1 -- 172.21.15.66:0/760050933 wait complete. 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.492+0000 7f432c8d9640 1 Processor -- start 2024-03-30T13:10:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.492+0000 7f432c8d9640 1 -- start start 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 0x7f43281f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d13d0 0x7f432815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d3ba0 0x7f432815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f432815eab0 con 0x7f43280b3760 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f432815e930 con 0x7f43280d3ba0 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f43280cf4c0 con 0x7f43280d13d0 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 0x7f43281f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 0x7f43281f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48246/0 (socket says 172.21.15.66:48246) 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 -- 172.21.15.66:0/1150248304 learned_addr learned my addr 172.21.15.66:0/1150248304 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f4326d9d640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d3ba0 0x7f432815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f4325d9b640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d13d0 0x7f432815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 -- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d13d0 msgr2=0x7f432815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d13d0 0x7f432815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 -- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d3ba0 msgr2=0x7f432815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d3ba0 0x7f432815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 -- 172.21.15.66:0/1150248304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43280b5120 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432659c640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 0x7f43281f5c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f432002bee0 tx=0x7f43200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320035040 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432003f070 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43280b5400 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.493+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432003bb40 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.494+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43280b59c0 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.494+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43280b98d0 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.495+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4320038020 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.495+0000 7f4316ffd640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f430806c530 0x7f430806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.495+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f43200ad4c0 con 0x7f43280b3760 2024-03-30T13:10:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.495+0000 7f4325d9b640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f430806c530 0x7f430806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.496+0000 7f4325d9b640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f430806c530 0x7f430806e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43280cecf0 tx=0x7f431c001a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.500+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f43280b98d0 con 0x7f43280b3760 2024-03-30T13:10:28.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.689+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4328075890 con 0x7f43280b3760 2024-03-30T13:10:28.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.689+0000 7f4316ffd640 1 -- 172.21.15.66:0/1150248304 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f432007b100 con 0x7f43280b3760 2024-03-30T13:10:28.693 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:28.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.691+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f430806c530 msgr2=0x7f430806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.691+0000 7f432c8d9640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f430806c530 0x7f430806e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43280cecf0 tx=0x7f431c001a90 comp rx=0 tx=0).stop 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.691+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 msgr2=0x7f43281f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.691+0000 7f432c8d9640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 0x7f43281f5c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f432002bee0 tx=0x7f43200046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 shutdown_connections 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f430806c530 0x7f430806e9f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43280d3ba0 0x7f432815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43280d13d0 0x7f432815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 --2- 172.21.15.66:0/1150248304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43280b3760 0x7f43281f5c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 >> 172.21.15.66:0/1150248304 conn(0x7f4328067710 msgr2=0x7f432815d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 shutdown_connections 2024-03-30T13:10:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.692+0000 7f432c8d9640 1 -- 172.21.15.66:0/1150248304 wait complete. 2024-03-30T13:10:28.706 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:28.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.813+0000 7f96684c4640 1 Processor -- start 2024-03-30T13:10:28.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.813+0000 7f96684c4640 1 -- start start 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f96684c4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f96600ba090 0x7f96600d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f96684c4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96600d2ff0 con 0x7f96600b5dc0 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f96684c4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96600d32d0 con 0x7f96600b73d0 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f96684c4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f96600d35b0 con 0x7f96600ba090 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f9665a5f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f96600ba090 0x7f96600d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f9666260640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f96600b5dc0 0x7f96600b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57318/0 (socket says 172.21.15.66:57318) 2024-03-30T13:10:28.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.814+0000 7f9666260640 1 -- 172.21.15.66:0/1271318466 learned_addr learned my addr 172.21.15.66:0/1271318466 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:28.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.815+0000 7f9665a5f640 1 -- 172.21.15.66:0/1271318466 >> v1:172.21.15.117:6789/0 conn(0x7f96600b73d0 legacy=0x7f96600b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.815+0000 7f9665a5f640 1 -- 172.21.15.66:0/1271318466 >> v1:172.21.15.66:6789/0 conn(0x7f96600b5dc0 legacy=0x7f96600b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.815+0000 7f9665a5f640 1 -- 172.21.15.66:0/1271318466 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600d3890 con 0x7f96600ba090 2024-03-30T13:10:28.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.815+0000 7f966525e640 1 -- 172.21.15.66:0/1271318466 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747440159 0 0) 0x7f96600d2ff0 con 0x7f96600b5dc0 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.815+0000 7f9665a5f640 1 --2- 172.21.15.66:0/1271318466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f96600d2680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f96500097d0 tx=0x7f965002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b59e672291e86c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.815+0000 7f966525e640 1 -- 172.21.15.66:0/1271318466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650034070 con 0x7f96600ba090 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f966525e640 1 -- 172.21.15.66:0/1271318466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650030610 con 0x7f96600ba090 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f966525e640 1 -- 172.21.15.66:0/1271318466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500309e0 con 0x7f96600ba090 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 -- 172.21.15.66:0/1271318466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 msgr2=0x7f96600d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 --2- 172.21.15.66:0/1271318466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f96600d2680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f96500097d0 tx=0x7f965002b960 comp rx=0 tx=0).stop 2024-03-30T13:10:28.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 -- 172.21.15.66:0/1271318466 shutdown_connections 2024-03-30T13:10:28.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 --2- 172.21.15.66:0/1271318466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f96600d2680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 -- 172.21.15.66:0/1271318466 >> 172.21.15.66:0/1271318466 conn(0x7f96600678f0 msgr2=0x7f96600d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:28.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 -- 172.21.15.66:0/1271318466 shutdown_connections 2024-03-30T13:10:28.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.816+0000 7f96684c4640 1 -- 172.21.15.66:0/1271318466 wait complete. 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.817+0000 7f96684c4640 1 Processor -- start 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.817+0000 7f96684c4640 1 -- start start 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f96684c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96600b5dc0 0x7f9660155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f96684c4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96600b73d0 0x7f9660160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f96684c4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f9660161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f96684c4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f966015e880 con 0x7f96600ba090 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f96684c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96600d4090 con 0x7f96600b5dc0 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f96684c4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f966015ea00 con 0x7f96600b73d0 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f9666a61640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f9660161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f9666260640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96600b5dc0 0x7f9660155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f9666a61640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f9660161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48254/0 (socket says 172.21.15.66:48254) 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f9665a5f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96600b73d0 0x7f9660160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.818+0000 7f9666a61640 1 -- 172.21.15.66:0/1150945767 learned_addr learned my addr 172.21.15.66:0/1150945767 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666a61640 1 -- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96600b73d0 msgr2=0x7f9660160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666a61640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96600b73d0 0x7f9660160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666a61640 1 -- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96600b5dc0 msgr2=0x7f9660155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666a61640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96600b5dc0 0x7f9660155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666a61640 1 -- 172.21.15.66:0/1150945767 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600cc190 con 0x7f96600ba090 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666260640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96600b5dc0 0x7f9660155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9666a61640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f9660161100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f965800c7d0 tx=0x7f965800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003b70 con 0x7f96600ba090 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9658003d30 con 0x7f96600ba090 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96600cc470 con 0x7f96600ba090 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965800aa90 con 0x7f96600ba090 2024-03-30T13:10:28.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.819+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96600cca30 con 0x7f96600ba090 2024-03-30T13:10:28.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.820+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9660075890 con 0x7f96600ba090 2024-03-30T13:10:28.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.820+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f965800ac30 con 0x7f96600ba090 2024-03-30T13:10:28.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.821+0000 7f9656ffd640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f964406c530 0x7f964406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:28.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.821+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f965808b910 con 0x7f96600ba090 2024-03-30T13:10:28.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.821+0000 7f9666260640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f964406c530 0x7f964406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:28.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.821+0000 7f9666260640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f964406c530 0x7f964406e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f965c007f70 tx=0x7f965c007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:28.825+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f96580594c0 con 0x7f96600ba090 2024-03-30T13:10:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.007+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f966015dfa0 con 0x7f96600ba090 2024-03-30T13:10:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.008+0000 7f9656ffd640 1 -- 172.21.15.66:0/1150945767 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f965805d7c0 con 0x7f96600ba090 2024-03-30T13:10:29.011 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f964406c530 msgr2=0x7f964406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f964406c530 0x7f964406e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f965c007f70 tx=0x7f965c007f00 comp rx=0 tx=0).stop 2024-03-30T13:10:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 msgr2=0x7f9660161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f9660161100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f965800c7d0 tx=0x7f965800cc90 comp rx=0 tx=0).stop 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 shutdown_connections 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f964406c530 0x7f964406e9f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96600ba090 0x7f9660161100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96600b73d0 0x7f9660160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 --2- 172.21.15.66:0/1150945767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96600b5dc0 0x7f9660155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.009+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 >> 172.21.15.66:0/1150945767 conn(0x7f96600678f0 msgr2=0x7f96600b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.010+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 shutdown_connections 2024-03-30T13:10:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.010+0000 7f96684c4640 1 -- 172.21.15.66:0/1150945767 wait complete. 2024-03-30T13:10:29.024 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:29.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 Processor -- start 2024-03-30T13:10:29.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 -- start start 2024-03-30T13:10:29.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 0x7f4f200d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4f200b73d0 0x7f4f200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f200d2ff0 con 0x7f4f200b5dc0 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4f200d32d0 con 0x7f4f200ba090 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.131+0000 7f4f26ee4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f4f200d35b0 con 0x7f4f200b73d0 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f1ffff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 0x7f4f200d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f24c80640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f4f200b5dc0 0x7f4f200b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57330/0 (socket says 172.21.15.66:57330) 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f1ffff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 0x7f4f200d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48256/0 (socket says 172.21.15.66:48256) 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f25481640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4f200b73d0 0x7f4f200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f24c80640 1 -- 172.21.15.66:0/3502313759 learned_addr learned my addr 172.21.15.66:0/3502313759 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f25481640 1 -- 172.21.15.66:0/3502313759 >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 msgr2=0x7f4f200d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f25481640 1 --2- 172.21.15.66:0/3502313759 >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 0x7f4f200d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f25481640 1 -- 172.21.15.66:0/3502313759 >> v1:172.21.15.66:6789/0 conn(0x7f4f200b5dc0 legacy=0x7f4f200b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f25481640 1 -- 172.21.15.66:0/3502313759 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f200d3890 con 0x7f4f200b73d0 2024-03-30T13:10:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.132+0000 7f4f1ffff640 1 --2- 172.21.15.66:0/3502313759 >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 0x7f4f200d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:29.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f25481640 1 --2- 172.21.15.66:0/3502313759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f200b7820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5941100c1806299e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f1f7fe640 1 -- 172.21.15.66:0/3502313759 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08035020 con 0x7f4f200b73d0 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f1f7fe640 1 -- 172.21.15.66:0/3502313759 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f08031830 con 0x7f4f200b73d0 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f1f7fe640 1 -- 172.21.15.66:0/3502313759 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08031c00 con 0x7f4f200b73d0 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 -- 172.21.15.66:0/3502313759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 msgr2=0x7f4f200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/3502313759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f200b7820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 -- 172.21.15.66:0/3502313759 shutdown_connections 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/3502313759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f200b7820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/3502313759 >> v2:172.21.15.66:3300/0 conn(0x7f4f200ba090 0x7f4f200d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 -- 172.21.15.66:0/3502313759 >> 172.21.15.66:0/3502313759 conn(0x7f4f200678f0 msgr2=0x7f4f200d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 -- 172.21.15.66:0/3502313759 shutdown_connections 2024-03-30T13:10:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.133+0000 7f4f26ee4640 1 -- 172.21.15.66:0/3502313759 wait complete. 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.135+0000 7f4f26ee4640 1 Processor -- start 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.135+0000 7f4f26ee4640 1 -- start start 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.135+0000 7f4f26ee4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f200b5dc0 0x7f4f200cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.135+0000 7f4f26ee4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f20160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f26ee4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 0x7f4f20160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f26ee4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2015e880 con 0x7f4f200ba090 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f26ee4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f200d4090 con 0x7f4f200b73d0 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f26ee4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2015ea00 con 0x7f4f200b5dc0 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 0x7f4f20160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 0x7f4f20160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48266/0 (socket says 172.21.15.66:48266) 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 -- 172.21.15.66:0/542133054 learned_addr learned my addr 172.21.15.66:0/542133054 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:29.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f1ffff640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f20160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f24c80640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f200b5dc0 0x7f4f200cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 -- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f200b5dc0 msgr2=0x7f4f200cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f200b5dc0 0x7f4f200cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 -- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 msgr2=0x7f4f20160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f20160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 -- 172.21.15.66:0/542133054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f201619c0 con 0x7f4f200ba090 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f25481640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 0x7f4f20160f40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4f080096f0 tx=0x7f4f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08035040 con 0x7f4f200ba090 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f0803b900 con 0x7f4f200ba090 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2015f6c0 con 0x7f4f200ba090 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0803bcf0 con 0x7f4f200ba090 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.136+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2015fc40 con 0x7f4f200ba090 2024-03-30T13:10:29.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.137+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f20075890 con 0x7f4f200ba090 2024-03-30T13:10:29.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.138+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4f08038020 con 0x7f4f200ba090 2024-03-30T13:10:29.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.138+0000 7f4f1d7fa640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4f0006c530 0x7f4f0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.138+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f4f080ad5f0 con 0x7f4f200ba090 2024-03-30T13:10:29.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.138+0000 7f4f24c80640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4f0006c530 0x7f4f0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.139+0000 7f4f24c80640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4f0006c530 0x7f4f0006e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f180012e0 tx=0x7f4f18001a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.143+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4f08031800 con 0x7f4f200ba090 2024-03-30T13:10:29.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.328+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f200caca0 con 0x7f4f200ba090 2024-03-30T13:10:29.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.329+0000 7f4f1d7fa640 1 -- 172.21.15.66:0/542133054 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f4f0807b1b0 con 0x7f4f200ba090 2024-03-30T13:10:29.332 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:29.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.330+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4f0006c530 msgr2=0x7f4f0006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.330+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4f0006c530 0x7f4f0006e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f180012e0 tx=0x7f4f18001a30 comp rx=0 tx=0).stop 2024-03-30T13:10:29.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.330+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 msgr2=0x7f4f20160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.330+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 0x7f4f20160f40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4f080096f0 tx=0x7f4f080046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 shutdown_connections 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4f0006c530 0x7f4f0006e9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f200ba090 0x7f4f20160f40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f200b73d0 0x7f4f20160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 --2- 172.21.15.66:0/542133054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f200b5dc0 0x7f4f200cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 >> 172.21.15.66:0/542133054 conn(0x7f4f200678f0 msgr2=0x7f4f200b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 shutdown_connections 2024-03-30T13:10:29.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.331+0000 7f4f26ee4640 1 -- 172.21.15.66:0/542133054 wait complete. 2024-03-30T13:10:29.345 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.452+0000 7f3155d3d640 1 Processor -- start 2024-03-30T13:10:29.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.452+0000 7f3155d3d640 1 -- start start 2024-03-30T13:10:29.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f3155d3d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3150072d50 0x7f31500b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f3155d3d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315015fc70 con 0x7f315015f800 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f3155d3d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31500b35e0 con 0x7f3150144cb0 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f3155d3d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f31500b3e80 con 0x7f3150072d50 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f314ffff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f315015f800 0x7f31500ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57338/0 (socket says 172.21.15.66:57338) 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f314ffff640 1 -- 172.21.15.66:0/1235882466 learned_addr learned my addr 172.21.15.66:0/1235882466 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.453+0000 7f314effd640 1 --2- 172.21.15.66:0/1235882466 >> v2:172.21.15.117:3300/0 conn(0x7f3150072d50 0x7f31500b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205652074 0 0) 0x7f315015fc70 con 0x7f315015f800 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3138005730 con 0x7f315015f800 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168682395 0 0) 0x7f31500b35e0 con 0x7f3150144cb0 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f315015fc70 con 0x7f3150144cb0 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314effd640 1 -- 172.21.15.66:0/1235882466 >> v1:172.21.15.117:6789/0 conn(0x7f3150144cb0 legacy=0x7f3150072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314effd640 1 -- 172.21.15.66:0/1235882466 >> v1:172.21.15.66:6789/0 conn(0x7f315015f800 legacy=0x7f31500ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.454+0000 7f314effd640 1 -- 172.21.15.66:0/1235882466 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31500b38c0 con 0x7f3150072d50 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f314effd640 1 --2- 172.21.15.66:0/1235882466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150072d50 0x7f31500b2f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31500b4000 tx=0x7f314402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd970f631a8d6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144034070 con 0x7f3150072d50 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3144030610 con 0x7f3150072d50 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f314e7fc640 1 -- 172.21.15.66:0/1235882466 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440309e0 con 0x7f3150072d50 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 -- 172.21.15.66:0/1235882466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150072d50 msgr2=0x7f31500b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1235882466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150072d50 0x7f31500b2f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31500b4000 tx=0x7f314402b960 comp rx=0 tx=0).stop 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 -- 172.21.15.66:0/1235882466 shutdown_connections 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1235882466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150072d50 0x7f31500b2f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 -- 172.21.15.66:0/1235882466 >> 172.21.15.66:0/1235882466 conn(0x7f31500b5b00 msgr2=0x7f315006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 -- 172.21.15.66:0/1235882466 shutdown_connections 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.455+0000 7f3155d3d640 1 -- 172.21.15.66:0/1235882466 wait complete. 2024-03-30T13:10:29.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.456+0000 7f3155d3d640 1 Processor -- start 2024-03-30T13:10:29.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 -- start start 2024-03-30T13:10:29.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3150072d50 0x7f31501438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150144cb0 0x7f315014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 0x7f315015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31500ce760 con 0x7f315015f800 2024-03-30T13:10:29.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31500cdb30 con 0x7f3150144cb0 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f3155d3d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f31500ce8e0 con 0x7f3150072d50 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f314ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 0x7f315015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f314ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 0x7f315015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48268/0 (socket says 172.21.15.66:48268) 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.457+0000 7f314ffff640 1 -- 172.21.15.66:0/1823829483 learned_addr learned my addr 172.21.15.66:0/1823829483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314f7fe640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3150072d50 0x7f31501438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314effd640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150144cb0 0x7f315014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314ffff640 1 -- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3150072d50 msgr2=0x7f31501438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314ffff640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3150072d50 0x7f31501438c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314ffff640 1 -- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150144cb0 msgr2=0x7f315014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314ffff640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150144cb0 0x7f315014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314ffff640 1 -- 172.21.15.66:0/1823829483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315013fc10 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f314ffff640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 0x7f315015d950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f314800cad0 tx=0x7f314800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148017070 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314800ac50 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3150142de0 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148012400 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.458+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31501433a0 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.459+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3150075890 con 0x7f315015f800 2024-03-30T13:10:29.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.460+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f314801a070 con 0x7f315015f800 2024-03-30T13:10:29.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.460+0000 7f3137fff640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f313c06c530 0x7f313c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.460+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f314808b780 con 0x7f315015f800 2024-03-30T13:10:29.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.460+0000 7f314f7fe640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f313c06c530 0x7f313c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.461+0000 7f314f7fe640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f313c06c530 0x7f313c06e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f31400011e0 tx=0x7f3140009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.464+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31480593c0 con 0x7f315015f800 2024-03-30T13:10:29.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.648+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f31500b4030 con 0x7f315015f800 2024-03-30T13:10:29.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.648+0000 7f3137fff640 1 -- 172.21.15.66:0/1823829483 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f314805d6c0 con 0x7f315015f800 2024-03-30T13:10:29.652 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:29.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f313c06c530 msgr2=0x7f313c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f313c06c530 0x7f313c06e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f31400011e0 tx=0x7f3140009040 comp rx=0 tx=0).stop 2024-03-30T13:10:29.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 msgr2=0x7f315015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 0x7f315015d950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f314800cad0 tx=0x7f314800cf90 comp rx=0 tx=0).stop 2024-03-30T13:10:29.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 shutdown_connections 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f313c06c530 0x7f313c06e9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f315015f800 0x7f315015d950 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3150144cb0 0x7f315014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 --2- 172.21.15.66:0/1823829483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3150072d50 0x7f31501438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 >> 172.21.15.66:0/1823829483 conn(0x7f31500b5b00 msgr2=0x7f315006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 shutdown_connections 2024-03-30T13:10:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.650+0000 7f3155d3d640 1 -- 172.21.15.66:0/1823829483 wait complete. 2024-03-30T13:10:29.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:29.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.768+0000 7fcdbc590640 1 Processor -- start 2024-03-30T13:10:29.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.768+0000 7fcdbc590640 1 -- start start 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbc590640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcdb4160540 0x7fcdb40b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbc590640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb40b5aa0 0x7fcdb40d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbc590640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb40b5f80 con 0x7fcdb4145090 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbc590640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb40b6260 con 0x7fcdb4160540 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbc590640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb40b6540 con 0x7fcdb40b5aa0 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdb9b2b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcdb4160540 0x7fcdb40b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdba32c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fcdb4145090 0x7fcdb415fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45420/0 (socket says 172.21.15.66:45420) 2024-03-30T13:10:29.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbab2d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb40b5aa0 0x7fcdb40d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdba32c640 1 -- 172.21.15.66:0/1437815781 learned_addr learned my addr 172.21.15.66:0/1437815781 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdb932a640 1 -- 172.21.15.66:0/1437815781 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435983386 0 0) 0x7fcdb40b5f80 con 0x7fcdb4145090 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbab2d640 1 -- 172.21.15.66:0/1437815781 >> v2:172.21.15.66:3300/0 conn(0x7fcdb4160540 msgr2=0x7fcdb40b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbab2d640 1 --2- 172.21.15.66:0/1437815781 >> v2:172.21.15.66:3300/0 conn(0x7fcdb4160540 0x7fcdb40b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.769+0000 7fcdbab2d640 1 -- 172.21.15.66:0/1437815781 >> v1:172.21.15.117:6789/0 conn(0x7fcdb4145090 legacy=0x7fcdb415fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.770+0000 7fcdbab2d640 1 -- 172.21.15.66:0/1437815781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb40b6820 con 0x7fcdb40b5aa0 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.770+0000 7fcdb9b2b640 1 --2- 172.21.15.66:0/1437815781 >> v2:172.21.15.66:3300/0 conn(0x7fcdb4160540 0x7fcdb40b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.770+0000 7fcdbab2d640 1 --2- 172.21.15.66:0/1437815781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb40b5aa0 0x7fcdb40d11c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009d90 tx=0x7fcda402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11a2bd095ff1e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.770+0000 7fcdb932a640 1 -- 172.21.15.66:0/1437815781 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4039070 con 0x7fcdb40b5aa0 2024-03-30T13:10:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.770+0000 7fcdb932a640 1 -- 172.21.15.66:0/1437815781 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda402cbd0 con 0x7fcdb40b5aa0 2024-03-30T13:10:29.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.770+0000 7fcdb932a640 1 -- 172.21.15.66:0/1437815781 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4034440 con 0x7fcdb40b5aa0 2024-03-30T13:10:29.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 -- 172.21.15.66:0/1437815781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb40b5aa0 msgr2=0x7fcdb40d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 --2- 172.21.15.66:0/1437815781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb40b5aa0 0x7fcdb40d11c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009d90 tx=0x7fcda402be20 comp rx=0 tx=0).stop 2024-03-30T13:10:29.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 -- 172.21.15.66:0/1437815781 shutdown_connections 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 --2- 172.21.15.66:0/1437815781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb40b5aa0 0x7fcdb40d11c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 --2- 172.21.15.66:0/1437815781 >> v2:172.21.15.66:3300/0 conn(0x7fcdb4160540 0x7fcdb40b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 -- 172.21.15.66:0/1437815781 >> 172.21.15.66:0/1437815781 conn(0x7fcdb406f8b0 msgr2=0x7fcdb406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 -- 172.21.15.66:0/1437815781 shutdown_connections 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.771+0000 7fcdbc590640 1 -- 172.21.15.66:0/1437815781 wait complete. 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.772+0000 7fcdbc590640 1 Processor -- start 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.772+0000 7fcdbc590640 1 -- start start 2024-03-30T13:10:29.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbc590640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdb40b5aa0 0x7fcdb40aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbc590640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb4145090 0x7fcdb4140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbc590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 0x7fcdb4140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbc590640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4145470 con 0x7fcdb4160540 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbc590640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4146000 con 0x7fcdb40b5aa0 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbc590640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb41455f0 con 0x7fcdb4145090 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbab2d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 0x7fcdb4140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdba32c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdb40b5aa0 0x7fcdb40aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbab2d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 0x7fcdb4140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48286/0 (socket says 172.21.15.66:48286) 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbab2d640 1 -- 172.21.15.66:0/2465203416 learned_addr learned my addr 172.21.15.66:0/2465203416 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdb9b2b640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb4145090 0x7fcdb4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbab2d640 1 -- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb4145090 msgr2=0x7fcdb4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.773+0000 7fcdbab2d640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb4145090 0x7fcdb4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdbab2d640 1 -- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdb40b5aa0 msgr2=0x7fcdb40aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdbab2d640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdb40b5aa0 0x7fcdb40aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdbab2d640 1 -- 172.21.15.66:0/2465203416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb415d890 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdba32c640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdb40b5aa0 0x7fcdb40aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdbab2d640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 0x7fcdb4140e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009d90 tx=0x7fcda4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4044020 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda403a040 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb415db70 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4039050 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.774+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb415e130 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.775+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb4075a10 con 0x7fcdb4160540 2024-03-30T13:10:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.775+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcda4003bb0 con 0x7fcdb4160540 2024-03-30T13:10:29.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.776+0000 7fcdb2ffd640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcda006c560 0x7fcda006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:29.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.776+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fcda40adee0 con 0x7fcdb4160540 2024-03-30T13:10:29.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.776+0000 7fcdba32c640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcda006c560 0x7fcda006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:29.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.776+0000 7fcdba32c640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcda006c560 0x7fcda006ea20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000f60 tx=0x7fcdac007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:29.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.780+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcda404e060 con 0x7fcdb4160540 2024-03-30T13:10:29.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.967+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdb406fd30 con 0x7fcdb4160540 2024-03-30T13:10:29.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.968+0000 7fcdb2ffd640 1 -- 172.21.15.66:0/2465203416 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7fcda407bb20 con 0x7fcdb4160540 2024-03-30T13:10:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:29.972 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcda006c560 msgr2=0x7fcda006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcda006c560 0x7fcda006ea20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000f60 tx=0x7fcdac007d50 comp rx=0 tx=0).stop 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 msgr2=0x7fcdb4140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 0x7fcdb4140e90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009d90 tx=0x7fcda4004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 shutdown_connections 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcda006c560 0x7fcda006ea20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdb4160540 0x7fcdb4140e90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcdb4145090 0x7fcdb4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 --2- 172.21.15.66:0/2465203416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdb40b5aa0 0x7fcdb40aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:29.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 >> 172.21.15.66:0/2465203416 conn(0x7fcdb406f8b0 msgr2=0x7fcdb40b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:29.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 shutdown_connections 2024-03-30T13:10:29.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:29.970+0000 7fcdbc590640 1 -- 172.21.15.66:0/2465203416 wait complete. 2024-03-30T13:10:29.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:29.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:30.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.092+0000 7ff4d2b9d640 1 Processor -- start 2024-03-30T13:10:30.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.092+0000 7ff4d2b9d640 1 -- start start 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d2b9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4cc0b4920 0x7ff4cc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d2b9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff4cc0d0b90 0x7ff4cc0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d2b9d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc0b54b0 con 0x7ff4cc0b4d00 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d2b9d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc0d0260 con 0x7ff4cc0d0b90 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d2b9d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc0cfca0 con 0x7ff4cc0b4920 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d239c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff4cc0d0b90 0x7ff4cc0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d239c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff4cc0d0b90 0x7ff4cc0d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48300/0 (socket says 172.21.15.66:48300) 2024-03-30T13:10:30.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d139a640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff4cc0b4d00 0x7ff4cc0d4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57350/0 (socket says 172.21.15.66:57350) 2024-03-30T13:10:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d239c640 1 -- 172.21.15.66:0/1131045596 learned_addr learned my addr 172.21.15.66:0/1131045596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.093+0000 7ff4d1b9b640 1 --2- 172.21.15.66:0/1131045596 >> v2:172.21.15.117:3300/0 conn(0x7ff4cc0b4920 0x7ff4cc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.094+0000 7ff4d239c640 1 -- 172.21.15.66:0/1131045596 >> v2:172.21.15.117:3300/0 conn(0x7ff4cc0b4920 msgr2=0x7ff4cc0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.094+0000 7ff4d239c640 1 --2- 172.21.15.66:0/1131045596 >> v2:172.21.15.117:3300/0 conn(0x7ff4cc0b4920 0x7ff4cc0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.094+0000 7ff4d239c640 1 -- 172.21.15.66:0/1131045596 >> v1:172.21.15.66:6789/0 conn(0x7ff4cc0b4d00 legacy=0x7ff4cc0d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.094+0000 7ff4d239c640 1 -- 172.21.15.66:0/1131045596 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc0cff80 con 0x7ff4cc0d0b90 2024-03-30T13:10:30.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.094+0000 7ff4d239c640 1 --2- 172.21.15.66:0/1131045596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0d1040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09dc10 tx=0x7ff4bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b98c3bfe6559c86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.094+0000 7ff4d0b99640 1 -- 172.21.15.66:0/1131045596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc02ca80 con 0x7ff4cc0d0b90 2024-03-30T13:10:30.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d0b99640 1 -- 172.21.15.66:0/1131045596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc02cc20 con 0x7ff4cc0d0b90 2024-03-30T13:10:30.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d0b99640 1 -- 172.21.15.66:0/1131045596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0343f0 con 0x7ff4cc0d0b90 2024-03-30T13:10:30.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/1131045596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0d0b90 msgr2=0x7ff4cc0d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/1131045596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0d1040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc09dc10 tx=0x7ff4bc02bee0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/1131045596 shutdown_connections 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/1131045596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0d1040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/1131045596 >> v2:172.21.15.117:3300/0 conn(0x7ff4cc0b4920 0x7ff4cc0b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/1131045596 >> 172.21.15.66:0/1131045596 conn(0x7ff4cc09d4e0 msgr2=0x7ff4cc0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/1131045596 shutdown_connections 2024-03-30T13:10:30.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.095+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/1131045596 wait complete. 2024-03-30T13:10:30.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.096+0000 7ff4d2b9d640 1 Processor -- start 2024-03-30T13:10:30.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.096+0000 7ff4d2b9d640 1 -- start start 2024-03-30T13:10:30.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d2b9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 0x7ff4cc1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d2b9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4cc0b4d00 0x7ff4cc09df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d2b9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d2b9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b3ff0 con 0x7ff4cc0b4920 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d2b9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0d2180 con 0x7ff4cc0d0b90 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d2b9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc0cf8c0 con 0x7ff4cc0b4d00 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d1b9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 0x7ff4cc1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d1b9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 0x7ff4cc1525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48306/0 (socket says 172.21.15.66:48306) 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d1b9b640 1 -- 172.21.15.66:0/343563452 learned_addr learned my addr 172.21.15.66:0/343563452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d139a640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4cc0b4d00 0x7ff4cc09df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.097+0000 7ff4d239c640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d1b9b640 1 -- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4cc0b4d00 msgr2=0x7ff4cc09df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d1b9b640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4cc0b4d00 0x7ff4cc09df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d1b9b640 1 -- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4cc0d0b90 msgr2=0x7ff4cc0b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d1b9b640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d1b9b640 1 -- 172.21.15.66:0/343563452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc0a1060 con 0x7ff4cc0b4920 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d239c640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d1b9b640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 0x7ff4cc1525b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e9b0 tx=0x7ff4c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4017070 con 0x7ff4cc0b4920 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c400c9a0 con 0x7ff4cc0b4920 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc0a1340 con 0x7ff4cc0b4920 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400cd90 con 0x7ff4cc0b4920 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.098+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc0a1900 con 0x7ff4cc0b4920 2024-03-30T13:10:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.099+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc075890 con 0x7ff4cc0b4920 2024-03-30T13:10:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.099+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff4c401a070 con 0x7ff4cc0b4920 2024-03-30T13:10:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.100+0000 7ff4ca7fc640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff4b806c560 0x7ff4b806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.100+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7ff4c408bd40 con 0x7ff4cc0b4920 2024-03-30T13:10:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.100+0000 7ff4d139a640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff4b806c560 0x7ff4b806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.100+0000 7ff4d139a640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff4b806c560 0x7ff4b806ea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc15f300 tx=0x7ff4c0007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.104+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff4c4059980 con 0x7ff4cc0b4920 2024-03-30T13:10:30.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.291+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4cc0a1d20 con 0x7ff4cc0b4920 2024-03-30T13:10:30.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.291+0000 7ff4ca7fc640 1 -- 172.21.15.66:0/343563452 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7ff4c405dc80 con 0x7ff4cc0b4920 2024-03-30T13:10:30.295 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff4b806c560 msgr2=0x7ff4b806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff4b806c560 0x7ff4b806ea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc15f300 tx=0x7ff4c0007e80 comp rx=0 tx=0).stop 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 msgr2=0x7ff4cc1525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 0x7ff4cc1525b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e9b0 tx=0x7ff4c400ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 shutdown_connections 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff4b806c560 0x7ff4b806ea20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4cc0d0b90 0x7ff4cc0b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4cc0b4d00 0x7ff4cc09df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 --2- 172.21.15.66:0/343563452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4cc0b4920 0x7ff4cc1525b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 >> 172.21.15.66:0/343563452 conn(0x7ff4cc09d4e0 msgr2=0x7ff4cc0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:30.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 shutdown_connections 2024-03-30T13:10:30.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.293+0000 7ff4d2b9d640 1 -- 172.21.15.66:0/343563452 wait complete. 2024-03-30T13:10:30.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:30.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.414+0000 7fa88bec8640 1 Processor -- start 2024-03-30T13:10:30.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.414+0000 7fa88bec8640 1 -- start start 2024-03-30T13:10:30.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88bec8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa88415e840 0x7fa8840b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88bec8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840d10b0 0x7fa8840b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88bec8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8840cfba0 con 0x7fa8840cdf50 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88bec8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa8840cfe60 con 0x7fa88415e840 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88bec8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa8840d0140 con 0x7fa8840d10b0 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa889c64640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa88415e840 0x7fa8840b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88a465640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa8840cdf50 0x7fa8840b0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57354/0 (socket says 172.21.15.66:57354) 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa88a465640 1 -- 172.21.15.66:0/2399353406 learned_addr learned my addr 172.21.15.66:0/2399353406 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa889463640 1 --2- 172.21.15.66:0/2399353406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840d10b0 0x7fa8840b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa889c64640 1 -- 172.21.15.66:0/2399353406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840d10b0 msgr2=0x7fa8840b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa889c64640 1 --2- 172.21.15.66:0/2399353406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840d10b0 0x7fa8840b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa889c64640 1 -- 172.21.15.66:0/2399353406 >> v1:172.21.15.66:6789/0 conn(0x7fa8840cdf50 legacy=0x7fa8840b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa889c64640 1 -- 172.21.15.66:0/2399353406 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8840af960 con 0x7fa88415e840 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.415+0000 7fa888c62640 1 -- 172.21.15.66:0/2399353406 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3293113977 0 0) 0x7fa8840cfba0 con 0x7fa8840cdf50 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.416+0000 7fa889c64640 1 --2- 172.21.15.66:0/2399353406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8840b0160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8800097a0 tx=0x7fa88002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366b96f2495874b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.416+0000 7fa888c62640 1 -- 172.21.15.66:0/2399353406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8800315a0 con 0x7fa88415e840 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.416+0000 7fa888c62640 1 -- 172.21.15.66:0/2399353406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa880037070 con 0x7fa88415e840 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.416+0000 7fa888c62640 1 -- 172.21.15.66:0/2399353406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880031960 con 0x7fa88415e840 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 -- 172.21.15.66:0/2399353406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 msgr2=0x7fa8840b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 --2- 172.21.15.66:0/2399353406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8840b0160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8800097a0 tx=0x7fa88002b840 comp rx=0 tx=0).stop 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 -- 172.21.15.66:0/2399353406 shutdown_connections 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 --2- 172.21.15.66:0/2399353406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840d10b0 0x7fa8840b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 --2- 172.21.15.66:0/2399353406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8840b0160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 -- 172.21.15.66:0/2399353406 >> 172.21.15.66:0/2399353406 conn(0x7fa8840d0380 msgr2=0x7fa8840d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:30.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 -- 172.21.15.66:0/2399353406 shutdown_connections 2024-03-30T13:10:30.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.417+0000 7fa88bec8640 1 -- 172.21.15.66:0/2399353406 wait complete. 2024-03-30T13:10:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.418+0000 7fa88bec8640 1 Processor -- start 2024-03-30T13:10:30.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.418+0000 7fa88bec8640 1 -- start start 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88bec8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840cdf50 0x7fa884161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88bec8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8840d10b0 0x7fa884159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88bec8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8841f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88bec8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa884068e90 con 0x7fa88415e840 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88bec8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa88415ec20 con 0x7fa8840d10b0 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88bec8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa884069010 con 0x7fa8840cdf50 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8841f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8841f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48334/0 (socket says 172.21.15.66:48334) 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 -- 172.21.15.66:0/4056961107 learned_addr learned my addr 172.21.15.66:0/4056961107 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa889c64640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840cdf50 0x7fa884161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa889463640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8840d10b0 0x7fa884159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 -- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840cdf50 msgr2=0x7fa884161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840cdf50 0x7fa884161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 -- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8840d10b0 msgr2=0x7fa884159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8840d10b0 0x7fa884159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 -- 172.21.15.66:0/4056961107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88415ffa0 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.419+0000 7fa88a465640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8841f44e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00c7a0 tx=0x7fa87c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.420+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c018070 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.420+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa87c00a640 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.420+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88415d2d0 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.420+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00aab0 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.420+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88415d890 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.421+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8840b1c60 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.421+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa87c01b070 con 0x7fa88415e840 2024-03-30T13:10:30.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.422+0000 7fa87a7fc640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa86806c530 0x7fa86806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.422+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fa87c08b050 con 0x7fa88415e840 2024-03-30T13:10:30.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.422+0000 7fa889c64640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa86806c530 0x7fa86806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.422+0000 7fa889c64640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa86806c530 0x7fa86806e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa880036000 tx=0x7fa88003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.426+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa8840b1c60 con 0x7fa88415e840 2024-03-30T13:10:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.608+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa884160720 con 0x7fa88415e840 2024-03-30T13:10:30.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.609+0000 7fa87a7fc640 1 -- 172.21.15.66:0/4056961107 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fa87c058c00 con 0x7fa88415e840 2024-03-30T13:10:30.613 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa86806c530 msgr2=0x7fa86806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa86806c530 0x7fa86806e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa880036000 tx=0x7fa88003d040 comp rx=0 tx=0).stop 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 msgr2=0x7fa8841f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8841f44e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00c7a0 tx=0x7fa87c00cc60 comp rx=0 tx=0).stop 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 shutdown_connections 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa86806c530 0x7fa86806e9f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa88415e840 0x7fa8841f44e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8840d10b0 0x7fa884159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 --2- 172.21.15.66:0/4056961107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8840cdf50 0x7fa884161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 >> 172.21.15.66:0/4056961107 conn(0x7fa8840d0380 msgr2=0x7fa88415bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 shutdown_connections 2024-03-30T13:10:30.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.611+0000 7fa88bec8640 1 -- 172.21.15.66:0/4056961107 wait complete. 2024-03-30T13:10:30.625 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:30.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.732+0000 7f2e48d00640 1 Processor -- start 2024-03-30T13:10:30.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.732+0000 7f2e48d00640 1 -- start start 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e48d00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e44071020 0x7f2e4406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e48d00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4409b370 con 0x7f2e4409aa30 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e48d00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e44162f40 con 0x7f2e4409aee0 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e48d00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2e440d0770 con 0x7f2e44071020 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e4259c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e44071020 0x7f2e4406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e3bfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2e4409aa30 0x7f2e44162120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57360/0 (socket says 172.21.15.66:57360) 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e3bfff640 1 -- 172.21.15.66:0/1468163593 learned_addr learned my addr 172.21.15.66:0/1468163593 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e4259c640 1 -- 172.21.15.66:0/1468163593 >> v1:172.21.15.117:6789/0 conn(0x7f2e4409aee0 legacy=0x7f2e44162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e4259c640 1 -- 172.21.15.66:0/1468163593 >> v1:172.21.15.66:6789/0 conn(0x7f2e4409aa30 legacy=0x7f2e44162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e41d9b640 1 -- 172.21.15.66:0/1468163593 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115080231 0 0) 0x7f2e44162f40 con 0x7f2e4409aee0 2024-03-30T13:10:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.733+0000 7f2e4259c640 1 -- 172.21.15.66:0/1468163593 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e440d1c10 con 0x7f2e44071020 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.734+0000 7f2e4259c640 1 --2- 172.21.15.66:0/1468163593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e44071020 0x7f2e4406c840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097a0 tx=0x7f2e3402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2758ac3cbfc56aa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.734+0000 7f2e41d9b640 1 -- 172.21.15.66:0/1468163593 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34034070 con 0x7f2e44071020 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.734+0000 7f2e41d9b640 1 -- 172.21.15.66:0/1468163593 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34030610 con 0x7f2e44071020 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.734+0000 7f2e41d9b640 1 -- 172.21.15.66:0/1468163593 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340309e0 con 0x7f2e44071020 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.734+0000 7f2e48d00640 1 -- 172.21.15.66:0/1468163593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e44071020 msgr2=0x7f2e4406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.735+0000 7f2e48d00640 1 --2- 172.21.15.66:0/1468163593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e44071020 0x7f2e4406c840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2e340097a0 tx=0x7f2e3402b960 comp rx=0 tx=0).stop 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.735+0000 7f2e48d00640 1 -- 172.21.15.66:0/1468163593 shutdown_connections 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.735+0000 7f2e48d00640 1 --2- 172.21.15.66:0/1468163593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e44071020 0x7f2e4406c840 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.735+0000 7f2e48d00640 1 -- 172.21.15.66:0/1468163593 >> 172.21.15.66:0/1468163593 conn(0x7f2e440b3ce0 msgr2=0x7f2e440b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.735+0000 7f2e48d00640 1 -- 172.21.15.66:0/1468163593 shutdown_connections 2024-03-30T13:10:30.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.735+0000 7f2e48d00640 1 -- 172.21.15.66:0/1468163593 wait complete. 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.736+0000 7f2e48d00640 1 Processor -- start 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.736+0000 7f2e48d00640 1 -- start start 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e48d00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e44071020 0x7f2e440b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e48d00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4409aa30 0x7f2e44156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e48d00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 0x7f2e4415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e48d00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4415f740 con 0x7f2e4409aee0 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e48d00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4415f460 con 0x7f2e44071020 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e48d00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4415fa20 con 0x7f2e4409aa30 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e42d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 0x7f2e4415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e42d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 0x7f2e4415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48362/0 (socket says 172.21.15.66:48362) 2024-03-30T13:10:30.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e42d9d640 1 -- 172.21.15.66:0/2459530467 learned_addr learned my addr 172.21.15.66:0/2459530467 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e44071020 0x7f2e440b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e3bfff640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4409aa30 0x7f2e44156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 -- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4409aa30 msgr2=0x7f2e44156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4409aa30 0x7f2e44156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 -- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 msgr2=0x7f2e4415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 0x7f2e4415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 -- 172.21.15.66:0/2459530467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4415d460 con 0x7f2e44071020 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e42d9d640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 0x7f2e4415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.737+0000 7f2e4259c640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e44071020 0x7f2e440b5630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e34033000 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.738+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34034040 con 0x7f2e44071020 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.738+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3403acc0 con 0x7f2e44071020 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.738+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e440b5d80 con 0x7f2e44071020 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.738+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34044630 con 0x7f2e44071020 2024-03-30T13:10:30.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.738+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e440b6340 con 0x7f2e44071020 2024-03-30T13:10:30.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.739+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44075890 con 0x7f2e44071020 2024-03-30T13:10:30.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.739+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e3404b060 con 0x7f2e44071020 2024-03-30T13:10:30.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.739+0000 7f2e3affd640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2406c530 0x7f2e2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.739+0000 7f2e3bfff640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2406c530 0x7f2e2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.739+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f2e340ad840 con 0x7f2e44071020 2024-03-30T13:10:30.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.740+0000 7f2e3bfff640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2406c530 0x7f2e2406e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e441f7830 tx=0x7f2e2c007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:30.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.744+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e4415d460 con 0x7f2e44071020 2024-03-30T13:10:30.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.929+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2e440b20d0 con 0x7f2e44071020 2024-03-30T13:10:30.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.929+0000 7f2e3affd640 1 -- 172.21.15.66:0/2459530467 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f2e3403f070 con 0x7f2e44071020 2024-03-30T13:10:30.934 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2406c530 msgr2=0x7f2e2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2406c530 0x7f2e2406e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e441f7830 tx=0x7f2e2c007e80 comp rx=0 tx=0).stop 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e44071020 msgr2=0x7f2e440b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e44071020 0x7f2e440b5630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e34033000 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 shutdown_connections 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2406c530 0x7f2e2406e9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409aee0 0x7f2e4415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4409aa30 0x7f2e44156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 --2- 172.21.15.66:0/2459530467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e44071020 0x7f2e440b5630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 >> 172.21.15.66:0/2459530467 conn(0x7f2e440b3ce0 msgr2=0x7f2e4406d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:30.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 shutdown_connections 2024-03-30T13:10:30.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:30.932+0000 7f2e48d00640 1 -- 172.21.15.66:0/2459530467 wait complete. 2024-03-30T13:10:30.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:31.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.053+0000 7fe09b9f4640 1 Processor -- start 2024-03-30T13:10:31.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.053+0000 7fe09b9f4640 1 -- start start 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe09b9f4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09415e5b0 0x7fe094162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe09b9f4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe094073310 0x7fe09406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe09b9f4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0941630b0 con 0x7fe0940d1be0 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe09b9f4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe094073880 con 0x7fe094073310 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe09b9f4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe0940d0e20 con 0x7fe09415e5b0 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe098f8f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe094073310 0x7fe09406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe099790640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09415e5b0 0x7fe094162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe099f91640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe0940d1be0 0x7fe0940b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45432/0 (socket says 172.21.15.66:45432) 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe098f8f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe094073310 0x7fe09406b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:35728/0 (socket says 172.21.15.66:35728) 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe099f91640 1 -- 172.21.15.66:0/4160704074 learned_addr learned my addr 172.21.15.66:0/4160704074 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:31.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe098f8f640 1 -- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09415e5b0 msgr2=0x7fe094162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe098f8f640 1 --2- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09415e5b0 0x7fe094162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe098f8f640 1 -- 172.21.15.66:0/4160704074 >> v1:172.21.15.117:6789/0 conn(0x7fe0940d1be0 legacy=0x7fe0940b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe098f8f640 1 -- 172.21.15.66:0/4160704074 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940d1100 con 0x7fe094073310 2024-03-30T13:10:31.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.054+0000 7fe08bfff640 1 -- 172.21.15.66:0/4160704074 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1564833584 0 0) 0x7fe0941630b0 con 0x7fe0940d1be0 2024-03-30T13:10:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.055+0000 7fe098f8f640 1 --2- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe09406b580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097a0 tx=0x7fe08402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753009c874e62de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.055+0000 7fe08bfff640 1 -- 172.21.15.66:0/4160704074 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840315a0 con 0x7fe094073310 2024-03-30T13:10:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.055+0000 7fe08bfff640 1 -- 172.21.15.66:0/4160704074 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe084037070 con 0x7fe094073310 2024-03-30T13:10:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.055+0000 7fe08bfff640 1 -- 172.21.15.66:0/4160704074 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084031960 con 0x7fe094073310 2024-03-30T13:10:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 -- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 msgr2=0x7fe09406b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe09406b580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097a0 tx=0x7fe08402b840 comp rx=0 tx=0).stop 2024-03-30T13:10:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 -- 172.21.15.66:0/4160704074 shutdown_connections 2024-03-30T13:10:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe09406b580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/4160704074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe09415e5b0 0x7fe094162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 -- 172.21.15.66:0/4160704074 >> 172.21.15.66:0/4160704074 conn(0x7fe09406e720 msgr2=0x7fe09406eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 -- 172.21.15.66:0/4160704074 shutdown_connections 2024-03-30T13:10:31.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.056+0000 7fe09b9f4640 1 -- 172.21.15.66:0/4160704074 wait complete. 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.057+0000 7fe09b9f4640 1 Processor -- start 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.057+0000 7fe09b9f4640 1 -- start start 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe09b9f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe0940d0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe09b9f4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0940d1be0 0x7fe094156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe09b9f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 0x7fe0940ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe09b9f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe09415e990 con 0x7fe09415e5b0 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe09b9f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940d16c0 con 0x7fe094073310 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe09b9f4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe09406bac0 con 0x7fe0940d1be0 2024-03-30T13:10:31.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 0x7fe0940ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 0x7fe0940ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48378/0 (socket says 172.21.15.66:48378) 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 -- 172.21.15.66:0/174587906 learned_addr learned my addr 172.21.15.66:0/174587906 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099790640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe0940d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe098f8f640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0940d1be0 0x7fe094156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 -- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0940d1be0 msgr2=0x7fe094156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0940d1be0 0x7fe094156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 -- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 msgr2=0x7fe0940d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe0940d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.058+0000 7fe099f91640 1 -- 172.21.15.66:0/174587906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940ce600 con 0x7fe09415e5b0 2024-03-30T13:10:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.059+0000 7fe099f91640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 0x7fe0940ae4f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00c7a0 tx=0x7fe08c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.059+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c018070 con 0x7fe09415e5b0 2024-03-30T13:10:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.059+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08c00a640 con 0x7fe09415e5b0 2024-03-30T13:10:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.059+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00aab0 con 0x7fe09415e5b0 2024-03-30T13:10:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.059+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0940cda80 con 0x7fe09415e5b0 2024-03-30T13:10:31.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.059+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0940ce040 con 0x7fe09415e5b0 2024-03-30T13:10:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.060+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe094075890 con 0x7fe09415e5b0 2024-03-30T13:10:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.060+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe08c01b070 con 0x7fe09415e5b0 2024-03-30T13:10:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.061+0000 7fe089ffb640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe07806c530 0x7fe07806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.061+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fe08c08aff0 con 0x7fe09415e5b0 2024-03-30T13:10:31.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.061+0000 7fe099790640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe07806c530 0x7fe07806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.061+0000 7fe099790640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe07806c530 0x7fe07806e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe090009aa0 tx=0x7fe090009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.065+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe08c058c30 con 0x7fe09415e5b0 2024-03-30T13:10:31.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.251+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe094072be0 con 0x7fe09415e5b0 2024-03-30T13:10:31.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.253+0000 7fe089ffb640 1 -- 172.21.15.66:0/174587906 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fe08c05cf30 con 0x7fe09415e5b0 2024-03-30T13:10:31.256 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe07806c530 msgr2=0x7fe07806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe07806c530 0x7fe07806e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe090009aa0 tx=0x7fe090009190 comp rx=0 tx=0).stop 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 msgr2=0x7fe0940ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 0x7fe0940ae4f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00c7a0 tx=0x7fe08c00cc60 comp rx=0 tx=0).stop 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 shutdown_connections 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe07806c530 0x7fe07806e9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09415e5b0 0x7fe0940ae4f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0940d1be0 0x7fe094156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 --2- 172.21.15.66:0/174587906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe094073310 0x7fe0940d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 >> 172.21.15.66:0/174587906 conn(0x7fe09406e720 msgr2=0x7fe094157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 shutdown_connections 2024-03-30T13:10:31.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.255+0000 7fe09b9f4640 1 -- 172.21.15.66:0/174587906 wait complete. 2024-03-30T13:10:31.277 INFO:tasks.cephfs_test_runner:Starting test: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:10:31.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:31.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.382+0000 7f3fd1c2a640 1 Processor -- start 2024-03-30T13:10:31.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.382+0000 7f3fd1c2a640 1 -- start start 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fd1c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc162af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fd1c2a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3fcc0735e0 0x7f3fcc071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fd1c2a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fcc163030 con 0x7f3fcc0cf140 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fd1c2a640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3fcc1480c0 con 0x7f3fcc0735e0 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fd1c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc0c9760 con 0x7f3fcc0ced60 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fcbfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3fcc0735e0 0x7f3fcc071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fcaffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3fcc0cf140 0x7f3fcc072ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57366/0 (socket says 172.21.15.66:57366) 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fcbfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3fcc0735e0 0x7f3fcc071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48388/0 (socket says 172.21.15.66:48388) 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fcaffd640 1 -- 172.21.15.66:0/1559804992 learned_addr learned my addr 172.21.15.66:0/1559804992 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fcb7fe640 1 --2- 172.21.15.66:0/1559804992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc162af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.383+0000 7f3fcbfff640 1 -- 172.21.15.66:0/1559804992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 msgr2=0x7f3fcc162af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fcbfff640 1 --2- 172.21.15.66:0/1559804992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc162af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fcbfff640 1 -- 172.21.15.66:0/1559804992 >> v1:172.21.15.66:6789/0 conn(0x7f3fcc0cf140 legacy=0x7f3fcc072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fcbfff640 1 -- 172.21.15.66:0/1559804992 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc0c9a40 con 0x7f3fcc0735e0 2024-03-30T13:10:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fca7fc640 1 -- 172.21.15.66:0/1559804992 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652872779 0 0) 0x7f3fc0001110 con 0x7f3fcc0cf140 2024-03-30T13:10:31.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fcbfff640 1 --2- 172.21.15.66:0/1559804992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc071740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009d90 tx=0x7f3fc402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177a3ff1580f6a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fca7fc640 1 -- 172.21.15.66:0/1559804992 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4039070 con 0x7f3fcc0735e0 2024-03-30T13:10:31.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fca7fc640 1 -- 172.21.15.66:0/1559804992 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc402cc50 con 0x7f3fcc0735e0 2024-03-30T13:10:31.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.384+0000 7f3fca7fc640 1 -- 172.21.15.66:0/1559804992 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4034440 con 0x7f3fcc0735e0 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/1559804992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 msgr2=0x7f3fcc071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/1559804992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc071740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009d90 tx=0x7f3fc402bea0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/1559804992 shutdown_connections 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/1559804992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc071740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/1559804992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc162af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/1559804992 >> 172.21.15.66:0/1559804992 conn(0x7f3fcc0b1c90 msgr2=0x7f3fcc0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/1559804992 shutdown_connections 2024-03-30T13:10:31.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.385+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/1559804992 wait complete. 2024-03-30T13:10:31.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.386+0000 7f3fd1c2a640 1 Processor -- start 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.386+0000 7f3fd1c2a640 1 -- start start 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fd1c2a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fd1c2a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fd1c2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc0cf140 0x7f3fcc0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fd1c2a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc0ca2e0 con 0x7f3fcc0735e0 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fd1c2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc0ca000 con 0x7f3fcc0cf140 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fd1c2a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc0c9310 con 0x7f3fcc0ced60 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc13fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48392/0 (socket says 172.21.15.66:48392) 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fcb7fe640 1 -- 172.21.15.66:0/65220332 learned_addr learned my addr 172.21.15.66:0/65220332 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fcbfff640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc0cf140 0x7f3fcc0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.387+0000 7f3fcaffd640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fcb7fe640 1 -- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 msgr2=0x7f3fcc15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fcb7fe640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc15dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fcb7fe640 1 -- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc0cf140 msgr2=0x7f3fcc0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fcb7fe640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc0cf140 0x7f3fcc0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fcb7fe640 1 -- 172.21.15.66:0/65220332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc140060 con 0x7f3fcc0735e0 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fcb7fe640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc13fb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00e9b0 tx=0x7f3fbc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc017070 con 0x7f3fcc0735e0 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00c9a0 con 0x7f3fcc0735e0 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00cd90 con 0x7f3fcc0735e0 2024-03-30T13:10:31.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc160100 con 0x7f3fcc0735e0 2024-03-30T13:10:31.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.388+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc160680 con 0x7f3fcc0735e0 2024-03-30T13:10:31.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.389+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc075a10 con 0x7f3fcc0735e0 2024-03-30T13:10:31.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.389+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3fbc01a070 con 0x7f3fcc0735e0 2024-03-30T13:10:31.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.390+0000 7f3fb3fff640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3fb806c530 0x7f3fb806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.390+0000 7f3fcaffd640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3fb806c530 0x7f3fb806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.390+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f3fbc08b9d0 con 0x7f3fcc0735e0 2024-03-30T13:10:31.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.390+0000 7f3fcaffd640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3fb806c530 0x7f3fb806e9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc145a40 tx=0x7f3fc0007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.394+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3fbc059610 con 0x7f3fcc0735e0 2024-03-30T13:10:31.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.579+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fcc146e50 con 0x7f3fcc0735e0 2024-03-30T13:10:31.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.580+0000 7f3fb3fff640 1 -- 172.21.15.66:0/65220332 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f3fbc05d910 con 0x7f3fcc0735e0 2024-03-30T13:10:31.584 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:31.584 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:31.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3fb806c530 msgr2=0x7f3fb806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3fb806c530 0x7f3fb806e9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc145a40 tx=0x7f3fc0007de0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 msgr2=0x7f3fcc13fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc13fb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00e9b0 tx=0x7f3fbc00ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 shutdown_connections 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3fb806c530 0x7f3fb806e9f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fcc0cf140 0x7f3fcc0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fcc0ced60 0x7f3fcc15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 --2- 172.21.15.66:0/65220332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fcc0735e0 0x7f3fcc13fb20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.582+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 >> 172.21.15.66:0/65220332 conn(0x7f3fcc0b1c90 msgr2=0x7f3fcc161e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.583+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 shutdown_connections 2024-03-30T13:10:31.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.583+0000 7f3fd1c2a640 1 -- 172.21.15.66:0/65220332 wait complete. 2024-03-30T13:10:31.597 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:31.598 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:31.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.705+0000 7fdcd0d60640 1 Processor -- start 2024-03-30T13:10:31.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.705+0000 7fdcd0d60640 1 -- start start 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcd0d60640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdccc15fa60 0x7fdccc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcd0d60640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fe40 0x7fdccc0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcd0d60640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdccc0d0510 con 0x7fdccc0d0d10 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcd0d60640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fdccc0d1180 con 0x7fdccc15fa60 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcd0d60640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc0cdef0 con 0x7fdccc15fe40 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdccad9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdccc0d0d10 0x7fdccc09aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:57380/0 (socket says 172.21.15.66:57380) 2024-03-30T13:10:31.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdccad9d640 1 -- 172.21.15.66:0/4008598445 learned_addr learned my addr 172.21.15.66:0/4008598445 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcca59c640 1 --2- 172.21.15.66:0/4008598445 >> v2:172.21.15.117:3300/0 conn(0x7fdccc15fa60 0x7fdccc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcc1d9b640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fe40 0x7fdccc0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcc9d9b640 1 -- 172.21.15.66:0/4008598445 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3290964493 0 0) 0x7fdccc0d0510 con 0x7fdccc0d0d10 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcc9d9b640 1 -- 172.21.15.66:0/4008598445 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcb8003520 con 0x7fdccc0d0d10 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcca59c640 1 -- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fe40 msgr2=0x7fdccc0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcca59c640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fe40 0x7fdccc0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcca59c640 1 -- 172.21.15.66:0/4008598445 >> v1:172.21.15.66:6789/0 conn(0x7fdccc0d0d10 legacy=0x7fdccc09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcca59c640 1 -- 172.21.15.66:0/4008598445 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc0b3590 con 0x7fdccc15fa60 2024-03-30T13:10:31.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.706+0000 7fdcc1d9b640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fe40 0x7fdccc0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:31.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.707+0000 7fdcca59c640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc15fa60 0x7fdccc0cffd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0097a0 tx=0x7fdcbc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49912c190d88dc49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.707+0000 7fdcc9d9b640 1 -- 172.21.15.66:0/4008598445 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc035020 con 0x7fdccc15fa60 2024-03-30T13:10:31.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.707+0000 7fdcc9d9b640 1 -- 172.21.15.66:0/4008598445 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc031830 con 0x7fdccc15fa60 2024-03-30T13:10:31.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.707+0000 7fdcc9d9b640 1 -- 172.21.15.66:0/4008598445 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031be0 con 0x7fdccc15fa60 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 -- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc15fa60 msgr2=0x7fdccc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc15fa60 0x7fdccc0cffd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0097a0 tx=0x7fdcbc02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 -- 172.21.15.66:0/4008598445 shutdown_connections 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fe40 0x7fdccc0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/4008598445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc15fa60 0x7fdccc0cffd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 -- 172.21.15.66:0/4008598445 >> 172.21.15.66:0/4008598445 conn(0x7fdccc0b3ce0 msgr2=0x7fdccc0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 -- 172.21.15.66:0/4008598445 shutdown_connections 2024-03-30T13:10:31.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.708+0000 7fdcd0d60640 1 -- 172.21.15.66:0/4008598445 wait complete. 2024-03-30T13:10:31.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.709+0000 7fdcd0d60640 1 Processor -- start 2024-03-30T13:10:31.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.709+0000 7fdcd0d60640 1 -- start start 2024-03-30T13:10:31.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcd0d60640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc0d0d10 0x7fdccc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcd0d60640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fa60 0x7fdccc0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcd0d60640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 0x7fdccc152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcd0d60640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0cdb80 con 0x7fdccc15fe40 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcd0d60640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0cd8a0 con 0x7fdccc0d0d10 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcd0d60640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc0b4a00 con 0x7fdccc15fa60 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdccad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 0x7fdccc152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdccad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 0x7fdccc152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48394/0 (socket says 172.21.15.66:48394) 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdccad9d640 1 -- 172.21.15.66:0/1800612381 learned_addr learned my addr 172.21.15.66:0/1800612381 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcca59c640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc0d0d10 0x7fdccc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdcc1d9b640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fa60 0x7fdccc0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdccad9d640 1 -- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fa60 msgr2=0x7fdccc0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.710+0000 7fdccad9d640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fa60 0x7fdccc0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdccad9d640 1 -- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc0d0d10 msgr2=0x7fdccc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdccad9d640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc0d0d10 0x7fdccc06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdccad9d640 1 -- 172.21.15.66:0/1800612381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc0cae50 con 0x7fdccc15fe40 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdcca59c640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc0d0d10 0x7fdccc06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:31.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdccad9d640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 0x7fdccc152620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400caa0 tx=0x7fdcc400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4017070 con 0x7fdccc15fe40 2024-03-30T13:10:31.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc400ac00 con 0x7fdccc15fe40 2024-03-30T13:10:31.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc0cb130 con 0x7fdccc15fe40 2024-03-30T13:10:31.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4012400 con 0x7fdccc15fe40 2024-03-30T13:10:31.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.711+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc0cb6f0 con 0x7fdccc15fe40 2024-03-30T13:10:31.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.712+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdccc075890 con 0x7fdccc15fe40 2024-03-30T13:10:31.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.712+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdcc401a070 con 0x7fdccc15fe40 2024-03-30T13:10:31.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.713+0000 7fdcc37fe640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdcac06c560 0x7fdcac06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.713+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fdcc408b780 con 0x7fdccc15fe40 2024-03-30T13:10:31.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.713+0000 7fdcca59c640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdcac06c560 0x7fdcac06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.713+0000 7fdcca59c640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdcac06c560 0x7fdcac06ea20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc036000 tx=0x7fdcbc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.717+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdcc40593c0 con 0x7fdccc15fe40 2024-03-30T13:10:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.902+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdccc0cea50 con 0x7fdccc15fe40 2024-03-30T13:10:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.902+0000 7fdcc37fe640 1 -- 172.21.15.66:0/1800612381 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fdcc405d6c0 con 0x7fdccc15fe40 2024-03-30T13:10:31.906 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdcac06c560 msgr2=0x7fdcac06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdcac06c560 0x7fdcac06ea20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc036000 tx=0x7fdcbc03c040 comp rx=0 tx=0).stop 2024-03-30T13:10:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 msgr2=0x7fdccc152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 0x7fdccc152620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400caa0 tx=0x7fdcc400cf60 comp rx=0 tx=0).stop 2024-03-30T13:10:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 shutdown_connections 2024-03-30T13:10:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdcac06c560 0x7fdcac06ea20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdccc15fe40 0x7fdccc152620 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdccc15fa60 0x7fdccc0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 --2- 172.21.15.66:0/1800612381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdccc0d0d10 0x7fdccc06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 >> 172.21.15.66:0/1800612381 conn(0x7fdccc0b3ce0 msgr2=0x7fdccc06f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 shutdown_connections 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:31.904+0000 7fdcd0d60640 1 -- 172.21.15.66:0/1800612381 wait complete. 2024-03-30T13:10:31.919 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:32.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.025+0000 7ff0c6b04640 1 Processor -- start 2024-03-30T13:10:32.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.025+0000 7ff0c6b04640 1 -- start start 2024-03-30T13:10:32.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0c6b04640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c00d4a70 0x7ff0c015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0c6b04640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c00b3c30 con 0x7ff0c00d27a0 2024-03-30T13:10:32.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0c6b04640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c0162110 con 0x7ff0c015c000 2024-03-30T13:10:32.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0c6b04640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c00d2c00 con 0x7ff0c00d4a70 2024-03-30T13:10:32.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0bffff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff0c00d27a0 0x7ff0c00d4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50302/0 (socket says 172.21.15.66:50302) 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0bffff640 1 -- 172.21.15.66:0/3260348557 learned_addr learned my addr 172.21.15.66:0/3260348557 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.026+0000 7ff0c50a1640 1 --2- 172.21.15.66:0/3260348557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c00d4a70 0x7ff0c015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726418018 0 0) 0x7ff0c00b3c30 con 0x7ff0c00d27a0 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0ac003520 con 0x7ff0c00d27a0 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707635149 0 0) 0x7ff0c0162110 con 0x7ff0c015c000 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c00b3c30 con 0x7ff0c015c000 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0c50a1640 1 -- 172.21.15.66:0/3260348557 >> v1:172.21.15.117:6789/0 conn(0x7ff0c015c000 legacy=0x7ff0c00b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0c50a1640 1 -- 172.21.15.66:0/3260348557 >> v1:172.21.15.66:6789/0 conn(0x7ff0c00d27a0 legacy=0x7ff0c00d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0c50a1640 1 -- 172.21.15.66:0/3260348557 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c01623d0 con 0x7ff0c00d4a70 2024-03-30T13:10:32.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.027+0000 7ff0c50a1640 1 --2- 172.21.15.66:0/3260348557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c00d4a70 0x7ff0c015edd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80097d0 tx=0x7ff0a802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a016345fa33ab381 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8035020 con 0x7ff0c00d4a70 2024-03-30T13:10:32.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a8031800 con 0x7ff0c00d4a70 2024-03-30T13:10:32.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0bf7fe640 1 -- 172.21.15.66:0/3260348557 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8031bd0 con 0x7ff0c00d4a70 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 -- 172.21.15.66:0/3260348557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c00d4a70 msgr2=0x7ff0c015edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/3260348557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c00d4a70 0x7ff0c015edd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80097d0 tx=0x7ff0a802b8a0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 -- 172.21.15.66:0/3260348557 shutdown_connections 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/3260348557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c00d4a70 0x7ff0c015edd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 -- 172.21.15.66:0/3260348557 >> 172.21.15.66:0/3260348557 conn(0x7ff0c00676e0 msgr2=0x7ff0c0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 -- 172.21.15.66:0/3260348557 shutdown_connections 2024-03-30T13:10:32.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.028+0000 7ff0c6b04640 1 -- 172.21.15.66:0/3260348557 wait complete. 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.029+0000 7ff0c6b04640 1 Processor -- start 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.029+0000 7ff0c6b04640 1 -- start start 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0c6b04640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0c00d27a0 0x7ff0c00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0c6b04640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 0x7ff0c00d3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0c6b04640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c015c000 0x7ff0c014cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0c6b04640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0162c70 con 0x7ff0c00d4a70 2024-03-30T13:10:32.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0c6b04640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0162990 con 0x7ff0c00d27a0 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0c6b04640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c015c3e0 con 0x7ff0c015c000 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 0x7ff0c00d3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 0x7ff0c00d3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42744/0 (socket says 172.21.15.66:42744) 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.030+0000 7ff0bffff640 1 -- 172.21.15.66:0/647728633 learned_addr learned my addr 172.21.15.66:0/647728633 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0c48a0640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0c00d27a0 0x7ff0c00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0c50a1640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c015c000 0x7ff0c014cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bffff640 1 -- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c015c000 msgr2=0x7ff0c014cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bffff640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c015c000 0x7ff0c014cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bffff640 1 -- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0c00d27a0 msgr2=0x7ff0c00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bffff640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0c00d27a0 0x7ff0c00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bffff640 1 -- 172.21.15.66:0/647728633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c00c83e0 con 0x7ff0c00d4a70 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bffff640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 0x7ff0c00d3890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000cbc0 tx=0x7ff0b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0017070 con 0x7ff0c00d4a70 2024-03-30T13:10:32.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b000aca0 con 0x7ff0c00d4a70 2024-03-30T13:10:32.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c00c86c0 con 0x7ff0c00d4a70 2024-03-30T13:10:32.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0012600 con 0x7ff0c00d4a70 2024-03-30T13:10:32.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.031+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c00c8c80 con 0x7ff0c00d4a70 2024-03-30T13:10:32.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.032+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c0075890 con 0x7ff0c00d4a70 2024-03-30T13:10:32.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.033+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff0b001a070 con 0x7ff0c00d4a70 2024-03-30T13:10:32.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.033+0000 7ff0bd7fa640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff0a006c560 0x7ff0a006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.033+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7ff0b008b870 con 0x7ff0c00d4a70 2024-03-30T13:10:32.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.033+0000 7ff0c48a0640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff0a006c560 0x7ff0a006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.033+0000 7ff0c48a0640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff0a006c560 0x7ff0a006ea20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8007fd0 tx=0x7ff0b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.037+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff0b0059370 con 0x7ff0c00d4a70 2024-03-30T13:10:32.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.221+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff0c00b9110 con 0x7ff0c00d4a70 2024-03-30T13:10:32.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.221+0000 7ff0bd7fa640 1 -- 172.21.15.66:0/647728633 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7ff0b005d670 con 0x7ff0c00d4a70 2024-03-30T13:10:32.225 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff0a006c560 msgr2=0x7ff0a006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff0a006c560 0x7ff0a006ea20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8007fd0 tx=0x7ff0b8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 msgr2=0x7ff0c00d3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 0x7ff0c00d3890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000cbc0 tx=0x7ff0b000a060 comp rx=0 tx=0).stop 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 shutdown_connections 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff0a006c560 0x7ff0a006ea20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0c015c000 0x7ff0c014cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0c00d4a70 0x7ff0c00d3890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 --2- 172.21.15.66:0/647728633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0c00d27a0 0x7ff0c00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 >> 172.21.15.66:0/647728633 conn(0x7ff0c00676e0 msgr2=0x7ff0c015df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 shutdown_connections 2024-03-30T13:10:32.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.223+0000 7ff0c6b04640 1 -- 172.21.15.66:0/647728633 wait complete. 2024-03-30T13:10:32.238 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:32.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.346+0000 7fad09037640 1 Processor -- start 2024-03-30T13:10:32.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.346+0000 7fad09037640 1 -- start start 2024-03-30T13:10:32.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad09037640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415eba0 0x7fad0406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad09037640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fad040b4800 0x7fad040cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad09037640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040d1170 con 0x7fad0415e7c0 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad09037640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fad040cc840 con 0x7fad040b4800 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad09037640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fad040d1450 con 0x7fad0415eba0 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fad040b4800 0x7fad040cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fad040b4800 0x7fad040cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42752/0 (socket says 172.21.15.66:42752) 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad02d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fad0415e7c0 0x7fad0406c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50310/0 (socket says 172.21.15.66:50310) 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 -- 172.21.15.66:0/1348220212 learned_addr learned my addr 172.21.15.66:0/1348220212 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7facfa59c640 1 --2- 172.21.15.66:0/1348220212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415eba0 0x7fad0406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 -- 172.21.15.66:0/1348220212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415eba0 msgr2=0x7fad0406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 --2- 172.21.15.66:0/1348220212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415eba0 0x7fad0406cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 -- 172.21.15.66:0/1348220212 >> v1:172.21.15.66:6789/0 conn(0x7fad0415e7c0 legacy=0x7fad0406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0359e640 1 -- 172.21.15.66:0/1348220212 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040d1730 con 0x7fad040b4800 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.347+0000 7fad0259c640 1 -- 172.21.15.66:0/1348220212 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219219574 0 0) 0x7fad040d1170 con 0x7fad0415e7c0 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad0359e640 1 --2- 172.21.15.66:0/1348220212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad040b4800 0x7fad040cccd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facfc009e50 tx=0x7facfc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67a3d33ccaa1566 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad0259c640 1 -- 172.21.15.66:0/1348220212 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc02cb00 con 0x7fad040b4800 2024-03-30T13:10:32.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad0259c640 1 -- 172.21.15.66:0/1348220212 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc02cca0 con 0x7fad040b4800 2024-03-30T13:10:32.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad0259c640 1 -- 172.21.15.66:0/1348220212 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc034610 con 0x7fad040b4800 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 -- 172.21.15.66:0/1348220212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad040b4800 msgr2=0x7fad040cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 --2- 172.21.15.66:0/1348220212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad040b4800 0x7fad040cccd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facfc009e50 tx=0x7facfc02bf60 comp rx=0 tx=0).stop 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 -- 172.21.15.66:0/1348220212 shutdown_connections 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 --2- 172.21.15.66:0/1348220212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad040b4800 0x7fad040cccd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 --2- 172.21.15.66:0/1348220212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415eba0 0x7fad0406cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 -- 172.21.15.66:0/1348220212 >> 172.21.15.66:0/1348220212 conn(0x7fad040b3ce0 msgr2=0x7fad040b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 -- 172.21.15.66:0/1348220212 shutdown_connections 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.348+0000 7fad09037640 1 -- 172.21.15.66:0/1348220212 wait complete. 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.350+0000 7fad09037640 1 Processor -- start 2024-03-30T13:10:32.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.350+0000 7fad09037640 1 -- start start 2024-03-30T13:10:32.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad040b4800 0x7fad040cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415e7c0 0x7fad04151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 0x7fad04151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad0406d630 con 0x7fad0415eba0 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad0406d4b0 con 0x7fad040b4800 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fad040cd210 con 0x7fad0415e7c0 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 0x7fad04151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 0x7fad04151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42766/0 (socket says 172.21.15.66:42766) 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 -- 172.21.15.66:0/358426530 learned_addr learned my addr 172.21.15.66:0/358426530 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad02d9d640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad040b4800 0x7fad040cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7facfa59c640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415e7c0 0x7fad04151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 -- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415e7c0 msgr2=0x7fad04151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415e7c0 0x7fad04151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 -- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad040b4800 msgr2=0x7fad040cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad040b4800 0x7fad040cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 -- 172.21.15.66:0/358426530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040cbbe0 con 0x7fad0415eba0 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad02d9d640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad040b4800 0x7fad040cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:32.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad0359e640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 0x7fad04151e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7facfc036040 tx=0x7facfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc043020 con 0x7fad0415eba0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc02ce50 con 0x7fad0415eba0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.351+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad04160720 con 0x7fad0415eba0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.352+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc009510 con 0x7fad0415eba0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.352+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad04160ce0 con 0x7fad0415eba0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.353+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad04075890 con 0x7fad0415eba0 2024-03-30T13:10:32.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.353+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7facfc046020 con 0x7fad0415eba0 2024-03-30T13:10:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.353+0000 7facfbfff640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7face406c530 0x7face406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.353+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7facfc0adf80 con 0x7fad0415eba0 2024-03-30T13:10:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.353+0000 7fad02d9d640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7face406c530 0x7face406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.354+0000 7fad02d9d640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7face406c530 0x7face406e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7facf4000f60 tx=0x7facf4007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.358+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7facfc04d030 con 0x7fad0415eba0 2024-03-30T13:10:32.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.541+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad040cc600 con 0x7fad0415eba0 2024-03-30T13:10:32.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.542+0000 7facfbfff640 1 -- 172.21.15.66:0/358426530 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fad040cbbe0 con 0x7fad0415eba0 2024-03-30T13:10:32.546 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.543+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7face406c530 msgr2=0x7face406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7face406c530 0x7face406e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7facf4000f60 tx=0x7facf4007d50 comp rx=0 tx=0).stop 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 msgr2=0x7fad04151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 0x7fad04151e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7facfc036040 tx=0x7facfc004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 shutdown_connections 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7face406c530 0x7face406e9f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad0415eba0 0x7fad04151e30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad0415e7c0 0x7fad04151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 --2- 172.21.15.66:0/358426530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad040b4800 0x7fad040cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 >> 172.21.15.66:0/358426530 conn(0x7fad040b3ce0 msgr2=0x7fad04153600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 shutdown_connections 2024-03-30T13:10:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.544+0000 7fad09037640 1 -- 172.21.15.66:0/358426530 wait complete. 2024-03-30T13:10:32.558 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.665+0000 7f91ac71b640 1 Processor -- start 2024-03-30T13:10:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.665+0000 7f91ac71b640 1 -- start start 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91ac71b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f91a4071020 0x7f91a406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91ac71b640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a409b370 con 0x7f91a409aa30 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91ac71b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a4162f40 con 0x7f91a409aee0 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91ac71b640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f91a40d0770 con 0x7f91a4071020 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91aa4b7640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f91a4071020 0x7f91a406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91a9cb6640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f91a409aa30 0x7f91a4162120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50316/0 (socket says 172.21.15.66:50316) 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91a9cb6640 1 -- 172.21.15.66:0/2449510208 learned_addr learned my addr 172.21.15.66:0/2449510208 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91aa4b7640 1 -- 172.21.15.66:0/2449510208 >> v1:172.21.15.117:6789/0 conn(0x7f91a409aee0 legacy=0x7f91a4162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91aa4b7640 1 -- 172.21.15.66:0/2449510208 >> v1:172.21.15.66:6789/0 conn(0x7f91a409aa30 legacy=0x7f91a4162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.666+0000 7f91aa4b7640 1 -- 172.21.15.66:0/2449510208 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a40d1c10 con 0x7f91a4071020 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.667+0000 7f91aa4b7640 1 --2- 172.21.15.66:0/2449510208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a4071020 0x7f91a406c840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f919c0097a0 tx=0x7f919c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b9e0d467c9caea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.667+0000 7f91a94b5640 1 -- 172.21.15.66:0/2449510208 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c034070 con 0x7f91a4071020 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.667+0000 7f91a94b5640 1 -- 172.21.15.66:0/2449510208 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c030610 con 0x7f91a4071020 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.667+0000 7f91a94b5640 1 -- 172.21.15.66:0/2449510208 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c0309e0 con 0x7f91a4071020 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 -- 172.21.15.66:0/2449510208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a4071020 msgr2=0x7f91a406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 --2- 172.21.15.66:0/2449510208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a4071020 0x7f91a406c840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f919c0097a0 tx=0x7f919c02b960 comp rx=0 tx=0).stop 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 -- 172.21.15.66:0/2449510208 shutdown_connections 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 --2- 172.21.15.66:0/2449510208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a4071020 0x7f91a406c840 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 -- 172.21.15.66:0/2449510208 >> 172.21.15.66:0/2449510208 conn(0x7f91a40b3ce0 msgr2=0x7f91a40b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 -- 172.21.15.66:0/2449510208 shutdown_connections 2024-03-30T13:10:32.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.668+0000 7f91ac71b640 1 -- 172.21.15.66:0/2449510208 wait complete. 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.669+0000 7f91ac71b640 1 Processor -- start 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.669+0000 7f91ac71b640 1 -- start start 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91ac71b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a4071020 0x7f91a41532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91ac71b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 0x7f91a41f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91ac71b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a409aee0 0x7f91a4157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91ac71b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415f740 con 0x7f91a409aa30 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91ac71b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415f460 con 0x7f91a409aee0 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91ac71b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f91a415fa20 con 0x7f91a4071020 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 0x7f91a41f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 0x7f91a41f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42780/0 (socket says 172.21.15.66:42780) 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 -- 172.21.15.66:0/863784337 learned_addr learned my addr 172.21.15.66:0/863784337 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:32.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91aa4b7640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a4071020 0x7f91a41532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91aacb8640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a409aee0 0x7f91a4157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 -- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a4071020 msgr2=0x7f91a41532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a4071020 0x7f91a41532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 -- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a409aee0 msgr2=0x7f91a4157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a409aee0 0x7f91a4157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 -- 172.21.15.66:0/863784337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41f8230 con 0x7f91a409aa30 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f91a9cb6640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 0x7f91a41f7ba0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f919400c480 tx=0x7f919400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.670+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194018070 con 0x7f91a409aa30 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.671+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9194005ce0 con 0x7f91a409aa30 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.671+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41f8510 con 0x7f91a409aa30 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.671+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91940072d0 con 0x7f91a409aa30 2024-03-30T13:10:32.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.671+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a415fd60 con 0x7f91a409aa30 2024-03-30T13:10:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.672+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4075890 con 0x7f91a409aa30 2024-03-30T13:10:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.672+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f919401b070 con 0x7f91a409aa30 2024-03-30T13:10:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.672+0000 7f919affd640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918806c530 0x7f918806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.672+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f919408adc0 con 0x7f91a409aa30 2024-03-30T13:10:32.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.672+0000 7f91aa4b7640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918806c530 0x7f918806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.673+0000 7f91aa4b7640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918806c530 0x7f918806e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f919c036000 tx=0x7f919c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.677+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9194058980 con 0x7f91a409aa30 2024-03-30T13:10:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.862+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f91a40703c0 con 0x7f91a409aa30 2024-03-30T13:10:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.862+0000 7f919affd640 1 -- 172.21.15.66:0/863784337 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f919405cc80 con 0x7f91a409aa30 2024-03-30T13:10:32.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918806c530 msgr2=0x7f918806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918806c530 0x7f918806e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f919c036000 tx=0x7f919c03b040 comp rx=0 tx=0).stop 2024-03-30T13:10:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 msgr2=0x7f91a41f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 0x7f91a41f7ba0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f919400c480 tx=0x7f919400c940 comp rx=0 tx=0).stop 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 shutdown_connections 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918806c530 0x7f918806e9f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a409aee0 0x7f91a4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91a409aa30 0x7f91a41f7ba0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.864+0000 7f91ac71b640 1 --2- 172.21.15.66:0/863784337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a4071020 0x7f91a41532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.865+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 >> 172.21.15.66:0/863784337 conn(0x7f91a40b3ce0 msgr2=0x7f91a406c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.865+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 shutdown_connections 2024-03-30T13:10:32.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.865+0000 7f91ac71b640 1 -- 172.21.15.66:0/863784337 wait complete. 2024-03-30T13:10:32.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-30T13:10:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.982+0000 7f2777daa640 1 Processor -- start 2024-03-30T13:10:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.983+0000 7f2777daa640 1 -- start start 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.983+0000 7f2777daa640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f277015e950 0x7f27700d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.983+0000 7f2777daa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f27700d30d0 0x7f27700b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.983+0000 7f2777daa640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27700d36f0 con 0x7f27700d48c0 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.983+0000 7f2777daa640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f27700d3cb0 con 0x7f277015e950 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.983+0000 7f2777daa640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f27700d4d00 con 0x7f27700d30d0 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775b46640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f277015e950 0x7f27700d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775345640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f27700d48c0 0x7f27700d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50326/0 (socket says 172.21.15.66:50326) 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775345640 1 -- 172.21.15.66:0/299894924 learned_addr learned my addr 172.21.15.66:0/299894924 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2776347640 1 --2- 172.21.15.66:0/299894924 >> v2:172.21.15.117:3300/0 conn(0x7f27700d30d0 0x7f27700b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775b46640 1 -- 172.21.15.66:0/299894924 >> v2:172.21.15.117:3300/0 conn(0x7f27700d30d0 msgr2=0x7f27700b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775b46640 1 --2- 172.21.15.66:0/299894924 >> v2:172.21.15.117:3300/0 conn(0x7f27700d30d0 0x7f27700b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775b46640 1 -- 172.21.15.66:0/299894924 >> v1:172.21.15.66:6789/0 conn(0x7f27700d48c0 legacy=0x7f27700d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2775b46640 1 -- 172.21.15.66:0/299894924 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27700d39d0 con 0x7f277015e950 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.984+0000 7f2774b44640 1 -- 172.21.15.66:0/299894924 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194492732 0 0) 0x7f27700d36f0 con 0x7f27700d48c0 2024-03-30T13:10:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2775b46640 1 --2- 172.21.15.66:0/299894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f277015e950 0x7f27700d4380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27700d3e30 tx=0x7f276802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17929b90d5fc84bf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2774b44640 1 -- 172.21.15.66:0/299894924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768034070 con 0x7f277015e950 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2774b44640 1 -- 172.21.15.66:0/299894924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2768030610 con 0x7f277015e950 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2774b44640 1 -- 172.21.15.66:0/299894924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27680309e0 con 0x7f277015e950 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 -- 172.21.15.66:0/299894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f277015e950 msgr2=0x7f27700d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 --2- 172.21.15.66:0/299894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f277015e950 0x7f27700d4380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27700d3e30 tx=0x7f276802b960 comp rx=0 tx=0).stop 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 -- 172.21.15.66:0/299894924 shutdown_connections 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 --2- 172.21.15.66:0/299894924 >> v2:172.21.15.117:3300/0 conn(0x7f27700d30d0 0x7f27700b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 --2- 172.21.15.66:0/299894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f277015e950 0x7f27700d4380 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 -- 172.21.15.66:0/299894924 >> 172.21.15.66:0/299894924 conn(0x7f27700676e0 msgr2=0x7f2770067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 -- 172.21.15.66:0/299894924 shutdown_connections 2024-03-30T13:10:32.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.985+0000 7f2777daa640 1 -- 172.21.15.66:0/299894924 wait complete. 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 Processor -- start 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 -- start start 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 0x7f277014c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27700d48c0 0x7f27700c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277015e950 0x7f27700cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f277015ed30 con 0x7f27700d30d0 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f27700b7da0 con 0x7f27700d48c0 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2777daa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f277015eeb0 con 0x7f277015e950 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2775b46640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 0x7f277014c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2775b46640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 0x7f277014c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42800/0 (socket says 172.21.15.66:42800) 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.987+0000 7f2775b46640 1 -- 172.21.15.66:0/1971539632 learned_addr learned my addr 172.21.15.66:0/1971539632 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2776347640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277015e950 0x7f27700cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775345640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27700d48c0 0x7f27700c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775b46640 1 -- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277015e950 msgr2=0x7f27700cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775b46640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277015e950 0x7f27700cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775b46640 1 -- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27700d48c0 msgr2=0x7f27700c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775b46640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27700d48c0 0x7f27700c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775b46640 1 -- 172.21.15.66:0/1971539632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27701f3ed0 con 0x7f27700d30d0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2775b46640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 0x7f277014c3e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2768009860 tx=0x7f27680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768034040 con 0x7f27700d30d0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276803aaa0 con 0x7f27700d30d0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27700caf90 con 0x7f27700d30d0 2024-03-30T13:10:32.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27680443f0 con 0x7f27700d30d0 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.988+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27700ca6f0 con 0x7f27700d30d0 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.989+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2770075a10 con 0x7f27700d30d0 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.989+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2768037070 con 0x7f27700d30d0 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.990+0000 7f276e7fc640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f275c06c560 0x7f275c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.990+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f27680ad810 con 0x7f27700d30d0 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.990+0000 7f2775345640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f275c06c560 0x7f275c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.990+0000 7f2775345640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f275c06c560 0x7f275c06ea20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f277015d850 tx=0x7f2764007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:32.994+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f276803f070 con 0x7f27700d30d0 2024-03-30T13:10:33.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.251+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f277006b580 con 0x7f27700d30d0 2024-03-30T13:10:33.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.257+0000 7f276e7fc640 1 -- 172.21.15.66:0/1971539632 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v66) ==== 150+0+0 (secure 0 0 0) 0x7f276807b3c0 con 0x7f27700d30d0 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f275c06c560 msgr2=0x7f275c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f275c06c560 0x7f275c06ea20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f277015d850 tx=0x7f2764007d50 comp rx=0 tx=0).stop 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 msgr2=0x7f277014c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 0x7f277014c3e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2768009860 tx=0x7f27680046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 shutdown_connections 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f275c06c560 0x7f275c06ea20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f277015e950 0x7f27700cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27700d48c0 0x7f27700c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 --2- 172.21.15.66:0/1971539632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27700d30d0 0x7f277014c3e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 >> 172.21.15.66:0/1971539632 conn(0x7f27700676e0 msgr2=0x7f27700b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 shutdown_connections 2024-03-30T13:10:33.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.259+0000 7f2777daa640 1 -- 172.21.15.66:0/1971539632 wait complete. 2024-03-30T13:10:33.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T13:10:33.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.380+0000 7f6bf1435640 1 Processor -- start 2024-03-30T13:10:33.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.380+0000 7f6bf1435640 1 -- start start 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6bf1435640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6bec0d27a0 0x7f6bec0d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6bf1435640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d48a0 0x7f6bec15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6bf1435640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bec0b3c30 con 0x7f6bec15c000 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6bf1435640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6bec0d2c00 con 0x7f6bec0d27a0 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6bf1435640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6bec0d4cf0 con 0x7f6bec0d48a0 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6bec0d27a0 0x7f6bec0d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6bec0d27a0 0x7f6bec0d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42816/0 (socket says 172.21.15.66:42816) 2024-03-30T13:10:33.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 -- 172.21.15.66:0/969142589 learned_addr learned my addr 172.21.15.66:0/969142589 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6beb7fe640 1 --2- 172.21.15.66:0/969142589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d48a0 0x7f6bec15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 -- 172.21.15.66:0/969142589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d48a0 msgr2=0x7f6bec15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 --2- 172.21.15.66:0/969142589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d48a0 0x7f6bec15edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 -- 172.21.15.66:0/969142589 >> v1:172.21.15.117:6789/0 conn(0x7f6bec15c000 legacy=0x7f6bec0b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6be3fff640 1 -- 172.21.15.66:0/969142589 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec162110 con 0x7f6bec0d27a0 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.381+0000 7f6bea7fc640 1 -- 172.21.15.66:0/969142589 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308740565 0 0) 0x7f6bec0b3c30 con 0x7f6bec15c000 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.382+0000 7f6be3fff640 1 --2- 172.21.15.66:0/969142589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec0d27a0 0x7f6bec0d4360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009870 tx=0x7f6bd402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770fe1ea8b340fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.382+0000 7f6bea7fc640 1 -- 172.21.15.66:0/969142589 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4035020 con 0x7f6bec0d27a0 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.382+0000 7f6bea7fc640 1 -- 172.21.15.66:0/969142589 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd4031640 con 0x7f6bec0d27a0 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.382+0000 7f6bea7fc640 1 -- 172.21.15.66:0/969142589 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd402d040 con 0x7f6bec0d27a0 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 -- 172.21.15.66:0/969142589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec0d27a0 msgr2=0x7f6bec0d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 --2- 172.21.15.66:0/969142589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec0d27a0 0x7f6bec0d4360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009870 tx=0x7f6bd402b8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 -- 172.21.15.66:0/969142589 shutdown_connections 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 --2- 172.21.15.66:0/969142589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d48a0 0x7f6bec15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 --2- 172.21.15.66:0/969142589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec0d27a0 0x7f6bec0d4360 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 -- 172.21.15.66:0/969142589 >> 172.21.15.66:0/969142589 conn(0x7f6bec0676e0 msgr2=0x7f6bec067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 -- 172.21.15.66:0/969142589 shutdown_connections 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.383+0000 7f6bf1435640 1 -- 172.21.15.66:0/969142589 wait complete. 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.384+0000 7f6bf1435640 1 Processor -- start 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.384+0000 7f6bf1435640 1 -- start start 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d27a0 0x7f6bec14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec0d48a0 0x7f6bec0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 0x7f6bec0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec162990 con 0x7f6bec15c000 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1626b0 con 0x7f6bec0d48a0 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6bec162c70 con 0x7f6bec0d27a0 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 0x7f6bec0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 0x7f6bec0ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42824/0 (socket says 172.21.15.66:42824) 2024-03-30T13:10:33.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 -- 172.21.15.66:0/1014331625 learned_addr learned my addr 172.21.15.66:0/1014331625 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beaffd640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d27a0 0x7f6bec14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6be3fff640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec0d48a0 0x7f6bec0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 -- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d27a0 msgr2=0x7f6bec14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d27a0 0x7f6bec14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 -- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec0d48a0 msgr2=0x7f6bec0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec0d48a0 0x7f6bec0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 -- 172.21.15.66:0/1014331625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec1579d0 con 0x7f6bec15c000 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6beb7fe640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 0x7f6bec0ca860 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6be400efc0 tx=0x7f6be400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4018020 con 0x7f6bec15c000 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be4014480 con 0x7f6bec15c000 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec0b5510 con 0x7f6bec15c000 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4014870 con 0x7f6bec15c000 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.385+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1f7d30 con 0x7f6bec15c000 2024-03-30T13:10:33.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.386+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bec075890 con 0x7f6bec15c000 2024-03-30T13:10:33.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.387+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6be4005d10 con 0x7f6bec15c000 2024-03-30T13:10:33.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.387+0000 7f6be37fe640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6bcc06c530 0x7f6bcc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.387+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f6be408c4c0 con 0x7f6bec15c000 2024-03-30T13:10:33.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.387+0000 7f6beaffd640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6bcc06c530 0x7f6bcc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.388+0000 7f6beaffd640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6bcc06c530 0x7f6bcc06e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc000f60 tx=0x7f6bdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.392+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6be4022070 con 0x7f6bec15c000 2024-03-30T13:10:33.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.577+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6bec068910 con 0x7f6bec15c000 2024-03-30T13:10:33.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.580+0000 7f6be37fe640 1 -- 172.21.15.66:0/1014331625 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f6bec1579d0 con 0x7f6bec15c000 2024-03-30T13:10:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6bcc06c530 msgr2=0x7f6bcc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6bcc06c530 0x7f6bcc06e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc000f60 tx=0x7f6bdc009040 comp rx=0 tx=0).stop 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 msgr2=0x7f6bec0ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 0x7f6bec0ca860 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6be400efc0 tx=0x7f6be400c490 comp rx=0 tx=0).stop 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 shutdown_connections 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6bcc06c530 0x7f6bcc06e9f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bec15c000 0x7f6bec0ca860 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec0d48a0 0x7f6bec0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 --2- 172.21.15.66:0/1014331625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bec0d27a0 0x7f6bec14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 >> 172.21.15.66:0/1014331625 conn(0x7f6bec0676e0 msgr2=0x7f6bec0b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.582+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 shutdown_connections 2024-03-30T13:10:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.583+0000 7f6bf1435640 1 -- 172.21.15.66:0/1014331625 wait complete. 2024-03-30T13:10:33.599 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T13:10:33.599 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:10:33.647 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:10:33.686 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T13:10:33.686 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:33.686 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:33.701 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:33.702 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:33.757 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:33.757 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:33.813 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:33.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.957+0000 7efc287fd640 1 Processor -- start 2024-03-30T13:10:33.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.957+0000 7efc287fd640 1 -- start start 2024-03-30T13:10:33.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc287fd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2015c400 0x7efc200cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc287fd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc200b43c0 0x7efc200b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc287fd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc200d2850 con 0x7efc200cf3c0 2024-03-30T13:10:33.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc287fd640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7efc200d2b30 con 0x7efc200b43c0 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc287fd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efc200d2e10 con 0x7efc2015c400 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc25d98640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efc200cf3c0 0x7efc200b3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50334/0 (socket says 172.21.15.66:50334) 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc25d98640 1 -- 172.21.15.66:0/3364058562 learned_addr learned my addr 172.21.15.66:0/3364058562 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc26d9a640 1 --2- 172.21.15.66:0/3364058562 >> v2:172.21.15.117:3300/0 conn(0x7efc200b43c0 0x7efc200b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc26599640 1 --2- 172.21.15.66:0/3364058562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2015c400 0x7efc200cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc25597640 1 -- 172.21.15.66:0/3364058562 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464478253 0 0) 0x7efc200d2850 con 0x7efc200cf3c0 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.958+0000 7efc25597640 1 -- 172.21.15.66:0/3364058562 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc08003520 con 0x7efc200cf3c0 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.959+0000 7efc26599640 1 -- 172.21.15.66:0/3364058562 >> v2:172.21.15.117:3300/0 conn(0x7efc200b43c0 msgr2=0x7efc200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.959+0000 7efc26599640 1 --2- 172.21.15.66:0/3364058562 >> v2:172.21.15.117:3300/0 conn(0x7efc200b43c0 0x7efc200b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.959+0000 7efc26599640 1 -- 172.21.15.66:0/3364058562 >> v1:172.21.15.66:6789/0 conn(0x7efc200cf3c0 legacy=0x7efc200b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.959+0000 7efc26599640 1 -- 172.21.15.66:0/3364058562 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc200ccd80 con 0x7efc2015c400 2024-03-30T13:10:33.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.959+0000 7efc26d9a640 1 --2- 172.21.15.66:0/3364058562 >> v2:172.21.15.117:3300/0 conn(0x7efc200b43c0 0x7efc200b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.959+0000 7efc26599640 1 --2- 172.21.15.66:0/3364058562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2015c400 0x7efc200cc840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc100097a0 tx=0x7efc1002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d318411d6e58540c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc25597640 1 -- 172.21.15.66:0/3364058562 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10031780 con 0x7efc2015c400 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc25597640 1 -- 172.21.15.66:0/3364058562 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc10037070 con 0x7efc2015c400 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc25597640 1 -- 172.21.15.66:0/3364058562 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10031b20 con 0x7efc2015c400 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 -- 172.21.15.66:0/3364058562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2015c400 msgr2=0x7efc200cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 --2- 172.21.15.66:0/3364058562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2015c400 0x7efc200cc840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc100097a0 tx=0x7efc1002b820 comp rx=0 tx=0).stop 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 -- 172.21.15.66:0/3364058562 shutdown_connections 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 --2- 172.21.15.66:0/3364058562 >> v2:172.21.15.117:3300/0 conn(0x7efc200b43c0 0x7efc200b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 --2- 172.21.15.66:0/3364058562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2015c400 0x7efc200cc840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 -- 172.21.15.66:0/3364058562 >> 172.21.15.66:0/3364058562 conn(0x7efc200672c0 msgr2=0x7efc200676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 -- 172.21.15.66:0/3364058562 shutdown_connections 2024-03-30T13:10:33.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.960+0000 7efc287fd640 1 -- 172.21.15.66:0/3364058562 wait complete. 2024-03-30T13:10:33.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.961+0000 7efc287fd640 1 Processor -- start 2024-03-30T13:10:33.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.961+0000 7efc287fd640 1 -- start start 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc287fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 0x7efc200c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc287fd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc200cf3c0 0x7efc200c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc287fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2015c400 0x7efc20159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc287fd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc200b5ac0 con 0x7efc200b43c0 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc287fd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efc200b5940 con 0x7efc2015c400 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc287fd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efc200b5c40 con 0x7efc200cf3c0 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc26599640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 0x7efc200c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc26599640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 0x7efc200c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42832/0 (socket says 172.21.15.66:42832) 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc26599640 1 -- 172.21.15.66:0/2387485398 learned_addr learned my addr 172.21.15.66:0/2387485398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:33.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc26d9a640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2015c400 0x7efc20159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.962+0000 7efc25d98640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc200cf3c0 0x7efc200c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26599640 1 -- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc200cf3c0 msgr2=0x7efc200c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26599640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc200cf3c0 0x7efc200c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26599640 1 -- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2015c400 msgr2=0x7efc20159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26599640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2015c400 0x7efc20159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26599640 1 -- 172.21.15.66:0/2387485398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc200c9af0 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26d9a640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2015c400 0x7efc20159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc26599640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 0x7efc200c92c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc1002b860 tx=0x7efc100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10032610 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc100327b0 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10032b80 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc200c9dd0 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.963+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc200ca390 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.964+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc20075a10 con 0x7efc200b43c0 2024-03-30T13:10:33.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.964+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc1004c060 con 0x7efc200b43c0 2024-03-30T13:10:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.965+0000 7efc1effd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efc0c06c530 0x7efc0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.965+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7efc100ad470 con 0x7efc200b43c0 2024-03-30T13:10:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.965+0000 7efc25d98640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efc0c06c530 0x7efc0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.965+0000 7efc25d98640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efc0c06c530 0x7efc0c06e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efc200d16b0 tx=0x7efc14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:33.969+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc10040070 con 0x7efc200b43c0 2024-03-30T13:10:34.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.154+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc20067980 con 0x7efc200b43c0 2024-03-30T13:10:34.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.155+0000 7efc1effd640 1 -- 172.21.15.66:0/2387485398 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7efc1002f050 con 0x7efc200b43c0 2024-03-30T13:10:34.159 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:34.159 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:34.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efc0c06c530 msgr2=0x7efc0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efc0c06c530 0x7efc0c06e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efc200d16b0 tx=0x7efc14009040 comp rx=0 tx=0).stop 2024-03-30T13:10:34.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 msgr2=0x7efc200c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 0x7efc200c92c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc1002b860 tx=0x7efc100046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 shutdown_connections 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efc0c06c530 0x7efc0c06e9f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2015c400 0x7efc20159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc200cf3c0 0x7efc200c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 --2- 172.21.15.66:0/2387485398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc200b43c0 0x7efc200c92c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 >> 172.21.15.66:0/2387485398 conn(0x7efc200672c0 msgr2=0x7efc200d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 shutdown_connections 2024-03-30T13:10:34.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.157+0000 7efc287fd640 1 -- 172.21.15.66:0/2387485398 wait complete. 2024-03-30T13:10:34.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:34.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.280+0000 7fd209f82640 1 Processor -- start 2024-03-30T13:10:34.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.280+0000 7fd209f82640 1 -- start start 2024-03-30T13:10:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd209f82640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd20409aee0 0x7fd2040d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd209f82640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20409b370 con 0x7fd20409aa30 2024-03-30T13:10:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd209f82640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20415f180 con 0x7fd204071020 2024-03-30T13:10:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd209f82640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd2040d0770 con 0x7fd20409aee0 2024-03-30T13:10:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd202ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd20409aa30 0x7fd2040d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50336/0 (socket says 172.21.15.66:50336) 2024-03-30T13:10:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd202ffd640 1 -- 172.21.15.66:0/1451678122 learned_addr learned my addr 172.21.15.66:0/1451678122 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd203fff640 1 --2- 172.21.15.66:0/1451678122 >> v2:172.21.15.117:3300/0 conn(0x7fd20409aee0 0x7fd2040d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910729623 0 0) 0x7fd20409b370 con 0x7fd20409aa30 2024-03-30T13:10:34.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1ec003520 con 0x7fd20409aa30 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 275283835 0 0) 0x7fd20415f180 con 0x7fd204071020 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd20409b370 con 0x7fd204071020 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.281+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1414057289 0 0) 0x7fd1ec003520 con 0x7fd20409aa30 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 >> v2:172.21.15.117:3300/0 conn(0x7fd20409aee0 msgr2=0x7fd2040d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 --2- 172.21.15.66:0/1451678122 >> v2:172.21.15.117:3300/0 conn(0x7fd20409aee0 0x7fd2040d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 >> v1:172.21.15.117:6789/0 conn(0x7fd204071020 legacy=0x7fd20406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20415f460 con 0x7fd20409aa30 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7fd1f8001670 con 0x7fd20409aa30 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 --2- 172.21.15.66:0/1451678122 >> v2:172.21.15.117:3300/0 conn(0x7fd20409aee0 0x7fd2040d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 >> v1:172.21.15.66:6789/0 conn(0x7fd20409aa30 legacy=0x7fd2040d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1ec003b40 0x7fd1ec003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1ec004800 0x7fd1ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd1ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd2040d0770 con 0x7fd1ec0051c0 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec00c040 con 0x7fd1ec003b40 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ec00c240 con 0x7fd1ec004800 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd2037fe640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1ec003b40 0x7fd1ec003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd1ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd202ffd640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1ec004800 0x7fd1ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 -- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1ec004800 msgr2=0x7fd1ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1ec004800 0x7fd1ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 -- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1ec003b40 msgr2=0x7fd1ec003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1ec003b40 0x7fd1ec003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd203fff640 1 -- 172.21.15.66:0/1451678122 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec003520 con 0x7fd1ec0051c0 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.282+0000 7fd202ffd640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1ec004800 0x7fd1ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd203fff640 1 -- 172.21.15.66:0/1451678122 reap_dead start 2024-03-30T13:10:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd203fff640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd1ec00b9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00e350 tx=0x7fd1ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82acb9cc2903ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc039070 con 0x7fd1ec0051c0 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc02ce20 con 0x7fd1ec0051c0 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd2027fc640 1 -- 172.21.15.66:0/1451678122 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc009de0 con 0x7fd1ec0051c0 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd209f82640 1 -- 172.21.15.66:0/1451678122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 msgr2=0x7fd1ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd209f82640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd1ec00b9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00e350 tx=0x7fd1ec005830 comp rx=0 tx=0).stop 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd209f82640 1 -- 172.21.15.66:0/1451678122 shutdown_connections 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd209f82640 1 --2- 172.21.15.66:0/1451678122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd1ec00b9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.283+0000 7fd209f82640 1 -- 172.21.15.66:0/1451678122 >> 172.21.15.66:0/1451678122 conn(0x7fd2040b3ce0 msgr2=0x7fd2040b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.284+0000 7fd209f82640 1 -- 172.21.15.66:0/1451678122 shutdown_connections 2024-03-30T13:10:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.284+0000 7fd209f82640 1 -- 172.21.15.66:0/1451678122 wait complete. 2024-03-30T13:10:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.285+0000 7fd209f82640 1 Processor -- start 2024-03-30T13:10:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.285+0000 7fd209f82640 1 -- start start 2024-03-30T13:10:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd20414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd20415bae0 0x7fd2040b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2040b6560 0x7fd2040afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd204071400 con 0x7fd1ec0051c0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd20415fa20 con 0x7fd20415bae0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd20406f620 con 0x7fd2040b6560 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd20414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd20414c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42846/0 (socket says 172.21.15.66:42846) 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 -- 172.21.15.66:0/2736799483 learned_addr learned my addr 172.21.15.66:0/2736799483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd202ffd640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd20415bae0 0x7fd2040b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd203fff640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2040b6560 0x7fd2040afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 -- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2040b6560 msgr2=0x7fd2040afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2040b6560 0x7fd2040afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 -- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd20415bae0 msgr2=0x7fd2040b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd20415bae0 0x7fd2040b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 -- 172.21.15.66:0/2736799483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20415c050 con 0x7fd1ec0051c0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd2037fe640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd20414c6a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400f800 tx=0x7fd1f400fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f401a070 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f4005320 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20415c330 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4007270 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.286+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20415c610 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.287+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204075890 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.288+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd1f401d070 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.288+0000 7fd1ebfff640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd1f006c530 0x7fd1f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.288+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fd1f408b160 con 0x7fd1ec0051c0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.288+0000 7fd202ffd640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd1f006c530 0x7fd1f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.289+0000 7fd202ffd640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd1f006c530 0x7fd1f006e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8001eb0 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.293+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd1f4058da0 con 0x7fd1ec0051c0 2024-03-30T13:10:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.479+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd20406cdb0 con 0x7fd1ec0051c0 2024-03-30T13:10:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.479+0000 7fd1ebfff640 1 -- 172.21.15.66:0/2736799483 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fd1f405d0a0 con 0x7fd1ec0051c0 2024-03-30T13:10:34.483 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:34.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd1f006c530 msgr2=0x7fd1f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd1f006c530 0x7fd1f006e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8001eb0 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:34.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 msgr2=0x7fd20414c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd20414c6a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400f800 tx=0x7fd1f400fcc0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 shutdown_connections 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd1f006c530 0x7fd1f006e9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2040b6560 0x7fd2040afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd20415bae0 0x7fd2040b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 --2- 172.21.15.66:0/2736799483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1ec0051c0 0x7fd20414c6a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 >> 172.21.15.66:0/2736799483 conn(0x7fd2040b3ce0 msgr2=0x7fd2040d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 shutdown_connections 2024-03-30T13:10:34.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.481+0000 7fd209f82640 1 -- 172.21.15.66:0/2736799483 wait complete. 2024-03-30T13:10:34.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:34.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.602+0000 7f2e4b115640 1 Processor -- start 2024-03-30T13:10:34.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.602+0000 7f2e4b115640 1 -- start start 2024-03-30T13:10:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e4b115640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e440b4a10 0x7f2e440b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e4b115640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e4409b140 0x7f2e440b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e4b115640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e440cd270 con 0x7f2e4415c500 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e4b115640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2e440cd550 con 0x7f2e4409b140 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e4b115640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e440cd830 con 0x7f2e440b4a10 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e4409b140 0x7f2e440b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2e4409b140 0x7f2e440b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42854/0 (socket says 172.21.15.66:42854) 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 -- 172.21.15.66:0/1378439760 learned_addr learned my addr 172.21.15.66:0/1378439760 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e43fff640 1 --2- 172.21.15.66:0/1378439760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e440b4a10 0x7f2e440b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 -- 172.21.15.66:0/1378439760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e440b4a10 msgr2=0x7f2e440b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 --2- 172.21.15.66:0/1378439760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e440b4a10 0x7f2e440b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 -- 172.21.15.66:0/1378439760 >> v1:172.21.15.117:6789/0 conn(0x7f2e4415c500 legacy=0x7f2e4409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.603+0000 7f2e496b2640 1 -- 172.21.15.66:0/1378439760 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e440cdb10 con 0x7f2e4409b140 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.604+0000 7f2e496b2640 1 --2- 172.21.15.66:0/1378439760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409b140 0x7f2e440b13c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009e80 tx=0x7f2e3c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de92bb8c6c6690e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.604+0000 7f2e437fe640 1 -- 172.21.15.66:0/1378439760 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c02cb70 con 0x7f2e4409b140 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.604+0000 7f2e437fe640 1 -- 172.21.15.66:0/1378439760 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c02cd10 con 0x7f2e4409b140 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.604+0000 7f2e437fe640 1 -- 172.21.15.66:0/1378439760 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c034610 con 0x7f2e4409b140 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 -- 172.21.15.66:0/1378439760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409b140 msgr2=0x7f2e440b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 --2- 172.21.15.66:0/1378439760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409b140 0x7f2e440b13c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009e80 tx=0x7f2e3c02bfd0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 -- 172.21.15.66:0/1378439760 shutdown_connections 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 --2- 172.21.15.66:0/1378439760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e4409b140 0x7f2e440b13c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 --2- 172.21.15.66:0/1378439760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e440b4a10 0x7f2e440b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 -- 172.21.15.66:0/1378439760 >> 172.21.15.66:0/1378439760 conn(0x7f2e440b3ce0 msgr2=0x7f2e440b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 -- 172.21.15.66:0/1378439760 shutdown_connections 2024-03-30T13:10:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.605+0000 7f2e4b115640 1 -- 172.21.15.66:0/1378439760 wait complete. 2024-03-30T13:10:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.606+0000 7f2e4b115640 1 Processor -- start 2024-03-30T13:10:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.606+0000 7f2e4b115640 1 -- start start 2024-03-30T13:10:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e4b115640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4409b140 0x7f2e4415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e4b115640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 0x7f2e4415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e4b115640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4415c500 0x7f2e44147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e4b115640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e440b1900 con 0x7f2e440b4a10 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e4b115640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e440ccee0 con 0x7f2e4409b140 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e4b115640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e440b1a80 con 0x7f2e4415c500 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 0x7f2e4415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 0x7f2e4415e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42858/0 (socket says 172.21.15.66:42858) 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 -- 172.21.15.66:0/3108723842 learned_addr learned my addr 172.21.15.66:0/3108723842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e496b2640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4415c500 0x7f2e44147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e48eb1640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4409b140 0x7f2e4415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 -- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4415c500 msgr2=0x7f2e44147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4415c500 0x7f2e44147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 -- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4409b140 msgr2=0x7f2e4415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4409b140 0x7f2e4415c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 -- 172.21.15.66:0/3108723842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4415f010 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.607+0000 7f2e43fff640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 0x7f2e4415e560 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9b0 tx=0x7f2e3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.608+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30017070 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.608+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3000c9a0 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.608+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4415f2f0 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.608+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000cd90 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.608+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4415ce20 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.609+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44075890 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.609+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e3001a070 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.610+0000 7f2e417fa640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2806c530 0x7f2e2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.610+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f2e3008bdb0 con 0x7f2e440b4a10 2024-03-30T13:10:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.610+0000 7f2e48eb1640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2806c530 0x7f2e2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.610+0000 7f2e48eb1640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2806c530 0x7f2e2806e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e380016e0 tx=0x7f2e38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.614+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e300599f0 con 0x7f2e440b4a10 2024-03-30T13:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.797+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e440ce9e0 con 0x7f2e440b4a10 2024-03-30T13:10:34.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.798+0000 7f2e417fa640 1 -- 172.21.15.66:0/3108723842 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f2e3005dcf0 con 0x7f2e440b4a10 2024-03-30T13:10:34.802 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2806c530 msgr2=0x7f2e2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2806c530 0x7f2e2806e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e380016e0 tx=0x7f2e38009040 comp rx=0 tx=0).stop 2024-03-30T13:10:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 msgr2=0x7f2e4415e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 0x7f2e4415e560 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9b0 tx=0x7f2e3000ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:34.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 shutdown_connections 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e2806c530 0x7f2e2806e9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e4415c500 0x7f2e44147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e440b4a10 0x7f2e4415e560 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 --2- 172.21.15.66:0/3108723842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4409b140 0x7f2e4415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 >> 172.21.15.66:0/3108723842 conn(0x7f2e440b3ce0 msgr2=0x7f2e440b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 shutdown_connections 2024-03-30T13:10:34.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.800+0000 7f2e4b115640 1 -- 172.21.15.66:0/3108723842 wait complete. 2024-03-30T13:10:34.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:34.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.921+0000 7f2fd04c8640 1 Processor -- start 2024-03-30T13:10:34.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.921+0000 7f2fd04c8640 1 -- start start 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.922+0000 7f2fd04c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80b5dc0 0x7f2fc80b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.922+0000 7f2fd04c8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2fc80ba090 0x7f2fc80d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.922+0000 7f2fd04c8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2fc806c620 0x7f2fc806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.922+0000 7f2fd04c8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc80b66e0 con 0x7f2fc80ba090 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.922+0000 7f2fd04c8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc80d2ff0 con 0x7f2fc806c620 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.922+0000 7f2fd04c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc80d32d0 con 0x7f2fc80b5dc0 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2fc80ba090 0x7f2fc80d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2fc80ba090 0x7f2fc80d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42872/0 (socket says 172.21.15.66:42872) 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 -- 172.21.15.66:0/1230428544 learned_addr learned my addr 172.21.15.66:0/1230428544 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fce264640 1 --2- 172.21.15.66:0/1230428544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80b5dc0 0x7f2fc80b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcea65640 1 --2- 172.21.15.66:0/1230428544 >> v2:172.21.15.117:3300/0 conn(0x7f2fc806c620 0x7f2fc806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 -- 172.21.15.66:0/1230428544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80b5dc0 msgr2=0x7f2fc80b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 --2- 172.21.15.66:0/1230428544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80b5dc0 0x7f2fc80b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 -- 172.21.15.66:0/1230428544 >> v2:172.21.15.117:3300/0 conn(0x7f2fc806c620 msgr2=0x7f2fc806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 --2- 172.21.15.66:0/1230428544 >> v2:172.21.15.117:3300/0 conn(0x7f2fc806c620 0x7f2fc806cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.923+0000 7f2fcda63640 1 -- 172.21.15.66:0/1230428544 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc80d35b0 con 0x7f2fc80ba090 2024-03-30T13:10:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.924+0000 7f2fcda63640 1 --2- 172.21.15.66:0/1230428544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc80ba090 0x7f2fc80d2680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80d2120 tx=0x7f2fb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75148f395d3d9376 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.924+0000 7f2fcd262640 1 -- 172.21.15.66:0/1230428544 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8035020 con 0x7f2fc80ba090 2024-03-30T13:10:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.924+0000 7f2fcd262640 1 -- 172.21.15.66:0/1230428544 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb8031870 con 0x7f2fc80ba090 2024-03-30T13:10:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.924+0000 7f2fcd262640 1 -- 172.21.15.66:0/1230428544 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8031c40 con 0x7f2fc80ba090 2024-03-30T13:10:34.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.924+0000 7f2fd04c8640 1 -- 172.21.15.66:0/1230428544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc80ba090 msgr2=0x7f2fc80d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.924+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/1230428544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc80ba090 0x7f2fc80d2680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80d2120 tx=0x7f2fb802b910 comp rx=0 tx=0).stop 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 -- 172.21.15.66:0/1230428544 shutdown_connections 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/1230428544 >> v2:172.21.15.117:3300/0 conn(0x7f2fc806c620 0x7f2fc806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/1230428544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc80ba090 0x7f2fc80d2680 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/1230428544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80b5dc0 0x7f2fc80b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 -- 172.21.15.66:0/1230428544 >> 172.21.15.66:0/1230428544 conn(0x7f2fc80678f0 msgr2=0x7f2fc80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 -- 172.21.15.66:0/1230428544 shutdown_connections 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.925+0000 7f2fd04c8640 1 -- 172.21.15.66:0/1230428544 wait complete. 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.926+0000 7f2fd04c8640 1 Processor -- start 2024-03-30T13:10:34.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.926+0000 7f2fd04c8640 1 -- start start 2024-03-30T13:10:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.926+0000 7f2fd04c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 0x7f2fc80ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.926+0000 7f2fd04c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc80b5dc0 0x7f2fc8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.926+0000 7f2fd04c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80ba090 0x7f2fc8158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fd04c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80d4090 con 0x7f2fc806c620 2024-03-30T13:10:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fd04c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80d3b70 con 0x7f2fc80b5dc0 2024-03-30T13:10:34.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fd04c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc815e880 con 0x7f2fc80ba090 2024-03-30T13:10:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 0x7f2fc80ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 0x7f2fc80ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42882/0 (socket says 172.21.15.66:42882) 2024-03-30T13:10:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 -- 172.21.15.66:0/199728411 learned_addr learned my addr 172.21.15.66:0/199728411 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fcda63640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc80b5dc0 0x7f2fc8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fcea65640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80ba090 0x7f2fc8158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 -- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80ba090 msgr2=0x7f2fc8158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80ba090 0x7f2fc8158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 -- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc80b5dc0 msgr2=0x7f2fc8157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc80b5dc0 0x7f2fc8157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 -- 172.21.15.66:0/199728411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc8160b00 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fce264640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 0x7f2fc80ca590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e9e0 tx=0x7f2fc400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4017070 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc400c9d0 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc8154ea0 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400cdc0 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.927+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc8155460 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.928+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc8075890 con 0x7f2fc806c620 2024-03-30T13:10:34.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.929+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2fc401a070 con 0x7f2fc806c620 2024-03-30T13:10:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.929+0000 7f2fbeffd640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fac06c560 0x7f2fac06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.929+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f2fc408be60 con 0x7f2fc806c620 2024-03-30T13:10:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.929+0000 7f2fcda63640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fac06c560 0x7f2fac06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.930+0000 7f2fcda63640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fac06c560 0x7f2fac06ea20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8036000 tx=0x7f2fb803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:34.934+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2fc4059980 con 0x7f2fc806c620 2024-03-30T13:10:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.117+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2fc815b490 con 0x7f2fc806c620 2024-03-30T13:10:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.118+0000 7f2fbeffd640 1 -- 172.21.15.66:0/199728411 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f2fc405dc80 con 0x7f2fc806c620 2024-03-30T13:10:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fac06c560 msgr2=0x7f2fac06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fac06c560 0x7f2fac06ea20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8036000 tx=0x7f2fb803c040 comp rx=0 tx=0).stop 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 msgr2=0x7f2fc80ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 0x7f2fc80ca590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e9e0 tx=0x7f2fc400eea0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 shutdown_connections 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fac06c560 0x7f2fac06ea20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc80ba090 0x7f2fc8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc80b5dc0 0x7f2fc8157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 --2- 172.21.15.66:0/199728411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc806c620 0x7f2fc80ca590 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 >> 172.21.15.66:0/199728411 conn(0x7f2fc80678f0 msgr2=0x7f2fc80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:35.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 shutdown_connections 2024-03-30T13:10:35.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.120+0000 7f2fd04c8640 1 -- 172.21.15.66:0/199728411 wait complete. 2024-03-30T13:10:35.135 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:35.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.242+0000 7f1dc83e6640 1 Processor -- start 2024-03-30T13:10:35.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.242+0000 7f1dc83e6640 1 -- start start 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc83e6640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1dc013f0c0 0x7f1dc006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc83e6640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1dc006d720 0x7f1dc015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc83e6640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc00ce9c0 con 0x7f1dc01603d0 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc83e6640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc0160e30 con 0x7f1dc006d720 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc83e6640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc0160b50 con 0x7f1dc013f0c0 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5981640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1dc006d720 0x7f1dc015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc6983640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1dc01603d0 0x7f1dc00cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50350/0 (socket says 172.21.15.66:50350) 2024-03-30T13:10:35.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5981640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1dc006d720 0x7f1dc015fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42890/0 (socket says 172.21.15.66:42890) 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc6983640 1 -- 172.21.15.66:0/3072949902 learned_addr learned my addr 172.21.15.66:0/3072949902 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc6182640 1 --2- 172.21.15.66:0/3072949902 >> v2:172.21.15.117:3300/0 conn(0x7f1dc013f0c0 0x7f1dc006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5981640 1 -- 172.21.15.66:0/3072949902 >> v2:172.21.15.117:3300/0 conn(0x7f1dc013f0c0 msgr2=0x7f1dc006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5981640 1 --2- 172.21.15.66:0/3072949902 >> v2:172.21.15.117:3300/0 conn(0x7f1dc013f0c0 0x7f1dc006d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5981640 1 -- 172.21.15.66:0/3072949902 >> v1:172.21.15.66:6789/0 conn(0x7f1dc01603d0 legacy=0x7f1dc00cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5981640 1 -- 172.21.15.66:0/3072949902 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc013fa40 con 0x7f1dc006d720 2024-03-30T13:10:35.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.243+0000 7f1dc5180640 1 -- 172.21.15.66:0/3072949902 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642483295 0 0) 0x7f1db8001490 con 0x7f1dc01603d0 2024-03-30T13:10:35.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.244+0000 7f1dc5981640 1 --2- 172.21.15.66:0/3072949902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc015fe90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1dc006e2b0 tx=0x7f1db002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e6a773d25d7d18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:35.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.244+0000 7f1dc5180640 1 -- 172.21.15.66:0/3072949902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0038070 con 0x7f1dc006d720 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.244+0000 7f1dc5180640 1 -- 172.21.15.66:0/3072949902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0031710 con 0x7f1dc006d720 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.244+0000 7f1dc5180640 1 -- 172.21.15.66:0/3072949902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0031ae0 con 0x7f1dc006d720 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 -- 172.21.15.66:0/3072949902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 msgr2=0x7f1dc015fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/3072949902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc015fe90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1dc006e2b0 tx=0x7f1db002b7b0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 -- 172.21.15.66:0/3072949902 shutdown_connections 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/3072949902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc015fe90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/3072949902 >> v2:172.21.15.117:3300/0 conn(0x7f1dc013f0c0 0x7f1dc006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 -- 172.21.15.66:0/3072949902 >> 172.21.15.66:0/3072949902 conn(0x7f1dc00b5b00 msgr2=0x7f1dc006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 -- 172.21.15.66:0/3072949902 shutdown_connections 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.245+0000 7f1dc83e6640 1 -- 172.21.15.66:0/3072949902 wait complete. 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.246+0000 7f1dc83e6640 1 Processor -- start 2024-03-30T13:10:35.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.246+0000 7f1dc83e6640 1 -- start start 2024-03-30T13:10:35.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc83e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc83e6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc013f0c0 0x7f1dc0155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc83e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc01603d0 0x7f1dc0141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc83e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0160800 con 0x7f1dc006d720 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc83e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0140000 con 0x7f1dc01603d0 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc83e6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc013f4a0 con 0x7f1dc013f0c0 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc013b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc013b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42898/0 (socket says 172.21.15.66:42898) 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 -- 172.21.15.66:0/1648160410 learned_addr learned my addr 172.21.15.66:0/1648160410 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc5981640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc013f0c0 0x7f1dc0155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6983640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc01603d0 0x7f1dc0141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 -- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc013f0c0 msgr2=0x7f1dc0155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc013f0c0 0x7f1dc0155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 -- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc01603d0 msgr2=0x7f1dc0141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc01603d0 0x7f1dc0141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 -- 172.21.15.66:0/1648160410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc013b960 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.247+0000 7f1dc6182640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc013b110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00e9b0 tx=0x7f1dbc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.248+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc017070 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.248+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc00c9a0 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.248+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc013bc40 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.248+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc00cd90 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.248+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc013c200 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.249+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc0075890 con 0x7f1dc006d720 2024-03-30T13:10:35.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.249+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1dbc01a070 con 0x7f1dc006d720 2024-03-30T13:10:35.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.250+0000 7f1db6ffd640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1da406c560 0x7f1da406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.250+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f1dbc08bdd0 con 0x7f1dc006d720 2024-03-30T13:10:35.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.250+0000 7f1dc5981640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1da406c560 0x7f1da406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.250+0000 7f1dc5981640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1da406c560 0x7f1da406ea20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1db0035000 tx=0x7f1db0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:35.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.254+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1dbc059980 con 0x7f1dc006d720 2024-03-30T13:10:35.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.437+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1dc006c2c0 con 0x7f1dc006d720 2024-03-30T13:10:35.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.438+0000 7f1db6ffd640 1 -- 172.21.15.66:0/1648160410 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f1dbc05dc80 con 0x7f1dc006d720 2024-03-30T13:10:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:35.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1da406c560 msgr2=0x7f1da406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1da406c560 0x7f1da406ea20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1db0035000 tx=0x7f1db0008030 comp rx=0 tx=0).stop 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 msgr2=0x7f1dc013b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc013b110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00e9b0 tx=0x7f1dbc00ee70 comp rx=0 tx=0).stop 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 shutdown_connections 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1da406c560 0x7f1da406ea20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc01603d0 0x7f1dc0141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc013f0c0 0x7f1dc0155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 --2- 172.21.15.66:0/1648160410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc006d720 0x7f1dc013b110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 >> 172.21.15.66:0/1648160410 conn(0x7f1dc00b5b00 msgr2=0x7f1dc006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 shutdown_connections 2024-03-30T13:10:35.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.440+0000 7f1dc83e6640 1 -- 172.21.15.66:0/1648160410 wait complete. 2024-03-30T13:10:35.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:10:35.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.560+0000 7f039abbd640 1 Processor -- start 2024-03-30T13:10:35.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.560+0000 7f039abbd640 1 -- start start 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f039abbd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f03940cec60 0x7f03940d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f039abbd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03940d3120 con 0x7f039415e7a0 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f039abbd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03940d3400 con 0x7f03940d3ee0 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f039abbd640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f03940cfc40 con 0x7f03940cec60 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f0393fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f03940cec60 0x7f03940d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f0398959640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f039415e7a0 0x7f03940cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50366/0 (socket says 172.21.15.66:50366) 2024-03-30T13:10:35.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f0398959640 1 -- 172.21.15.66:0/612893268 learned_addr learned my addr 172.21.15.66:0/612893268 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:35.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f0393fff640 1 -- 172.21.15.66:0/612893268 >> v1:172.21.15.117:6789/0 conn(0x7f03940d3ee0 legacy=0x7f03940b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f0393fff640 1 -- 172.21.15.66:0/612893268 >> v1:172.21.15.66:6789/0 conn(0x7f039415e7a0 legacy=0x7f03940cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f0393fff640 1 -- 172.21.15.66:0/612893268 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03940d36e0 con 0x7f03940cec60 2024-03-30T13:10:35.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.561+0000 7f03937fe640 1 -- 172.21.15.66:0/612893268 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 267448072 0 0) 0x7f03940d3120 con 0x7f039415e7a0 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.562+0000 7f0393fff640 1 --2- 172.21.15.66:0/612893268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940cec60 0x7f03940d12b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f03800097a0 tx=0x7f038002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f71e840e0d18cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.562+0000 7f03937fe640 1 -- 172.21.15.66:0/612893268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380035020 con 0x7f03940cec60 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.562+0000 7f03937fe640 1 -- 172.21.15.66:0/612893268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0380031830 con 0x7f03940cec60 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.562+0000 7f03937fe640 1 -- 172.21.15.66:0/612893268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380031c00 con 0x7f03940cec60 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.562+0000 7f039abbd640 1 -- 172.21.15.66:0/612893268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940cec60 msgr2=0x7f03940d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.562+0000 7f039abbd640 1 --2- 172.21.15.66:0/612893268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940cec60 0x7f03940d12b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f03800097a0 tx=0x7f038002b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.563+0000 7f039abbd640 1 -- 172.21.15.66:0/612893268 shutdown_connections 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.563+0000 7f039abbd640 1 --2- 172.21.15.66:0/612893268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940cec60 0x7f03940d12b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.563+0000 7f039abbd640 1 -- 172.21.15.66:0/612893268 >> 172.21.15.66:0/612893268 conn(0x7f03940676b0 msgr2=0x7f0394067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.563+0000 7f039abbd640 1 -- 172.21.15.66:0/612893268 shutdown_connections 2024-03-30T13:10:35.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.563+0000 7f039abbd640 1 -- 172.21.15.66:0/612893268 wait complete. 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.564+0000 7f039abbd640 1 Processor -- start 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.564+0000 7f039abbd640 1 -- start start 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03940cec60 0x7f039415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940d3ee0 0x7f03940b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 0x7f03940cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03940d1970 con 0x7f0394150d30 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03940d17f0 con 0x7f03940cec60 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f03940d1af0 con 0x7f03940d3ee0 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 0x7f03940cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 0x7f03940cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42926/0 (socket says 172.21.15.66:42926) 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 -- 172.21.15.66:0/3612612168 learned_addr learned my addr 172.21.15.66:0/3612612168 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f0398959640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03940cec60 0x7f039415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f0393fff640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940d3ee0 0x7f03940b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 -- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940d3ee0 msgr2=0x7f03940b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940d3ee0 0x7f03940b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 -- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03940cec60 msgr2=0x7f039415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03940cec60 0x7f039415fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 -- 172.21.15.66:0/3612612168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03941604e0 con 0x7f0394150d30 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039915a640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 0x7f03940cdae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f038c00c6b0 tx=0x7f038c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c003b70 con 0x7f0394150d30 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c003d30 con 0x7f0394150d30 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c005310 con 0x7f0394150d30 2024-03-30T13:10:35.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.565+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03941607c0 con 0x7f0394150d30 2024-03-30T13:10:35.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.566+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0394160190 con 0x7f0394150d30 2024-03-30T13:10:35.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.567+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03940cb460 con 0x7f0394150d30 2024-03-30T13:10:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.567+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f038c0054b0 con 0x7f0394150d30 2024-03-30T13:10:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.568+0000 7f03917fa640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c530 0x7f037806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.568+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f038c08b830 con 0x7f0394150d30 2024-03-30T13:10:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.568+0000 7f0398959640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c530 0x7f037806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:35.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.568+0000 7f0398959640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c530 0x7f037806e9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f60 tx=0x7f0388007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:35.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.573+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f03940cb460 con 0x7f0394150d30 2024-03-30T13:10:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.768+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0394075900 con 0x7f0394150d30 2024-03-30T13:10:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.769+0000 7f03917fa640 1 -- 172.21.15.66:0/3612612168 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f038c0593e0 con 0x7f0394150d30 2024-03-30T13:10:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:10:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.770+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c530 msgr2=0x7f037806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.770+0000 7f039abbd640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c530 0x7f037806e9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f60 tx=0x7f0388007e40 comp rx=0 tx=0).stop 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.770+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 msgr2=0x7f03940cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.770+0000 7f039abbd640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 0x7f03940cdae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f038c00c6b0 tx=0x7f038c00cb70 comp rx=0 tx=0).stop 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 shutdown_connections 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c530 0x7f037806e9f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0394150d30 0x7f03940cdae0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940d3ee0 0x7f03940b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 --2- 172.21.15.66:0/3612612168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03940cec60 0x7f039415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 >> 172.21.15.66:0/3612612168 conn(0x7f03940676b0 msgr2=0x7f03940b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 shutdown_connections 2024-03-30T13:10:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:35.771+0000 7f039abbd640 1 -- 172.21.15.66:0/3612612168 wait complete. 2024-03-30T13:10:35.785 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-30T13:10:35.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:35.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-30T13:10:35.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:10:35.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:35.818 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T13:10:35.873 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:35.873 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:35.889 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:35.889 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:35.945 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:35.945 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:35.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:36.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.102+0000 7ff23d1cd640 1 Processor -- start 2024-03-30T13:10:36.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.102+0000 7ff23d1cd640 1 -- start start 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23d1cd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d4a90 0x7ff2380d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23d1cd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2380d2f00 0x7ff2380b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23d1cd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2380d36f0 con 0x7ff23815e950 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23d1cd640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff2380d3cb0 con 0x7ff2380d2f00 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23d1cd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff2380d39d0 con 0x7ff2380d4a90 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff236d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff23815e950 0x7ff2380d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50378/0 (socket says 172.21.15.66:50378) 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff236d9d640 1 -- 172.21.15.66:0/1943230347 learned_addr learned my addr 172.21.15.66:0/1943230347 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23759e640 1 --2- 172.21.15.66:0/1943230347 >> v2:172.21.15.117:3300/0 conn(0x7ff2380d2f00 0x7ff2380b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.103+0000 7ff23659c640 1 --2- 172.21.15.66:0/1943230347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d4a90 0x7ff2380d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff235d9b640 1 -- 172.21.15.66:0/1943230347 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185618450 0 0) 0x7ff2380d36f0 con 0x7ff23815e950 2024-03-30T13:10:36.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff235d9b640 1 -- 172.21.15.66:0/1943230347 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff220003520 con 0x7ff23815e950 2024-03-30T13:10:36.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff23659c640 1 -- 172.21.15.66:0/1943230347 >> v2:172.21.15.117:3300/0 conn(0x7ff2380d2f00 msgr2=0x7ff2380b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:36.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff23659c640 1 --2- 172.21.15.66:0/1943230347 >> v2:172.21.15.117:3300/0 conn(0x7ff2380d2f00 0x7ff2380b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff23659c640 1 -- 172.21.15.66:0/1943230347 >> v1:172.21.15.66:6789/0 conn(0x7ff23815e950 legacy=0x7ff2380d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff23659c640 1 -- 172.21.15.66:0/1943230347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2380b7ac0 con 0x7ff2380d4a90 2024-03-30T13:10:36.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.104+0000 7ff23759e640 1 --2- 172.21.15.66:0/1943230347 >> v2:172.21.15.117:3300/0 conn(0x7ff2380d2f00 0x7ff2380b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:36.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23659c640 1 --2- 172.21.15.66:0/1943230347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d4a90 0x7ff2380d29c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0097a0 tx=0x7ff22c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34688c2c25f3c1c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff235d9b640 1 -- 172.21.15.66:0/1943230347 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c034070 con 0x7ff2380d4a90 2024-03-30T13:10:36.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff235d9b640 1 -- 172.21.15.66:0/1943230347 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22c030800 con 0x7ff2380d4a90 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff235d9b640 1 -- 172.21.15.66:0/1943230347 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c030bd0 con 0x7ff2380d4a90 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 -- 172.21.15.66:0/1943230347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d4a90 msgr2=0x7ff2380d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/1943230347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d4a90 0x7ff2380d29c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0097a0 tx=0x7ff22c02b940 comp rx=0 tx=0).stop 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 -- 172.21.15.66:0/1943230347 shutdown_connections 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/1943230347 >> v2:172.21.15.117:3300/0 conn(0x7ff2380d2f00 0x7ff2380b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/1943230347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d4a90 0x7ff2380d29c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 -- 172.21.15.66:0/1943230347 >> 172.21.15.66:0/1943230347 conn(0x7ff2380676e0 msgr2=0x7ff238067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 -- 172.21.15.66:0/1943230347 shutdown_connections 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.105+0000 7ff23d1cd640 1 -- 172.21.15.66:0/1943230347 wait complete. 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.107+0000 7ff23d1cd640 1 Processor -- start 2024-03-30T13:10:36.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.107+0000 7ff23d1cd640 1 -- start start 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d2f00 0x7ff2380cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 0x7ff23814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23815e950 0x7ff2380ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff23815ed30 con 0x7ff2380d4a90 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2380d33b0 con 0x7ff23815e950 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff23815eeb0 con 0x7ff2380d2f00 2024-03-30T13:10:36.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 0x7ff23814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 0x7ff23814b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42942/0 (socket says 172.21.15.66:42942) 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 -- 172.21.15.66:0/2949497763 learned_addr learned my addr 172.21.15.66:0/2949497763 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23759e640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23815e950 0x7ff2380ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff236d9d640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d2f00 0x7ff2380cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 -- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d2f00 msgr2=0x7ff2380cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d2f00 0x7ff2380cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 -- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23815e950 msgr2=0x7ff2380ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23815e950 0x7ff2380ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 -- 172.21.15.66:0/2949497763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23814be20 con 0x7ff2380d4a90 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23659c640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 0x7ff23814b360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff22c02bf50 tx=0x7ff22c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c034040 con 0x7ff2380d4a90 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22c030e30 con 0x7ff2380d4a90 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23814c100 con 0x7ff2380d4a90 2024-03-30T13:10:36.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c006e10 con 0x7ff2380d4a90 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.108+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23814c6c0 con 0x7ff2380d4a90 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.109+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238075a10 con 0x7ff2380d4a90 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.110+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff22c037070 con 0x7ff2380d4a90 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.110+0000 7ff21f7fe640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff22406c530 0x7ff22406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.110+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7ff22c0ad9b0 con 0x7ff2380d4a90 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.110+0000 7ff236d9d640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff22406c530 0x7ff22406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.111+0000 7ff236d9d640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff22406c530 0x7ff22406e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff230005880 tx=0x7ff230009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.115+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff22c07bc40 con 0x7ff2380d4a90 2024-03-30T13:10:36.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.300+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2380d1480 con 0x7ff2380d4a90 2024-03-30T13:10:36.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.301+0000 7ff21f7fe640 1 -- 172.21.15.66:0/2949497763 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7ff22c07b540 con 0x7ff2380d4a90 2024-03-30T13:10:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:36.305 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff22406c530 msgr2=0x7ff22406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff22406c530 0x7ff22406e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff230005880 tx=0x7ff230009040 comp rx=0 tx=0).stop 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 msgr2=0x7ff23814b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 0x7ff23814b360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff22c02bf50 tx=0x7ff22c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 shutdown_connections 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff22406c530 0x7ff22406e9f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23815e950 0x7ff2380ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2380d4a90 0x7ff23814b360 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 --2- 172.21.15.66:0/2949497763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2380d2f00 0x7ff2380cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 >> 172.21.15.66:0/2949497763 conn(0x7ff2380676e0 msgr2=0x7ff2380d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:36.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 shutdown_connections 2024-03-30T13:10:36.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.303+0000 7ff23d1cd640 1 -- 172.21.15.66:0/2949497763 wait complete. 2024-03-30T13:10:36.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:36.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:36.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.425+0000 7f5b19732640 1 Processor -- start 2024-03-30T13:10:36.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b19732640 1 -- start start 2024-03-30T13:10:36.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b19732640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5b14167120 0x7f5b14169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b19732640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b140d2ff0 con 0x7f5b140b5dc0 2024-03-30T13:10:36.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b19732640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b140d32d0 con 0x7f5b140ba090 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b19732640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5b140d2d90 con 0x7f5b14167120 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b137fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5b14167120 0x7f5b14169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b12ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5b140b5dc0 0x7f5b140b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50382/0 (socket says 172.21.15.66:50382) 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b137fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5b14167120 0x7f5b14169510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42950/0 (socket says 172.21.15.66:42950) 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.426+0000 7f5b12ffd640 1 -- 172.21.15.66:0/3071438783 learned_addr learned my addr 172.21.15.66:0/3071438783 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.427+0000 7f5b137fe640 1 -- 172.21.15.66:0/3071438783 >> v1:172.21.15.117:6789/0 conn(0x7f5b140ba090 legacy=0x7f5b140d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.427+0000 7f5b137fe640 1 -- 172.21.15.66:0/3071438783 >> v1:172.21.15.66:6789/0 conn(0x7f5b140b5dc0 legacy=0x7f5b140b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.427+0000 7f5b11ffb640 1 -- 172.21.15.66:0/3071438783 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939211233 0 0) 0x7f5b140d32d0 con 0x7f5b140ba090 2024-03-30T13:10:36.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.427+0000 7f5b137fe640 1 -- 172.21.15.66:0/3071438783 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b140d35b0 con 0x7f5b14167120 2024-03-30T13:10:36.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b137fe640 1 --2- 172.21.15.66:0/3071438783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14167120 0x7f5b14169510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009dc0 tx=0x7f5b0402bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa0171ef01dead4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b11ffb640 1 -- 172.21.15.66:0/3071438783 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04039070 con 0x7f5b14167120 2024-03-30T13:10:36.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b11ffb640 1 -- 172.21.15.66:0/3071438783 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0402caa0 con 0x7f5b14167120 2024-03-30T13:10:36.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b11ffb640 1 -- 172.21.15.66:0/3071438783 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04034440 con 0x7f5b14167120 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 -- 172.21.15.66:0/3071438783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14167120 msgr2=0x7f5b14169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 --2- 172.21.15.66:0/3071438783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14167120 0x7f5b14169510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009dc0 tx=0x7f5b0402bd70 comp rx=0 tx=0).stop 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 -- 172.21.15.66:0/3071438783 shutdown_connections 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 --2- 172.21.15.66:0/3071438783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14167120 0x7f5b14169510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 -- 172.21.15.66:0/3071438783 >> 172.21.15.66:0/3071438783 conn(0x7f5b140678f0 msgr2=0x7f5b140d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 -- 172.21.15.66:0/3071438783 shutdown_connections 2024-03-30T13:10:36.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.428+0000 7f5b19732640 1 -- 172.21.15.66:0/3071438783 wait complete. 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 Processor -- start 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 -- start start 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 0x7f5b14156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b140ba090 0x7f5b1415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b14167120 0x7f5b1415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140d4090 con 0x7f5b140b5dc0 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140d3b70 con 0x7f5b14167120 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.430+0000 7f5b19732640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5b140b83e0 con 0x7f5b140ba090 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 0x7f5b14156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 0x7f5b14156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42960/0 (socket says 172.21.15.66:42960) 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 -- 172.21.15.66:0/4234478545 learned_addr learned my addr 172.21.15.66:0/4234478545 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:36.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b127fc640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b140ba090 0x7f5b1415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b137fe640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b14167120 0x7f5b1415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 -- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b140ba090 msgr2=0x7f5b1415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b140ba090 0x7f5b1415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 -- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b14167120 msgr2=0x7f5b1415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b14167120 0x7f5b1415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 -- 172.21.15.66:0/4234478545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b14157160 con 0x7f5b140b5dc0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b12ffd640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 0x7f5b14156c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c7b0 tx=0x7f5b0c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c018070 con 0x7f5b140b5dc0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0072d0 con 0x7f5b140b5dc0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1415f810 con 0x7f5b140b5dc0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0047e0 con 0x7f5b140b5dc0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.431+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1415f250 con 0x7f5b140b5dc0 2024-03-30T13:10:36.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.432+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b140b8c30 con 0x7f5b140b5dc0 2024-03-30T13:10:36.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.433+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5b0c01b070 con 0x7f5b140b5dc0 2024-03-30T13:10:36.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.433+0000 7f5b0b7fe640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5af406c530 0x7f5af406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.433+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f5b0c08adb0 con 0x7f5b140b5dc0 2024-03-30T13:10:36.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.433+0000 7f5b127fc640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5af406c530 0x7f5af406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.434+0000 7f5b127fc640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5af406c530 0x7f5af406e9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b140d2d60 tx=0x7f5afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.437+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5b140b8c30 con 0x7f5b140b5dc0 2024-03-30T13:10:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.622+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b14075890 con 0x7f5b140b5dc0 2024-03-30T13:10:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.622+0000 7f5b0b7fe640 1 -- 172.21.15.66:0/4234478545 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f5b0c058ab0 con 0x7f5b140b5dc0 2024-03-30T13:10:36.626 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:36.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5af406c530 msgr2=0x7f5af406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5af406c530 0x7f5af406e9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b140d2d60 tx=0x7f5afc009040 comp rx=0 tx=0).stop 2024-03-30T13:10:36.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 msgr2=0x7f5b14156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 0x7f5b14156c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c7b0 tx=0x7f5b0c00cc70 comp rx=0 tx=0).stop 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 shutdown_connections 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5af406c530 0x7f5af406e9f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b14167120 0x7f5b1415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b140ba090 0x7f5b1415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 --2- 172.21.15.66:0/4234478545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b140b5dc0 0x7f5b14156c20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 >> 172.21.15.66:0/4234478545 conn(0x7f5b140678f0 msgr2=0x7f5b140d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.624+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 shutdown_connections 2024-03-30T13:10:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.625+0000 7f5b19732640 1 -- 172.21.15.66:0/4234478545 wait complete. 2024-03-30T13:10:36.639 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.746+0000 7fb7cdd9d640 1 Processor -- start 2024-03-30T13:10:36.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.746+0000 7fb7cdd9d640 1 -- start start 2024-03-30T13:10:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7cdd9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7c815c220 0x7fb7c80cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7cdd9d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c80ccb20 con 0x7fb7c815c600 2024-03-30T13:10:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7cdd9d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c80ccda0 con 0x7fb7c806e280 2024-03-30T13:10:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7cdd9d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c80677d0 con 0x7fb7c815c220 2024-03-30T13:10:36.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7ccd9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7c815c220 0x7fb7c80cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7c7fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb7c815c600 0x7fb7c806db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50398/0 (socket says 172.21.15.66:50398) 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7ccd9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7c815c220 0x7fb7c80cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42966/0 (socket says 172.21.15.66:42966) 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7c7fff640 1 -- 172.21.15.66:0/3523566057 learned_addr learned my addr 172.21.15.66:0/3523566057 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7ccd9b640 1 -- 172.21.15.66:0/3523566057 >> v1:172.21.15.117:6789/0 conn(0x7fb7c806e280 legacy=0x7fb7c806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7ccd9b640 1 -- 172.21.15.66:0/3523566057 >> v1:172.21.15.66:6789/0 conn(0x7fb7c815c600 legacy=0x7fb7c806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7c77fe640 1 -- 172.21.15.66:0/3523566057 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801926629 0 0) 0x7fb7c80ccda0 con 0x7fb7c806e280 2024-03-30T13:10:36.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.747+0000 7fb7ccd9b640 1 -- 172.21.15.66:0/3523566057 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c8067460 con 0x7fb7c815c220 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.748+0000 7fb7ccd9b640 1 --2- 172.21.15.66:0/3523566057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c80cafb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0097a0 tx=0x7fb7bc02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5cac96731db775 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.748+0000 7fb7c77fe640 1 -- 172.21.15.66:0/3523566057 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc037070 con 0x7fb7c815c220 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.748+0000 7fb7c77fe640 1 -- 172.21.15.66:0/3523566057 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7bc030710 con 0x7fb7c815c220 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.748+0000 7fb7c77fe640 1 -- 172.21.15.66:0/3523566057 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc030ae0 con 0x7fb7c815c220 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/3523566057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 msgr2=0x7fb7c80cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/3523566057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c80cafb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0097a0 tx=0x7fb7bc02b790 comp rx=0 tx=0).stop 2024-03-30T13:10:36.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/3523566057 shutdown_connections 2024-03-30T13:10:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/3523566057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c80cafb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/3523566057 >> 172.21.15.66:0/3523566057 conn(0x7fb7c80cd270 msgr2=0x7fb7c80cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/3523566057 shutdown_connections 2024-03-30T13:10:36.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.749+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/3523566057 wait complete. 2024-03-30T13:10:36.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.750+0000 7fb7cdd9d640 1 Processor -- start 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.750+0000 7fb7cdd9d640 1 -- start start 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cdd9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c806e280 0x7fb7c80c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cdd9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c81e4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cdd9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c815c600 0x7fb7c81e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cdd9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c80cb670 con 0x7fb7c815c220 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cdd9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c80cb4f0 con 0x7fb7c806e280 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cdd9d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c806e750 con 0x7fb7c815c600 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c81e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c81e4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42974/0 (socket says 172.21.15.66:42974) 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 -- 172.21.15.66:0/639540656 learned_addr learned my addr 172.21.15.66:0/639540656 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7ccd9b640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c806e280 0x7fb7c80c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7cd59c640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c815c600 0x7fb7c81e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 -- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c815c600 msgr2=0x7fb7c81e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c815c600 0x7fb7c81e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 -- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c806e280 msgr2=0x7fb7c80c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c806e280 0x7fb7c80c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.751+0000 7fb7c7fff640 1 -- 172.21.15.66:0/639540656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c80c7220 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.752+0000 7fb7c7fff640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c81e4960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c760 tx=0x7fb7b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.752+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4018070 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.752+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b40074e0 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.752+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4007690 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.752+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c80b5e10 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.752+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c8145e10 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.753+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c8075890 con 0x7fb7c815c220 2024-03-30T13:10:36.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.753+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb7b4004a20 con 0x7fb7c815c220 2024-03-30T13:10:36.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.754+0000 7fb7c57fa640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7ac06c530 0x7fb7ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:36.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.754+0000 7fb7ccd9b640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7ac06c530 0x7fb7ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:36.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.754+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fb7b408b150 con 0x7fb7c815c220 2024-03-30T13:10:36.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.754+0000 7fb7ccd9b640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7ac06c530 0x7fb7ac06e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc034000 tx=0x7fb7bc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:36.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.758+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb7c80c7220 con 0x7fb7c815c220 2024-03-30T13:10:36.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.943+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7c815e4c0 con 0x7fb7c815c220 2024-03-30T13:10:36.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.944+0000 7fb7c57fa640 1 -- 172.21.15.66:0/639540656 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fb7c80ccb20 con 0x7fb7c815c220 2024-03-30T13:10:36.948 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:36.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7ac06c530 msgr2=0x7fb7ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7ac06c530 0x7fb7ac06e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc034000 tx=0x7fb7bc02f000 comp rx=0 tx=0).stop 2024-03-30T13:10:36.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 msgr2=0x7fb7c81e4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:36.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c81e4960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c760 tx=0x7fb7b400cc20 comp rx=0 tx=0).stop 2024-03-30T13:10:36.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 shutdown_connections 2024-03-30T13:10:36.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb7ac06c530 0x7fb7ac06e9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c815c600 0x7fb7c81e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7c815c220 0x7fb7c81e4960 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 --2- 172.21.15.66:0/639540656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c806e280 0x7fb7c80c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:36.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 >> 172.21.15.66:0/639540656 conn(0x7fb7c80cd270 msgr2=0x7fb7c80cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:36.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 shutdown_connections 2024-03-30T13:10:36.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:36.946+0000 7fb7cdd9d640 1 -- 172.21.15.66:0/639540656 wait complete. 2024-03-30T13:10:36.961 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:37.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.069+0000 7effbcf59640 1 Processor -- start 2024-03-30T13:10:37.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.069+0000 7effbcf59640 1 -- start start 2024-03-30T13:10:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.069+0000 7effbcf59640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7effb8167120 0x7effb8169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.069+0000 7effbcf59640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb80d2ff0 con 0x7effb80ba090 2024-03-30T13:10:37.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.069+0000 7effbcf59640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb80d32d0 con 0x7effb80b5dc0 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.069+0000 7effbcf59640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7effb80d2d90 con 0x7effb8167120 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effb6d9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7effb8167120 0x7effb8169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effaffff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7effb80ba090 0x7effb80d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50406/0 (socket says 172.21.15.66:50406) 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effaffff640 1 -- 172.21.15.66:0/1042911833 learned_addr learned my addr 172.21.15.66:0/1042911833 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effb6d9d640 1 -- 172.21.15.66:0/1042911833 >> v1:172.21.15.117:6789/0 conn(0x7effb80b5dc0 legacy=0x7effb80b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effb6d9d640 1 -- 172.21.15.66:0/1042911833 >> v1:172.21.15.66:6789/0 conn(0x7effb80ba090 legacy=0x7effb80d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effb5d9b640 1 -- 172.21.15.66:0/1042911833 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434542791 0 0) 0x7effb80d2ff0 con 0x7effb80ba090 2024-03-30T13:10:37.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.070+0000 7effb6d9d640 1 -- 172.21.15.66:0/1042911833 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb80d35b0 con 0x7effb8167120 2024-03-30T13:10:37.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effb6d9d640 1 --2- 172.21.15.66:0/1042911833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb8167120 0x7effb8169510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effb0009dc0 tx=0x7effb002bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0a7c6538b1ad52 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effb5d9b640 1 -- 172.21.15.66:0/1042911833 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0039070 con 0x7effb8167120 2024-03-30T13:10:37.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effb5d9b640 1 -- 172.21.15.66:0/1042911833 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb002ccd0 con 0x7effb8167120 2024-03-30T13:10:37.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effb5d9b640 1 -- 172.21.15.66:0/1042911833 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0034660 con 0x7effb8167120 2024-03-30T13:10:37.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 -- 172.21.15.66:0/1042911833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb8167120 msgr2=0x7effb8169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 --2- 172.21.15.66:0/1042911833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb8167120 0x7effb8169510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effb0009dc0 tx=0x7effb002bf20 comp rx=0 tx=0).stop 2024-03-30T13:10:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 -- 172.21.15.66:0/1042911833 shutdown_connections 2024-03-30T13:10:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 --2- 172.21.15.66:0/1042911833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb8167120 0x7effb8169510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 -- 172.21.15.66:0/1042911833 >> 172.21.15.66:0/1042911833 conn(0x7effb80678f0 msgr2=0x7effb80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 -- 172.21.15.66:0/1042911833 shutdown_connections 2024-03-30T13:10:37.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.071+0000 7effbcf59640 1 -- 172.21.15.66:0/1042911833 wait complete. 2024-03-30T13:10:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 Processor -- start 2024-03-30T13:10:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 -- start start 2024-03-30T13:10:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb80b5dc0 0x7effb8156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 0x7effb81614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8167120 0x7effb815b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effb80d4090 con 0x7effb80ba090 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7effb80d3b70 con 0x7effb8167120 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.073+0000 7effbcf59640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7effb80b83e0 con 0x7effb80b5dc0 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 0x7effb81614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 0x7effb81614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42986/0 (socket says 172.21.15.66:42986) 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 -- 172.21.15.66:0/560273884 learned_addr learned my addr 172.21.15.66:0/560273884 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effb6d9d640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8167120 0x7effb815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effb659c640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb80b5dc0 0x7effb8156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 -- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb80b5dc0 msgr2=0x7effb8156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb80b5dc0 0x7effb8156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 -- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8167120 msgr2=0x7effb815b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8167120 0x7effb815b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 -- 172.21.15.66:0/560273884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb81619e0 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaffff640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 0x7effb81614a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7effa000c4d0 tx=0x7effa000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0018070 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa00074e0 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb8160850 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004a20 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.074+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb8158360 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.075+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb8075890 con 0x7effb80ba090 2024-03-30T13:10:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.076+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7effa001b070 con 0x7effb80ba090 2024-03-30T13:10:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.076+0000 7effaeffd640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7eff9806c530 0x7eff9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.076+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7effa008ace0 con 0x7effb80ba090 2024-03-30T13:10:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.076+0000 7effb659c640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7eff9806c530 0x7eff9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.077+0000 7effb659c640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7eff9806c530 0x7eff9806e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effb80cfbc0 tx=0x7effa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.081+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7effa0058920 con 0x7effb80ba090 2024-03-30T13:10:37.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.264+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7effb80b59b0 con 0x7effb80ba090 2024-03-30T13:10:37.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.269+0000 7effaeffd640 1 -- 172.21.15.66:0/560273884 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7effa005cc20 con 0x7effb80ba090 2024-03-30T13:10:37.273 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7eff9806c530 msgr2=0x7eff9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7eff9806c530 0x7eff9806e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effb80cfbc0 tx=0x7effa8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 msgr2=0x7effb81614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 0x7effb81614a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7effa000c4d0 tx=0x7effa000c990 comp rx=0 tx=0).stop 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 shutdown_connections 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7eff9806c530 0x7eff9806e9f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8167120 0x7effb815b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effb80ba090 0x7effb81614a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 --2- 172.21.15.66:0/560273884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effb80b5dc0 0x7effb8156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 >> 172.21.15.66:0/560273884 conn(0x7effb80678f0 msgr2=0x7effb80d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 shutdown_connections 2024-03-30T13:10:37.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.272+0000 7effbcf59640 1 -- 172.21.15.66:0/560273884 wait complete. 2024-03-30T13:10:37.286 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:37.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.393+0000 7fec0ef15640 1 Processor -- start 2024-03-30T13:10:37.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.393+0000 7fec0ef15640 1 -- start start 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ef15640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec0815a710 0x7fec080b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ef15640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fec0815aaf0 0x7fec080b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ef15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec080b4ba0 0x7fec0806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ef15640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fec080b41d0 con 0x7fec0815aaf0 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ef15640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fec080b35c0 con 0x7fec0815a710 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ef15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fec080b38a0 con 0x7fec080b4ba0 2024-03-30T13:10:37.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fec0815aaf0 0x7fec080b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fec0815aaf0 0x7fec080b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42988/0 (socket says 172.21.15.66:42988) 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 -- 172.21.15.66:0/1468199263 learned_addr learned my addr 172.21.15.66:0/1468199263 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0d4b2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec080b4ba0 0x7fec0806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec0ccb1640 1 --2- 172.21.15.66:0/1468199263 >> v2:172.21.15.117:3300/0 conn(0x7fec0815a710 0x7fec080b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 -- 172.21.15.66:0/1468199263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec080b4ba0 msgr2=0x7fec0806d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 --2- 172.21.15.66:0/1468199263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec080b4ba0 0x7fec0806d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 -- 172.21.15.66:0/1468199263 >> v2:172.21.15.117:3300/0 conn(0x7fec0815a710 msgr2=0x7fec080b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 --2- 172.21.15.66:0/1468199263 >> v2:172.21.15.117:3300/0 conn(0x7fec0815a710 0x7fec080b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.394+0000 7fec07fff640 1 -- 172.21.15.66:0/1468199263 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec080b3b80 con 0x7fec0815aaf0 2024-03-30T13:10:37.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.395+0000 7fec07fff640 1 --2- 172.21.15.66:0/1468199263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080b4660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec080b3ab0 tx=0x7febf402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a524d703976579 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.395+0000 7fec077fe640 1 -- 172.21.15.66:0/1468199263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf403b070 con 0x7fec0815aaf0 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.395+0000 7fec077fe640 1 -- 172.21.15.66:0/1468199263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf4036440 con 0x7fec0815aaf0 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.395+0000 7fec077fe640 1 -- 172.21.15.66:0/1468199263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4036810 con 0x7fec0815aaf0 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 -- 172.21.15.66:0/1468199263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 msgr2=0x7fec080b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1468199263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080b4660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec080b3ab0 tx=0x7febf402e0c0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 -- 172.21.15.66:0/1468199263 shutdown_connections 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1468199263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec080b4ba0 0x7fec0806d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1468199263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080b4660 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1468199263 >> v2:172.21.15.117:3300/0 conn(0x7fec0815a710 0x7fec080b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 -- 172.21.15.66:0/1468199263 >> 172.21.15.66:0/1468199263 conn(0x7fec080b5b00 msgr2=0x7fec0806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 -- 172.21.15.66:0/1468199263 shutdown_connections 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.396+0000 7fec0ef15640 1 -- 172.21.15.66:0/1468199263 wait complete. 2024-03-30T13:10:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.397+0000 7fec0ef15640 1 Processor -- start 2024-03-30T13:10:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.397+0000 7fec0ef15640 1 -- start start 2024-03-30T13:10:37.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ef15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec0815a710 0x7fec080a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ef15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ef15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec08159450 0x7fec080a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ef15640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec080718c0 con 0x7fec0815aaf0 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ef15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fec08072c50 con 0x7fec0815a710 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ef15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fec08071a40 con 0x7fec08159450 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080a99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43004/0 (socket says 172.21.15.66:43004) 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 -- 172.21.15.66:0/1405344830 learned_addr learned my addr 172.21.15.66:0/1405344830 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0ccb1640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec0815a710 0x7fec080a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec0d4b2640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec08159450 0x7fec080a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 -- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec08159450 msgr2=0x7fec080a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec08159450 0x7fec080a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 -- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec0815a710 msgr2=0x7fec080a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec0815a710 0x7fec080a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.398+0000 7fec07fff640 1 -- 172.21.15.66:0/1405344830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec081572d0 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.399+0000 7fec07fff640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080a99d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7febf4000c00 tx=0x7febf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.399+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4045020 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.399+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf4041620 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.399+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec08159930 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.399+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf403b040 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.399+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec08157850 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.400+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec08075890 con 0x7fec0815aaf0 2024-03-30T13:10:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.400+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7febf4048020 con 0x7fec0815aaf0 2024-03-30T13:10:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.401+0000 7fec057fa640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7febec06c530 0x7febec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.401+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7febf40b0250 con 0x7fec0815aaf0 2024-03-30T13:10:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.401+0000 7fec0ccb1640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7febec06c530 0x7febec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.401+0000 7fec0ccb1640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7febec06c530 0x7febec06e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febfc009ac0 tx=0x7febfc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.405+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7febf404f030 con 0x7fec0815aaf0 2024-03-30T13:10:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.588+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fec0806dde0 con 0x7fec0815aaf0 2024-03-30T13:10:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.589+0000 7fec057fa640 1 -- 172.21.15.66:0/1405344830 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7febf407de90 con 0x7fec0815aaf0 2024-03-30T13:10:37.592 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7febec06c530 msgr2=0x7febec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7febec06c530 0x7febec06e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febfc009ac0 tx=0x7febfc009190 comp rx=0 tx=0).stop 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 msgr2=0x7fec080a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080a99d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7febf4000c00 tx=0x7febf40046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 shutdown_connections 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7febec06c530 0x7febec06e9f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec08159450 0x7fec080a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec0815aaf0 0x7fec080a99d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 --2- 172.21.15.66:0/1405344830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec0815a710 0x7fec080a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 >> 172.21.15.66:0/1405344830 conn(0x7fec080b5b00 msgr2=0x7fec080cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 shutdown_connections 2024-03-30T13:10:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.591+0000 7fec0ef15640 1 -- 172.21.15.66:0/1405344830 wait complete. 2024-03-30T13:10:37.606 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:10:37.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.710+0000 7f27b5e88640 1 Processor -- start 2024-03-30T13:10:37.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.710+0000 7f27b5e88640 1 -- start start 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27b5e88640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f27b015e7a0 0x7f27b00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27b5e88640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f27b00cec60 0x7f27b00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27b5e88640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b00d3120 con 0x7f27b00d3ee0 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27b5e88640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f27b00cfc40 con 0x7f27b015e7a0 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27b5e88640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f27b00d3400 con 0x7f27b00cec60 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27af7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f27b015e7a0 0x7f27b00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27affff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f27b00d3ee0 0x7f27b00b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50416/0 (socket says 172.21.15.66:50416) 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27affff640 1 -- 172.21.15.66:0/16719223 learned_addr learned my addr 172.21.15.66:0/16719223 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:37.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27aeffd640 1 --2- 172.21.15.66:0/16719223 >> v2:172.21.15.117:3300/0 conn(0x7f27b00cec60 0x7f27b00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27af7fe640 1 -- 172.21.15.66:0/16719223 >> v2:172.21.15.117:3300/0 conn(0x7f27b00cec60 msgr2=0x7f27b00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27af7fe640 1 --2- 172.21.15.66:0/16719223 >> v2:172.21.15.117:3300/0 conn(0x7f27b00cec60 0x7f27b00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27af7fe640 1 -- 172.21.15.66:0/16719223 >> v1:172.21.15.66:6789/0 conn(0x7f27b00d3ee0 legacy=0x7f27b00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.711+0000 7f27af7fe640 1 -- 172.21.15.66:0/16719223 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b00d36e0 con 0x7f27b015e7a0 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.712+0000 7f27af7fe640 1 --2- 172.21.15.66:0/16719223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b015e7a0 0x7f27b00cf3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f27b00d5460 tx=0x7f27a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b281294f317c2220 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.712+0000 7f27ae7fc640 1 -- 172.21.15.66:0/16719223 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00315a0 con 0x7f27b015e7a0 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.712+0000 7f27ae7fc640 1 -- 172.21.15.66:0/16719223 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a0037070 con 0x7f27b015e7a0 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.712+0000 7f27ae7fc640 1 -- 172.21.15.66:0/16719223 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0031960 con 0x7f27b015e7a0 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 -- 172.21.15.66:0/16719223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b015e7a0 msgr2=0x7f27b00cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 --2- 172.21.15.66:0/16719223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b015e7a0 0x7f27b00cf3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f27b00d5460 tx=0x7f27a002b840 comp rx=0 tx=0).stop 2024-03-30T13:10:37.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 -- 172.21.15.66:0/16719223 shutdown_connections 2024-03-30T13:10:37.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 --2- 172.21.15.66:0/16719223 >> v2:172.21.15.117:3300/0 conn(0x7f27b00cec60 0x7f27b00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 --2- 172.21.15.66:0/16719223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b015e7a0 0x7f27b00cf3f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 -- 172.21.15.66:0/16719223 >> 172.21.15.66:0/16719223 conn(0x7f27b00676b0 msgr2=0x7f27b0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:37.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 -- 172.21.15.66:0/16719223 shutdown_connections 2024-03-30T13:10:37.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.713+0000 7f27b5e88640 1 -- 172.21.15.66:0/16719223 wait complete. 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.714+0000 7f27b5e88640 1 Processor -- start 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.714+0000 7f27b5e88640 1 -- start start 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27b5e88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b00cec60 0x7f27b014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27b5e88640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 0x7f27b014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27b5e88640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b014ff30 0x7f27b014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27b5e88640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27b00d17f0 con 0x7f27b00d3ee0 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27b5e88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f27b00cf930 con 0x7f27b00cec60 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27b5e88640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f27b00d1970 con 0x7f27b014ff30 2024-03-30T13:10:37.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 0x7f27b014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 0x7f27b014f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43008/0 (socket says 172.21.15.66:43008) 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 -- 172.21.15.66:0/2846945677 learned_addr learned my addr 172.21.15.66:0/2846945677 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27af7fe640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b00cec60 0x7f27b014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27affff640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b014ff30 0x7f27b014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 -- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b014ff30 msgr2=0x7f27b014e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b014ff30 0x7f27b014e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 -- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b00cec60 msgr2=0x7f27b014d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b00cec60 0x7f27b014d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.715+0000 7f27aeffd640 1 -- 172.21.15.66:0/2846945677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b014d680 con 0x7f27b00d3ee0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.716+0000 7f27aeffd640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 0x7f27b014f9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f27a400e9e0 tx=0x7f27a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.716+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4017070 con 0x7f27b00d3ee0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.716+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a400c970 con 0x7f27b00d3ee0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.716+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b00cdf10 con 0x7f27b00d3ee0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.716+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a400cd40 con 0x7f27b00d3ee0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.716+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b014dc70 con 0x7f27b00d3ee0 2024-03-30T13:10:37.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.717+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b0150340 con 0x7f27b00d3ee0 2024-03-30T13:10:37.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.717+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f27a401a070 con 0x7f27b00d3ee0 2024-03-30T13:10:37.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.718+0000 7f2797fff640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f279c06c560 0x7f279c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:37.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.718+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f27a408bd40 con 0x7f27b00d3ee0 2024-03-30T13:10:37.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.718+0000 7f27af7fe640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f279c06c560 0x7f279c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:37.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.718+0000 7f27af7fe640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f279c06c560 0x7f279c06ea20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27a0036000 tx=0x7f27a003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:37.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.722+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f27a4059860 con 0x7f27b00d3ee0 2024-03-30T13:10:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.918+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27b0067a90 con 0x7f27b00d3ee0 2024-03-30T13:10:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.919+0000 7f2797fff640 1 -- 172.21.15.66:0/2846945677 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f27a405db60 con 0x7f27b00d3ee0 2024-03-30T13:10:37.922 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f279c06c560 msgr2=0x7f279c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f279c06c560 0x7f279c06ea20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27a0036000 tx=0x7f27a003d040 comp rx=0 tx=0).stop 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 msgr2=0x7f27b014f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 0x7f27b014f9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f27a400e9e0 tx=0x7f27a400eea0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 shutdown_connections 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f279c06c560 0x7f279c06ea20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b014ff30 0x7f27b014e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27b00d3ee0 0x7f27b014f9f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 --2- 172.21.15.66:0/2846945677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b00cec60 0x7f27b014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 >> 172.21.15.66:0/2846945677 conn(0x7f27b00676b0 msgr2=0x7f27b0160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 shutdown_connections 2024-03-30T13:10:37.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:37.921+0000 7f27b5e88640 1 -- 172.21.15.66:0/2846945677 wait complete. 2024-03-30T13:10:37.935 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-30T13:10:37.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:37.936 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-30T13:10:37.964 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:10:37.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:37.965 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T13:10:38.025 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:38.025 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:38.041 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:38.042 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:38.097 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:10:38.097 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:38.154 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:38.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.297+0000 7f30d5cd9640 1 Processor -- start 2024-03-30T13:10:38.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.297+0000 7f30d5cd9640 1 -- start start 2024-03-30T13:10:38.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30d5cd9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30d5cd9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f30d0072340 0x7f30d00cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30d5cd9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f30d00727a0 0x7f30d00cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30d5cd9640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f30d00cd610 con 0x7f30d0072340 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30d5cd9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f30d01449c0 con 0x7f30d00727a0 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30d5cd9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00cf5b0 con 0x7f30d01445e0 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f30d0072340 0x7f30d00cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f30d0072340 0x7f30d00cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43022/0 (socket says 172.21.15.66:43022) 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 -- 172.21.15.66:0/1501894924 learned_addr learned my addr 172.21.15.66:0/1501894924 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30cffff640 1 --2- 172.21.15.66:0/1501894924 >> v2:172.21.15.117:3300/0 conn(0x7f30d00727a0 0x7f30d00cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30cf7fe640 1 --2- 172.21.15.66:0/1501894924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d00cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 -- 172.21.15.66:0/1501894924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 msgr2=0x7f30d00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:38.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 --2- 172.21.15.66:0/1501894924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d00cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 -- 172.21.15.66:0/1501894924 >> v2:172.21.15.117:3300/0 conn(0x7f30d00727a0 msgr2=0x7f30d00cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 --2- 172.21.15.66:0/1501894924 >> v2:172.21.15.117:3300/0 conn(0x7f30d00727a0 0x7f30d00cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.298+0000 7f30ceffd640 1 -- 172.21.15.66:0/1501894924 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d0144e10 con 0x7f30d0072340 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.299+0000 7f30ceffd640 1 --2- 172.21.15.66:0/1501894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00cb690 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f30d0074110 tx=0x7f30c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37519086ba899811 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.299+0000 7f30ce7fc640 1 -- 172.21.15.66:0/1501894924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4035020 con 0x7f30d0072340 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.299+0000 7f30ce7fc640 1 -- 172.21.15.66:0/1501894924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c4031640 con 0x7f30d0072340 2024-03-30T13:10:38.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.299+0000 7f30ce7fc640 1 -- 172.21.15.66:0/1501894924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c402d040 con 0x7f30d0072340 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 -- 172.21.15.66:0/1501894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 msgr2=0x7f30d00cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/1501894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00cb690 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f30d0074110 tx=0x7f30c402b8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 -- 172.21.15.66:0/1501894924 shutdown_connections 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/1501894924 >> v2:172.21.15.117:3300/0 conn(0x7f30d00727a0 0x7f30d00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/1501894924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00cb690 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/1501894924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d00cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 -- 172.21.15.66:0/1501894924 >> 172.21.15.66:0/1501894924 conn(0x7f30d00b1c90 msgr2=0x7f30d00b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 -- 172.21.15.66:0/1501894924 shutdown_connections 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.300+0000 7f30d5cd9640 1 -- 172.21.15.66:0/1501894924 wait complete. 2024-03-30T13:10:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.301+0000 7f30d5cd9640 1 Processor -- start 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.301+0000 7f30d5cd9640 1 -- start start 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30d5cd9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30d5cd9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00727a0 0x7f30d015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30d5cd9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d013f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30d5cd9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00cad60 con 0x7f30d0072340 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30d5cd9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30d015edb0 con 0x7f30d00727a0 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30d5cd9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00cb040 con 0x7f30d01445e0 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43032/0 (socket says 172.21.15.66:43032) 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 -- 172.21.15.66:0/2328083864 learned_addr learned my addr 172.21.15.66:0/2328083864 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30ceffd640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00727a0 0x7f30d015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cffff640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d013f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 -- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 msgr2=0x7f30d013f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d013f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 -- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00727a0 msgr2=0x7f30d015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00727a0 0x7f30d015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30cf7fe640 1 -- 172.21.15.66:0/2328083864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d01406f0 con 0x7f30d0072340 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.302+0000 7f30ceffd640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00727a0 0x7f30d015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.303+0000 7f30cf7fe640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00b3c00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30c000bd10 tx=0x7f30c000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.303+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c001a020 con 0x7f30d0072340 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.303+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c000eb10 con 0x7f30d0072340 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.303+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d01409d0 con 0x7f30d0072340 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.303+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0016480 con 0x7f30d0072340 2024-03-30T13:10:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.303+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d0140f90 con 0x7f30d0072340 2024-03-30T13:10:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.304+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d015ebd0 con 0x7f30d0072340 2024-03-30T13:10:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.304+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f30c001d020 con 0x7f30d0072340 2024-03-30T13:10:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.305+0000 7f30b7fff640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30bc06c530 0x7f30bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.305+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f30c008dcd0 con 0x7f30d0072340 2024-03-30T13:10:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.305+0000 7f30ceffd640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30bc06c530 0x7f30bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.305+0000 7f30ceffd640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30bc06c530 0x7f30bc06e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f30c4036000 tx=0x7f30c403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.309+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f30d015ebd0 con 0x7f30d0072340 2024-03-30T13:10:38.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.495+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30d0075a10 con 0x7f30d0072340 2024-03-30T13:10:38.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.497+0000 7f30b7fff640 1 -- 172.21.15.66:0/2328083864 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f30c0024070 con 0x7f30d0072340 2024-03-30T13:10:38.500 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:38.501 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:38.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30bc06c530 msgr2=0x7f30bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30bc06c530 0x7f30bc06e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f30c4036000 tx=0x7f30c403c040 comp rx=0 tx=0).stop 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 msgr2=0x7f30d00b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00b3c00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30c000bd10 tx=0x7f30c000be00 comp rx=0 tx=0).stop 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 shutdown_connections 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f30bc06c530 0x7f30bc06e9f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30d01445e0 0x7f30d013f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d00727a0 0x7f30d015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 --2- 172.21.15.66:0/2328083864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30d0072340 0x7f30d00b3c00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 >> 172.21.15.66:0/2328083864 conn(0x7f30d00b1c90 msgr2=0x7f30d0143c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 shutdown_connections 2024-03-30T13:10:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.499+0000 7f30d5cd9640 1 -- 172.21.15.66:0/2328083864 wait complete. 2024-03-30T13:10:38.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:38.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:38.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.621+0000 7ff827074640 1 Processor -- start 2024-03-30T13:10:38.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.621+0000 7ff827074640 1 -- start start 2024-03-30T13:10:38.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff827074640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff8200b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff827074640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff8200ba090 0x7ff8200d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff827074640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8200b66e0 con 0x7ff82006c620 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff827074640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff8200d2ff0 con 0x7ff8200ba090 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff827074640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff8200d32d0 con 0x7ff8200b5dc0 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff824e10640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff8200b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff81ffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff8200ba090 0x7ff8200d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff824e10640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff8200b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:49570/0 (socket says 172.21.15.66:49570) 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff824e10640 1 -- 172.21.15.66:0/4290486624 learned_addr learned my addr 172.21.15.66:0/4290486624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.622+0000 7ff825611640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff82006c620 0x7ff82006cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45712/0 (socket says 172.21.15.66:45712) 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.623+0000 7ff824e10640 1 -- 172.21.15.66:0/4290486624 >> v2:172.21.15.117:3300/0 conn(0x7ff8200ba090 msgr2=0x7ff8200d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.623+0000 7ff824e10640 1 --2- 172.21.15.66:0/4290486624 >> v2:172.21.15.117:3300/0 conn(0x7ff8200ba090 0x7ff8200d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.623+0000 7ff824e10640 1 -- 172.21.15.66:0/4290486624 >> v1:172.21.15.117:6789/0 conn(0x7ff82006c620 legacy=0x7ff82006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.623+0000 7ff824e10640 1 -- 172.21.15.66:0/4290486624 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8200d35b0 con 0x7ff8200b5dc0 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.623+0000 7ff824e10640 1 --2- 172.21.15.66:0/4290486624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff8200b61a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff818009e50 tx=0x7ff81802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3289067eba26f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff81f7fe640 1 -- 172.21.15.66:0/4290486624 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81802cb00 con 0x7ff8200b5dc0 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff81f7fe640 1 -- 172.21.15.66:0/4290486624 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81802cca0 con 0x7ff8200b5dc0 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff81f7fe640 1 -- 172.21.15.66:0/4290486624 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818034610 con 0x7ff8200b5dc0 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 -- 172.21.15.66:0/4290486624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 msgr2=0x7ff8200b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 --2- 172.21.15.66:0/4290486624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff8200b61a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff818009e50 tx=0x7ff81802bf60 comp rx=0 tx=0).stop 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 -- 172.21.15.66:0/4290486624 shutdown_connections 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 --2- 172.21.15.66:0/4290486624 >> v2:172.21.15.117:3300/0 conn(0x7ff8200ba090 0x7ff8200d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 --2- 172.21.15.66:0/4290486624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff8200b61a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 -- 172.21.15.66:0/4290486624 >> 172.21.15.66:0/4290486624 conn(0x7ff8200678f0 msgr2=0x7ff8200d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:38.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 -- 172.21.15.66:0/4290486624 shutdown_connections 2024-03-30T13:10:38.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.624+0000 7ff827074640 1 -- 172.21.15.66:0/4290486624 wait complete. 2024-03-30T13:10:38.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.625+0000 7ff827074640 1 Processor -- start 2024-03-30T13:10:38.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.625+0000 7ff827074640 1 -- start start 2024-03-30T13:10:38.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff827074640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 0x7ff820155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff827074640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff820161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff827074640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8200ba090 0x7ff820156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff827074640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200d4090 con 0x7ff82006c620 2024-03-30T13:10:38.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff827074640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8200d3b70 con 0x7ff8200ba090 2024-03-30T13:10:38.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff827074640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff8200b83e0 con 0x7ff8200b5dc0 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff824e10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 0x7ff820155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff824e10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 0x7ff820155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43038/0 (socket says 172.21.15.66:43038) 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff824e10640 1 -- 172.21.15.66:0/1790435544 learned_addr learned my addr 172.21.15.66:0/1790435544 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff81ffff640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff820161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.626+0000 7ff825611640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8200ba090 0x7ff820156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff824e10640 1 -- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 msgr2=0x7ff820161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff824e10640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff820161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff824e10640 1 -- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8200ba090 msgr2=0x7ff820156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff824e10640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8200ba090 0x7ff820156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff824e10640 1 -- 172.21.15.66:0/1790435544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8201549e0 con 0x7ff82006c620 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff824e10640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 0x7ff820155120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff818036040 tx=0x7ff818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818043020 con 0x7ff82006c620 2024-03-30T13:10:38.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81802ce50 con 0x7ff82006c620 2024-03-30T13:10:38.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff820154cc0 con 0x7ff82006c620 2024-03-30T13:10:38.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818009510 con 0x7ff82006c620 2024-03-30T13:10:38.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.627+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82015f8b0 con 0x7ff82006c620 2024-03-30T13:10:38.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.628+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8200b9230 con 0x7ff82006c620 2024-03-30T13:10:38.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.629+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff818046020 con 0x7ff82006c620 2024-03-30T13:10:38.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.629+0000 7ff81d7fa640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff80c06c530 0x7ff80c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.629+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7ff8180ae000 con 0x7ff82006c620 2024-03-30T13:10:38.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.629+0000 7ff81ffff640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff80c06c530 0x7ff80c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.630+0000 7ff81ffff640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff80c06c530 0x7ff80c06e9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8201480c0 tx=0x7ff814009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.634+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff8200b9230 con 0x7ff82006c620 2024-03-30T13:10:38.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.817+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff820075890 con 0x7ff82006c620 2024-03-30T13:10:38.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.817+0000 7ff81d7fa640 1 -- 172.21.15.66:0/1790435544 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7ff81804d030 con 0x7ff82006c620 2024-03-30T13:10:38.821 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:38.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff80c06c530 msgr2=0x7ff80c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff80c06c530 0x7ff80c06e9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8201480c0 tx=0x7ff814009190 comp rx=0 tx=0).stop 2024-03-30T13:10:38.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 msgr2=0x7ff820155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 0x7ff820155120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff818036040 tx=0x7ff818004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 shutdown_connections 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff80c06c530 0x7ff80c06e9f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8200ba090 0x7ff820156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8200b5dc0 0x7ff820161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 --2- 172.21.15.66:0/1790435544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff82006c620 0x7ff820155120 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.819+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 >> 172.21.15.66:0/1790435544 conn(0x7ff8200678f0 msgr2=0x7ff8200b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.820+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 shutdown_connections 2024-03-30T13:10:38.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.820+0000 7ff827074640 1 -- 172.21.15.66:0/1790435544 wait complete. 2024-03-30T13:10:38.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.941+0000 7fd665704640 1 Processor -- start 2024-03-30T13:10:38.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.941+0000 7fd665704640 1 -- start start 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd665704640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd66013f0c0 0x7fd66006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd665704640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6600ce9c0 con 0x7fd66006d720 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd665704640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd660160b50 con 0x7fd6601605a0 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd665704640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd660160e30 con 0x7fd66013f0c0 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd65effd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd66013f0c0 0x7fd66006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd657fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd66006d720 0x7fd66015fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50432/0 (socket says 172.21.15.66:50432) 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd657fff640 1 -- 172.21.15.66:0/106286910 learned_addr learned my addr 172.21.15.66:0/106286910 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd65effd640 1 -- 172.21.15.66:0/106286910 >> v1:172.21.15.117:6789/0 conn(0x7fd6601605a0 legacy=0x7fd6600cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd65effd640 1 -- 172.21.15.66:0/106286910 >> v1:172.21.15.66:6789/0 conn(0x7fd66006d720 legacy=0x7fd66015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd65effd640 1 -- 172.21.15.66:0/106286910 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66013fa40 con 0x7fd66013f0c0 2024-03-30T13:10:38.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.942+0000 7fd65e7fc640 1 -- 172.21.15.66:0/106286910 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492855045 0 0) 0x7fd648001110 con 0x7fd66006d720 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.943+0000 7fd65effd640 1 --2- 172.21.15.66:0/106286910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd66006d1e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6500097a0 tx=0x7fd65002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a9d63c21ba6141 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.943+0000 7fd65e7fc640 1 -- 172.21.15.66:0/106286910 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500315a0 con 0x7fd66013f0c0 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.943+0000 7fd65e7fc640 1 -- 172.21.15.66:0/106286910 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd650037070 con 0x7fd66013f0c0 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.943+0000 7fd65e7fc640 1 -- 172.21.15.66:0/106286910 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650031960 con 0x7fd66013f0c0 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 -- 172.21.15.66:0/106286910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 msgr2=0x7fd66006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 --2- 172.21.15.66:0/106286910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd66006d1e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6500097a0 tx=0x7fd65002b840 comp rx=0 tx=0).stop 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 -- 172.21.15.66:0/106286910 shutdown_connections 2024-03-30T13:10:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 --2- 172.21.15.66:0/106286910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd66006d1e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 -- 172.21.15.66:0/106286910 >> 172.21.15.66:0/106286910 conn(0x7fd6600b5b00 msgr2=0x7fd66006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:38.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 -- 172.21.15.66:0/106286910 shutdown_connections 2024-03-30T13:10:38.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.944+0000 7fd665704640 1 -- 172.21.15.66:0/106286910 wait complete. 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.945+0000 7fd665704640 1 Processor -- start 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.945+0000 7fd665704640 1 -- start start 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd665704640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66006d720 0x7fd66013e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd665704640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd660157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd665704640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6601605a0 0x7fd660071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd665704640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd66013f4a0 con 0x7fd66013f0c0 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd665704640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd660140000 con 0x7fd66006d720 2024-03-30T13:10:38.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd665704640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd66013f620 con 0x7fd6601605a0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd660157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd660157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43058/0 (socket says 172.21.15.66:43058) 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 -- 172.21.15.66:0/2930142354 learned_addr learned my addr 172.21.15.66:0/2930142354 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd65effd640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66006d720 0x7fd66013e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd65f7fe640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6601605a0 0x7fd660071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 -- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6601605a0 msgr2=0x7fd660071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6601605a0 0x7fd660071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 -- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66006d720 msgr2=0x7fd66013e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66006d720 0x7fd66013e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 -- 172.21.15.66:0/2930142354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66015a260 con 0x7fd66013f0c0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.946+0000 7fd657fff640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd660157020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c3f0 tx=0x7fd64800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.947+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6480074e0 con 0x7fd66013f0c0 2024-03-30T13:10:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.947+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6480047e0 con 0x7fd66013f0c0 2024-03-30T13:10:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.947+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004c10 con 0x7fd66013f0c0 2024-03-30T13:10:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.947+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd660142f60 con 0x7fd66013f0c0 2024-03-30T13:10:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.947+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd660143200 con 0x7fd66013f0c0 2024-03-30T13:10:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.948+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd660143400 con 0x7fd66013f0c0 2024-03-30T13:10:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.948+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd648011070 con 0x7fd66013f0c0 2024-03-30T13:10:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.948+0000 7fd6577fe640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd64006c530 0x7fd64006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.949+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fd64808b510 con 0x7fd66013f0c0 2024-03-30T13:10:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.949+0000 7fd65effd640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd64006c530 0x7fd64006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.949+0000 7fd65effd640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd64006c530 0x7fd64006e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd650036000 tx=0x7fd65003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:38.953+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd660143400 con 0x7fd66013f0c0 2024-03-30T13:10:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.137+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd660075890 con 0x7fd66013f0c0 2024-03-30T13:10:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.138+0000 7fd6577fe640 1 -- 172.21.15.66:0/2930142354 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fd6600ce9c0 con 0x7fd66013f0c0 2024-03-30T13:10:39.141 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:39.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd64006c530 msgr2=0x7fd64006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd64006c530 0x7fd64006e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd650036000 tx=0x7fd65003d040 comp rx=0 tx=0).stop 2024-03-30T13:10:39.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 msgr2=0x7fd660157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd660157020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c3f0 tx=0x7fd64800c8b0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 shutdown_connections 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd64006c530 0x7fd64006e9f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6601605a0 0x7fd660071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd66013f0c0 0x7fd660157020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 --2- 172.21.15.66:0/2930142354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66006d720 0x7fd66013e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.139+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 >> 172.21.15.66:0/2930142354 conn(0x7fd6600b5b00 msgr2=0x7fd66006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.140+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 shutdown_connections 2024-03-30T13:10:39.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.140+0000 7fd665704640 1 -- 172.21.15.66:0/2930142354 wait complete. 2024-03-30T13:10:39.154 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:39.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.260+0000 7f500c8c3640 1 Processor -- start 2024-03-30T13:10:39.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.260+0000 7f500c8c3640 1 -- start start 2024-03-30T13:10:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.261+0000 7f500c8c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f500815e570 0x7f50080b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.261+0000 7f500c8c3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f500815e950 con 0x7f50080cf7f0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.261+0000 7f500c8c3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50080b8890 con 0x7f50080ba0b0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.261+0000 7f500c8c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f50080b94a0 con 0x7f500815e570 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.261+0000 7f5006d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f50080cf7f0 0x7f50080d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50444/0 (socket says 172.21.15.66:50444) 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.261+0000 7f5006d9d640 1 -- 172.21.15.66:0/787759476 learned_addr learned my addr 172.21.15.66:0/787759476 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500659c640 1 --2- 172.21.15.66:0/787759476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f500815e570 0x7f50080b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545593868 0 0) 0x7f500815e950 con 0x7f50080cf7f0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ff0003520 con 0x7f50080cf7f0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401018791 0 0) 0x7f50080b8890 con 0x7f50080ba0b0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f500815e950 con 0x7f50080ba0b0 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500659c640 1 -- 172.21.15.66:0/787759476 >> v1:172.21.15.117:6789/0 conn(0x7f50080ba0b0 legacy=0x7f50080cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500659c640 1 -- 172.21.15.66:0/787759476 >> v1:172.21.15.66:6789/0 conn(0x7f50080cf7f0 legacy=0x7f50080d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.262+0000 7f500659c640 1 -- 172.21.15.66:0/787759476 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50080b8b70 con 0x7f500815e570 2024-03-30T13:10:39.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500659c640 1 --2- 172.21.15.66:0/787759476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f500815e570 0x7f50080b7f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80097d0 tx=0x7f4ff802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3728364d856a16c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8035020 con 0x7f500815e570 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff8031890 con 0x7f500815e570 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500559a640 1 -- 172.21.15.66:0/787759476 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8031c60 con 0x7f500815e570 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500c8c3640 1 -- 172.21.15.66:0/787759476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f500815e570 msgr2=0x7f50080b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500c8c3640 1 --2- 172.21.15.66:0/787759476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f500815e570 0x7f50080b7f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80097d0 tx=0x7f4ff802b930 comp rx=0 tx=0).stop 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500c8c3640 1 -- 172.21.15.66:0/787759476 shutdown_connections 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500c8c3640 1 --2- 172.21.15.66:0/787759476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f500815e570 0x7f50080b7f90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.263+0000 7f500c8c3640 1 -- 172.21.15.66:0/787759476 >> 172.21.15.66:0/787759476 conn(0x7f50080676e0 msgr2=0x7f5008067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.264+0000 7f500c8c3640 1 -- 172.21.15.66:0/787759476 shutdown_connections 2024-03-30T13:10:39.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.264+0000 7f500c8c3640 1 -- 172.21.15.66:0/787759476 wait complete. 2024-03-30T13:10:39.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.265+0000 7f500c8c3640 1 Processor -- start 2024-03-30T13:10:39.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.265+0000 7f500c8c3640 1 -- start start 2024-03-30T13:10:39.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500c8c3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50080ba0b0 0x7f500814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500c8c3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 0x7f500814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500c8c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500815e570 0x7f50080c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500c8c3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50080b84d0 con 0x7f50080cf7f0 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500c8c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50080b9130 con 0x7f500815e570 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500c8c3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f50080b8650 con 0x7f50080ba0b0 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 0x7f500814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50080ba0b0 0x7f500814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 0x7f500814e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43064/0 (socket says 172.21.15.66:43064) 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 -- 172.21.15.66:0/2451435512 learned_addr learned my addr 172.21.15.66:0/2451435512 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5006d9d640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500815e570 0x7f50080c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 -- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50080ba0b0 msgr2=0x7f500814de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50080ba0b0 0x7f500814de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 -- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500815e570 msgr2=0x7f50080c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500815e570 0x7f50080c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f5005d9b640 1 -- 172.21.15.66:0/2451435512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500814efe0 con 0x7f50080cf7f0 2024-03-30T13:10:39.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.266+0000 7f500659c640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50080ba0b0 0x7f500814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:39.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.267+0000 7f5005d9b640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 0x7f500814e540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c530 tx=0x7f4ffc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.269+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0052d0 con 0x7f50080cf7f0 2024-03-30T13:10:39.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.269+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ffc005490 con 0x7f50080cf7f0 2024-03-30T13:10:39.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.269+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500814f2c0 con 0x7f50080cf7f0 2024-03-30T13:10:39.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.269+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc012870 con 0x7f50080cf7f0 2024-03-30T13:10:39.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.269+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500814fcc0 con 0x7f50080cf7f0 2024-03-30T13:10:39.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.270+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5008075890 con 0x7f50080cf7f0 2024-03-30T13:10:39.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.271+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4ffc012a10 con 0x7f50080cf7f0 2024-03-30T13:10:39.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.271+0000 7f4ff6ffd640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4fe806c560 0x7f4fe806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.271+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f4ffc08b610 con 0x7f50080cf7f0 2024-03-30T13:10:39.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.271+0000 7f500659c640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4fe806c560 0x7f4fe806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.272+0000 7f500659c640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4fe806c560 0x7f4fe806ea20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8036000 tx=0x7f4ff803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.276+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4ffc0591d0 con 0x7f50080cf7f0 2024-03-30T13:10:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.461+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5008159a90 con 0x7f50080cf7f0 2024-03-30T13:10:39.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.462+0000 7f4ff6ffd640 1 -- 172.21.15.66:0/2451435512 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f4ffc05d4d0 con 0x7f50080cf7f0 2024-03-30T13:10:39.466 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4fe806c560 msgr2=0x7f4fe806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4fe806c560 0x7f4fe806ea20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8036000 tx=0x7f4ff803c040 comp rx=0 tx=0).stop 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 msgr2=0x7f500814e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 0x7f500814e540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c530 tx=0x7f4ffc00c9f0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 shutdown_connections 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4fe806c560 0x7f4fe806ea20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500815e570 0x7f50080c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50080cf7f0 0x7f500814e540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 --2- 172.21.15.66:0/2451435512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50080ba0b0 0x7f500814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 >> 172.21.15.66:0/2451435512 conn(0x7f50080676e0 msgr2=0x7f5008067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 shutdown_connections 2024-03-30T13:10:39.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.464+0000 7f500c8c3640 1 -- 172.21.15.66:0/2451435512 wait complete. 2024-03-30T13:10:39.479 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.585+0000 7f09ef5aa640 1 Processor -- start 2024-03-30T13:10:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.585+0000 7f09ef5aa640 1 -- start start 2024-03-30T13:10:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ef5aa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f09e8071020 0x7f09e806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ef5aa640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09e809aa30 0x7f09e809aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ef5aa640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80d1c10 con 0x7f09e8162120 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ef5aa640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f09e80d0770 con 0x7f09e809aa30 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ef5aa640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f09e815f180 con 0x7f09e8071020 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ecb45640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09e809aa30 0x7f09e809aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09edb47640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f09e8162120 0x7f09e81625b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50446/0 (socket says 172.21.15.66:50446) 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09edb47640 1 -- 172.21.15.66:0/3141727814 learned_addr learned my addr 172.21.15.66:0/3141727814 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.586+0000 7f09ed346640 1 --2- 172.21.15.66:0/3141727814 >> v2:172.21.15.117:3300/0 conn(0x7f09e8071020 0x7f09e806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09ed346640 1 -- 172.21.15.66:0/3141727814 >> v2:172.21.15.66:3300/0 conn(0x7f09e809aa30 msgr2=0x7f09e809aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09ed346640 1 --2- 172.21.15.66:0/3141727814 >> v2:172.21.15.66:3300/0 conn(0x7f09e809aa30 0x7f09e809aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09ed346640 1 -- 172.21.15.66:0/3141727814 >> v1:172.21.15.66:6789/0 conn(0x7f09e8162120 legacy=0x7f09e81625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09e7fff640 1 -- 172.21.15.66:0/3141727814 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127033512 0 0) 0x7f09e80d1c10 con 0x7f09e8162120 2024-03-30T13:10:39.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09ed346640 1 -- 172.21.15.66:0/3141727814 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e815f460 con 0x7f09e8071020 2024-03-30T13:10:39.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09ecb45640 1 --2- 172.21.15.66:0/3141727814 >> v2:172.21.15.66:3300/0 conn(0x7f09e809aa30 0x7f09e809aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:39.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.587+0000 7f09ed346640 1 --2- 172.21.15.66:0/3141727814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8071020 0x7f09e806c840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09e80682a0 tx=0x7f09d802bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c9d6ee835441c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09e7fff640 1 -- 172.21.15.66:0/3141727814 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8039070 con 0x7f09e8071020 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09e7fff640 1 -- 172.21.15.66:0/3141727814 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d802cc70 con 0x7f09e8071020 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09e7fff640 1 -- 172.21.15.66:0/3141727814 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8034660 con 0x7f09e8071020 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 -- 172.21.15.66:0/3141727814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8071020 msgr2=0x7f09e806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/3141727814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8071020 0x7f09e806c840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09e80682a0 tx=0x7f09d802bec0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 -- 172.21.15.66:0/3141727814 shutdown_connections 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/3141727814 >> v2:172.21.15.66:3300/0 conn(0x7f09e809aa30 0x7f09e809aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/3141727814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8071020 0x7f09e806c840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 -- 172.21.15.66:0/3141727814 >> 172.21.15.66:0/3141727814 conn(0x7f09e80b3ce0 msgr2=0x7f09e80b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:39.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 -- 172.21.15.66:0/3141727814 shutdown_connections 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.588+0000 7f09ef5aa640 1 -- 172.21.15.66:0/3141727814 wait complete. 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.589+0000 7f09ef5aa640 1 Processor -- start 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.589+0000 7f09ef5aa640 1 -- start start 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ef5aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 0x7f09e81f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ef5aa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e809aa30 0x7f09e81f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ef5aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8162120 0x7f09e806f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ef5aa640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8071400 con 0x7f09e8071020 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ef5aa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09e815fa20 con 0x7f09e8162120 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ef5aa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f09e806cd80 con 0x7f09e809aa30 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ed346640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 0x7f09e81f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ed346640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 0x7f09e81f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43074/0 (socket says 172.21.15.66:43074) 2024-03-30T13:10:39.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ed346640 1 -- 172.21.15.66:0/1792953174 learned_addr learned my addr 172.21.15.66:0/1792953174 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09ecb45640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e809aa30 0x7f09e81f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.590+0000 7f09edb47640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8162120 0x7f09e806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ed346640 1 -- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e809aa30 msgr2=0x7f09e81f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ed346640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e809aa30 0x7f09e81f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ed346640 1 -- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8162120 msgr2=0x7f09e806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ed346640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8162120 0x7f09e806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ed346640 1 -- 172.21.15.66:0/1792953174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e815afd0 con 0x7f09e8071020 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ed346640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 0x7f09e81f3b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09d8037040 tx=0x7f09d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8039050 con 0x7f09e8071020 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8045070 con 0x7f09e8071020 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d803e850 con 0x7f09e8071020 2024-03-30T13:10:39.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e815b2b0 con 0x7f09e8071020 2024-03-30T13:10:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.591+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e80b65f0 con 0x7f09e8071020 2024-03-30T13:10:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.592+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e8075890 con 0x7f09e8071020 2024-03-30T13:10:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.592+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f09d803e9f0 con 0x7f09e8071020 2024-03-30T13:10:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.593+0000 7f09e5ffb640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d406c530 0x7f09d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.593+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f09d80ae700 con 0x7f09e8071020 2024-03-30T13:10:39.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.593+0000 7f09ecb45640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d406c530 0x7f09d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.593+0000 7f09ecb45640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d406c530 0x7f09d406e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009560 tx=0x7f09dc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.597+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f09d802ce20 con 0x7f09e8071020 2024-03-30T13:10:39.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.783+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f09e80ce500 con 0x7f09e8071020 2024-03-30T13:10:39.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.784+0000 7f09e5ffb640 1 -- 172.21.15.66:0/1792953174 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f09d807c340 con 0x7f09e8071020 2024-03-30T13:10:39.787 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d406c530 msgr2=0x7f09d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d406c530 0x7f09d406e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009560 tx=0x7f09dc009190 comp rx=0 tx=0).stop 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 msgr2=0x7f09e81f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 0x7f09e81f3b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09d8037040 tx=0x7f09d8004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 shutdown_connections 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d406c530 0x7f09d406e9f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e8162120 0x7f09e806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09e809aa30 0x7f09e81f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 --2- 172.21.15.66:0/1792953174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e8071020 0x7f09e81f3b50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 >> 172.21.15.66:0/1792953174 conn(0x7f09e80b3ce0 msgr2=0x7f09e80b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 shutdown_connections 2024-03-30T13:10:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.786+0000 7f09ef5aa640 1 -- 172.21.15.66:0/1792953174 wait complete. 2024-03-30T13:10:39.801 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:10:39.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.906+0000 7f1fd3b6d640 1 Processor -- start 2024-03-30T13:10:39.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.906+0000 7f1fd3b6d640 1 -- start start 2024-03-30T13:10:39.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.906+0000 7f1fd3b6d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1fcc15e770 0x7f1fcc162310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd3b6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d0560 0x7f1fcc0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd3b6d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fcc1628f0 con 0x7f1fcc0d1f20 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd3b6d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f1fcc0b5200 con 0x7f1fcc15e770 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd3b6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc162bd0 con 0x7f1fcc0d0560 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd1108640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1fcc0d1f20 0x7f1fcc0cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50452/0 (socket says 172.21.15.66:50452) 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd1108640 1 -- 172.21.15.66:0/2319288023 learned_addr learned my addr 172.21.15.66:0/2319288023 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd1909640 1 --2- 172.21.15.66:0/2319288023 >> v2:172.21.15.117:3300/0 conn(0x7f1fcc15e770 0x7f1fcc162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd210a640 1 --2- 172.21.15.66:0/2319288023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d0560 0x7f1fcc0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd0907640 1 -- 172.21.15.66:0/2319288023 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049369620 0 0) 0x7f1fcc1628f0 con 0x7f1fcc0d1f20 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd0907640 1 -- 172.21.15.66:0/2319288023 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fb8003520 con 0x7f1fcc0d1f20 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd210a640 1 -- 172.21.15.66:0/2319288023 >> v2:172.21.15.117:3300/0 conn(0x7f1fcc15e770 msgr2=0x7f1fcc162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd210a640 1 --2- 172.21.15.66:0/2319288023 >> v2:172.21.15.117:3300/0 conn(0x7f1fcc15e770 0x7f1fcc162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd210a640 1 -- 172.21.15.66:0/2319288023 >> v1:172.21.15.66:6789/0 conn(0x7f1fcc0d1f20 legacy=0x7f1fcc0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd210a640 1 -- 172.21.15.66:0/2319288023 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc162eb0 con 0x7f1fcc0d0560 2024-03-30T13:10:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.907+0000 7f1fd1909640 1 --2- 172.21.15.66:0/2319288023 >> v2:172.21.15.117:3300/0 conn(0x7f1fcc15e770 0x7f1fcc162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.908+0000 7f1fd210a640 1 --2- 172.21.15.66:0/2319288023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d0560 0x7f1fcc0b5690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009d90 tx=0x7f1fc402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c933e01634ad673b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.908+0000 7f1fd0907640 1 -- 172.21.15.66:0/2319288023 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4039070 con 0x7f1fcc0d0560 2024-03-30T13:10:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.908+0000 7f1fd0907640 1 -- 172.21.15.66:0/2319288023 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc402cc50 con 0x7f1fcc0d0560 2024-03-30T13:10:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.908+0000 7f1fd0907640 1 -- 172.21.15.66:0/2319288023 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4034440 con 0x7f1fcc0d0560 2024-03-30T13:10:39.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/2319288023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d0560 msgr2=0x7f1fcc0b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/2319288023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d0560 0x7f1fcc0b5690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009d90 tx=0x7f1fc402bea0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/2319288023 shutdown_connections 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/2319288023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d0560 0x7f1fcc0b5690 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/2319288023 >> v2:172.21.15.117:3300/0 conn(0x7f1fcc15e770 0x7f1fcc162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/2319288023 >> 172.21.15.66:0/2319288023 conn(0x7f1fcc06f8f0 msgr2=0x7f1fcc06fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/2319288023 shutdown_connections 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.909+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/2319288023 wait complete. 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.910+0000 7f1fd3b6d640 1 Processor -- start 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.910+0000 7f1fd3b6d640 1 -- start start 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 0x7f1fcc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d1f20 0x7f1fcc0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc15e770 0x7f1fcc146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc06c790 con 0x7f1fcc0d0560 2024-03-30T13:10:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc15eb50 con 0x7f1fcc15e770 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc06c910 con 0x7f1fcc0d1f20 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 0x7f1fcc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 0x7f1fcc0cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43082/0 (socket says 172.21.15.66:43082) 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 -- 172.21.15.66:0/1272941075 learned_addr learned my addr 172.21.15.66:0/1272941075 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1108640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d1f20 0x7f1fcc0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd210a640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc15e770 0x7f1fcc146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 -- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d1f20 msgr2=0x7f1fcc0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d1f20 0x7f1fcc0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 -- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc15e770 msgr2=0x7f1fcc146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc15e770 0x7f1fcc146cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 -- 172.21.15.66:0/1272941075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc0c7910 con 0x7f1fcc0d0560 2024-03-30T13:10:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd210a640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc15e770 0x7f1fcc146cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd1909640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 0x7f1fcc0cbaf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ebe0 tx=0x7f1fc800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8017070 con 0x7f1fcc0d0560 2024-03-30T13:10:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc800cc30 con 0x7f1fcc0d0560 2024-03-30T13:10:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.911+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc0c7bf0 con 0x7f1fcc0d0560 2024-03-30T13:10:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.912+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8013470 con 0x7f1fcc0d0560 2024-03-30T13:10:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.912+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc0c81b0 con 0x7f1fcc0d0560 2024-03-30T13:10:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.912+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc0cc0f0 con 0x7f1fcc0d0560 2024-03-30T13:10:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.913+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1fc801a070 con 0x7f1fcc0d0560 2024-03-30T13:10:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.913+0000 7f1fc27fc640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1fb006c530 0x7f1fb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.913+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f1fc808bd40 con 0x7f1fcc0d0560 2024-03-30T13:10:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.913+0000 7f1fd1108640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1fb006c530 0x7f1fb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.914+0000 7f1fd1108640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1fb006c530 0x7f1fb006e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc15b6e0 tx=0x7f1fbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:39.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:39.918+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1fc80598f0 con 0x7f1fcc0d0560 2024-03-30T13:10:40.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.114+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1fcc0b3b40 con 0x7f1fcc0d0560 2024-03-30T13:10:40.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.114+0000 7f1fc27fc640 1 -- 172.21.15.66:0/1272941075 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f1fcc0b3b40 con 0x7f1fcc0d0560 2024-03-30T13:10:40.118 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:10:40.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1fb006c530 msgr2=0x7f1fb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1fb006c530 0x7f1fb006e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc15b6e0 tx=0x7f1fbc009040 comp rx=0 tx=0).stop 2024-03-30T13:10:40.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 msgr2=0x7f1fcc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 0x7f1fcc0cbaf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ebe0 tx=0x7f1fc800c520 comp rx=0 tx=0).stop 2024-03-30T13:10:40.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 shutdown_connections 2024-03-30T13:10:40.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1fb006c530 0x7f1fb006e9f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc15e770 0x7f1fcc146cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0d1f20 0x7f1fcc0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 --2- 172.21.15.66:0/1272941075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc0d0560 0x7f1fcc0cbaf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 >> 172.21.15.66:0/1272941075 conn(0x7f1fcc06f8f0 msgr2=0x7f1fcc1615e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:40.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 shutdown_connections 2024-03-30T13:10:40.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.116+0000 7f1fd3b6d640 1 -- 172.21.15.66:0/1272941075 wait complete. 2024-03-30T13:10:40.131 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-30T13:10:40.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:40.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-30T13:10:40.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:10:40.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:40.164 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T13:10:40.219 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:40.219 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:40.236 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:40.237 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:40.293 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:10:40.293 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:10:40.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:40.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.451+0000 7face3950640 1 Processor -- start 2024-03-30T13:10:40.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.451+0000 7face3950640 1 -- start start 2024-03-30T13:10:40.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face3950640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7facdc0d0f10 0x7facdc152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face3950640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0b4430 0x7facdc153300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face3950640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0cfe70 con 0x7facdc0cccf0 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face3950640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7facdc0d0150 con 0x7facdc0d0f10 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face3950640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0d0430 con 0x7facdc0b4430 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face0eeb640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7facdc0d0f10 0x7facdc152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face0eeb640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7facdc0d0f10 0x7facdc152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43084/0 (socket says 172.21.15.66:43084) 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face0eeb640 1 -- 172.21.15.66:0/1212532461 learned_addr learned my addr 172.21.15.66:0/1212532461 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.452+0000 7face1eed640 1 --2- 172.21.15.66:0/1212532461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0b4430 0x7facdc153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.453+0000 7face0eeb640 1 -- 172.21.15.66:0/1212532461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0b4430 msgr2=0x7facdc153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.453+0000 7face0eeb640 1 --2- 172.21.15.66:0/1212532461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0b4430 0x7facdc153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.453+0000 7face0eeb640 1 -- 172.21.15.66:0/1212532461 >> v1:172.21.15.117:6789/0 conn(0x7facdc0cccf0 legacy=0x7facdc0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.453+0000 7facd3fff640 1 -- 172.21.15.66:0/1212532461 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977742578 0 0) 0x7facdc0cfe70 con 0x7facdc0cccf0 2024-03-30T13:10:40.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.453+0000 7face0eeb640 1 -- 172.21.15.66:0/1212532461 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0d0710 con 0x7facdc0d0f10 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.453+0000 7face0eeb640 1 --2- 172.21.15.66:0/1212532461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0d0f10 0x7facdc152dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faccc009870 tx=0x7faccc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66d83afe60e3b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7facd3fff640 1 -- 172.21.15.66:0/1212532461 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc034070 con 0x7facdc0d0f10 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7facd3fff640 1 -- 172.21.15.66:0/1212532461 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc030840 con 0x7facdc0d0f10 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7facd3fff640 1 -- 172.21.15.66:0/1212532461 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc030c10 con 0x7facdc0d0f10 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 -- 172.21.15.66:0/1212532461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0d0f10 msgr2=0x7facdc152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 --2- 172.21.15.66:0/1212532461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0d0f10 0x7facdc152dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faccc009870 tx=0x7faccc02b980 comp rx=0 tx=0).stop 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 -- 172.21.15.66:0/1212532461 shutdown_connections 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 --2- 172.21.15.66:0/1212532461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0b4430 0x7facdc153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 --2- 172.21.15.66:0/1212532461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0d0f10 0x7facdc152dc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 -- 172.21.15.66:0/1212532461 >> 172.21.15.66:0/1212532461 conn(0x7facdc0672c0 msgr2=0x7facdc0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:40.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 -- 172.21.15.66:0/1212532461 shutdown_connections 2024-03-30T13:10:40.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.454+0000 7face3950640 1 -- 172.21.15.66:0/1212532461 wait complete. 2024-03-30T13:10:40.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.455+0000 7face3950640 1 Processor -- start 2024-03-30T13:10:40.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.455+0000 7face3950640 1 -- start start 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face3950640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 0x7facdc1521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face3950640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0cccf0 0x7facdc0c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face3950640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0d0f10 0x7facdc0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face3950640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0cd0d0 con 0x7facdc0b4430 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face3950640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facdc153840 con 0x7facdc0cccf0 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face3950640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0cd250 con 0x7facdc0d0f10 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face16ec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 0x7facdc1521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face16ec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 0x7facdc1521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43090/0 (socket says 172.21.15.66:43090) 2024-03-30T13:10:40.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face16ec640 1 -- 172.21.15.66:0/665265402 learned_addr learned my addr 172.21.15.66:0/665265402 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face0eeb640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0cccf0 0x7facdc0c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.456+0000 7face1eed640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0d0f10 0x7facdc0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face16ec640 1 -- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0d0f10 msgr2=0x7facdc0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face16ec640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0d0f10 0x7facdc0ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face16ec640 1 -- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0cccf0 msgr2=0x7facdc0c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face16ec640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0cccf0 0x7facdc0c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face16ec640 1 -- 172.21.15.66:0/665265402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc1526e0 con 0x7facdc0b4430 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face1eed640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0d0f10 0x7facdc0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face16ec640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 0x7facdc1521a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7facd800c4b0 tx=0x7facd800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8017070 con 0x7facdc0b4430 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8005480 con 0x7facdc0b4430 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face3950640 1 -- 172.21.15.66:0/665265402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc0cb670 con 0x7facdc0b4430 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8003830 con 0x7facdc0b4430 2024-03-30T13:10:40.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.457+0000 7face3950640 1 -- 172.21.15.66:0/665265402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc0cbc30 con 0x7facdc0b4430 2024-03-30T13:10:40.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.458+0000 7face3950640 1 -- 172.21.15.66:0/665265402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc075a10 con 0x7facdc0b4430 2024-03-30T13:10:40.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.458+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7facd801a070 con 0x7facdc0b4430 2024-03-30T13:10:40.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.459+0000 7facd1ffb640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7facc006c530 0x7facc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.459+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7facd808adf0 con 0x7facdc0b4430 2024-03-30T13:10:40.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.459+0000 7face0eeb640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7facc006c530 0x7facc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.459+0000 7face0eeb640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7facc006c530 0x7facc006e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faccc036000 tx=0x7faccc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:40.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.463+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7facdc1526e0 con 0x7facdc0b4430 2024-03-30T13:10:40.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.650+0000 7face3950640 1 -- 172.21.15.66:0/665265402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facdc15fdc0 con 0x7facdc0b4430 2024-03-30T13:10:40.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.651+0000 7facd1ffb640 1 -- 172.21.15.66:0/665265402 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7facd8058a30 con 0x7facdc0b4430 2024-03-30T13:10:40.655 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:40.655 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 -- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7facc006c530 msgr2=0x7facc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7facc006c530 0x7facc006e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faccc036000 tx=0x7faccc03c040 comp rx=0 tx=0).stop 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 -- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 msgr2=0x7facdc1521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 0x7facdc1521a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7facd800c4b0 tx=0x7facd800c970 comp rx=0 tx=0).stop 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 -- 172.21.15.66:0/665265402 shutdown_connections 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7facc006c530 0x7facc006e9f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc0d0f10 0x7facdc0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0cccf0 0x7facdc0c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 --2- 172.21.15.66:0/665265402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facdc0b4430 0x7facdc1521a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 -- 172.21.15.66:0/665265402 >> 172.21.15.66:0/665265402 conn(0x7facdc0672c0 msgr2=0x7facdc0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:40.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 -- 172.21.15.66:0/665265402 shutdown_connections 2024-03-30T13:10:40.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.653+0000 7face3950640 1 -- 172.21.15.66:0/665265402 wait complete. 2024-03-30T13:10:40.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:40.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:40.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.775+0000 7fa383f4d640 1 Processor -- start 2024-03-30T13:10:40.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.775+0000 7fa383f4d640 1 -- start start 2024-03-30T13:10:40.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa383f4d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa37c071020 0x7fa37c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa383f4d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c09aa30 0x7fa37c09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa383f4d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa37c0d1c10 con 0x7fa37c162120 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa383f4d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa37c0d0770 con 0x7fa37c071020 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa383f4d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c15f180 con 0x7fa37c09aa30 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa381ce9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa37c071020 0x7fa37c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa3824ea640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa37c162120 0x7fa37c1625b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50460/0 (socket says 172.21.15.66:50460) 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa3824ea640 1 -- 172.21.15.66:0/3654515034 learned_addr learned my addr 172.21.15.66:0/3654515034 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.776+0000 7fa3814e8640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c09aa30 0x7fa37c09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa381ce9640 1 -- 172.21.15.66:0/3654515034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c09aa30 msgr2=0x7fa37c09aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa381ce9640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c09aa30 0x7fa37c09aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa381ce9640 1 -- 172.21.15.66:0/3654515034 >> v1:172.21.15.66:6789/0 conn(0x7fa37c162120 legacy=0x7fa37c1625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa381ce9640 1 -- 172.21.15.66:0/3654515034 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c15f460 con 0x7fa37c071020 2024-03-30T13:10:40.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa3814e8640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c09aa30 0x7fa37c09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa381ce9640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c06c840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178231fdeb37c730 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa380ce7640 1 -- 172.21.15.66:0/3654515034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c035020 con 0x7fa37c071020 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa380ce7640 1 -- 172.21.15.66:0/3654515034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36c031810 con 0x7fa37c071020 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.777+0000 7fa380ce7640 1 -- 172.21.15.66:0/3654515034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c031be0 con 0x7fa37c071020 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 -- 172.21.15.66:0/3654515034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 msgr2=0x7fa37c06c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c06c840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c02b8b0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 -- 172.21.15.66:0/3654515034 shutdown_connections 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c09aa30 0x7fa37c09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 --2- 172.21.15.66:0/3654515034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c06c840 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 -- 172.21.15.66:0/3654515034 >> 172.21.15.66:0/3654515034 conn(0x7fa37c0b3ce0 msgr2=0x7fa37c0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:40.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 -- 172.21.15.66:0/3654515034 shutdown_connections 2024-03-30T13:10:40.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.778+0000 7fa383f4d640 1 -- 172.21.15.66:0/3654515034 wait complete. 2024-03-30T13:10:40.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.779+0000 7fa383f4d640 1 Processor -- start 2024-03-30T13:10:40.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.779+0000 7fa383f4d640 1 -- start start 2024-03-30T13:10:40.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa383f4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c1e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa383f4d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37c09aa30 0x7fa37c1f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa383f4d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c162120 0x7fa37c0b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa383f4d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c071400 con 0x7fa37c071020 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa383f4d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c15fa20 con 0x7fa37c09aa30 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa383f4d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c06cd80 con 0x7fa37c162120 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c1e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c1e85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43104/0 (socket says 172.21.15.66:43104) 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 -- 172.21.15.66:0/2463417065 learned_addr learned my addr 172.21.15.66:0/2463417065 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa3824ea640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c162120 0x7fa37c0b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa3814e8640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37c09aa30 0x7fa37c1f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 -- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c162120 msgr2=0x7fa37c0b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c162120 0x7fa37c0b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 -- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37c09aa30 msgr2=0x7fa37c1f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37c09aa30 0x7fa37c1f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.780+0000 7fa381ce9640 1 -- 172.21.15.66:0/2463417065 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c1e8b30 con 0x7fa37c071020 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.781+0000 7fa381ce9640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c1e85f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:40.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.781+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c006e10 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.781+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36c03ad60 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.781+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c1e7550 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.781+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c044630 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.781+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c1e7b10 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.782+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c075890 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.782+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa36c04b060 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.783+0000 7fa37a7fc640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa36806c560 0x7fa36806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.783+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fa36c0ad770 con 0x7fa37c071020 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.783+0000 7fa3814e8640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa36806c560 0x7fa36806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:40.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.783+0000 7fa3814e8640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa36806c560 0x7fa36806ea20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa370009560 tx=0x7fa370009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:40.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.787+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa36c03f070 con 0x7fa37c071020 2024-03-30T13:10:40.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.974+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa37c09b6a0 con 0x7fa37c071020 2024-03-30T13:10:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.974+0000 7fa37a7fc640 1 -- 172.21.15.66:0/2463417065 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fa36c07b320 con 0x7fa37c071020 2024-03-30T13:10:40.979 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa36806c560 msgr2=0x7fa36806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa36806c560 0x7fa36806ea20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa370009560 tx=0x7fa370009190 comp rx=0 tx=0).stop 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 msgr2=0x7fa37c1e85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c1e85f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 shutdown_connections 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa36806c560 0x7fa36806ea20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa37c162120 0x7fa37c0b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37c09aa30 0x7fa37c1f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 --2- 172.21.15.66:0/2463417065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa37c071020 0x7fa37c1e85f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 >> 172.21.15.66:0/2463417065 conn(0x7fa37c0b3ce0 msgr2=0x7fa37c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 shutdown_connections 2024-03-30T13:10:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:40.977+0000 7fa383f4d640 1 -- 172.21.15.66:0/2463417065 wait complete. 2024-03-30T13:10:40.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:41.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.098+0000 7efe2bc4a640 1 Processor -- start 2024-03-30T13:10:41.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.098+0000 7efe2bc4a640 1 -- start start 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2bc4a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe2409aee0 0x7efe240d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2bc4a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe2409b370 con 0x7efe24071020 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2bc4a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe2415f180 con 0x7efe2409aa30 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2bc4a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7efe240d0770 con 0x7efe2409aee0 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe299e6640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efe24071020 0x7efe2406c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50466/0 (socket says 172.21.15.66:50466) 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe291e5640 1 --1- 172.21.15.66:0/1475728891 >> v1:172.21.15.117:6789/0 conn(0x7efe2409aa30 0x7efe240d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:45742/0 (socket says 172.21.15.66:45742) 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2a1e7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe2409aee0 0x7efe240d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe299e6640 1 -- 172.21.15.66:0/1475728891 learned_addr learned my addr 172.21.15.66:0/1475728891 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:41.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe289e4640 1 -- 172.21.15.66:0/1475728891 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941214463 0 0) 0x7efe2409b370 con 0x7efe24071020 2024-03-30T13:10:41.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2a1e7640 1 -- 172.21.15.66:0/1475728891 >> v1:172.21.15.117:6789/0 conn(0x7efe2409aa30 legacy=0x7efe240d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2a1e7640 1 -- 172.21.15.66:0/1475728891 >> v1:172.21.15.66:6789/0 conn(0x7efe24071020 legacy=0x7efe2406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.099+0000 7efe2a1e7640 1 -- 172.21.15.66:0/1475728891 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2415f460 con 0x7efe2409aee0 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.100+0000 7efe2a1e7640 1 --2- 172.21.15.66:0/1475728891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe240d1310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009720 tx=0x7efe1c02b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b4eeedbc93f470 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.100+0000 7efe289e4640 1 -- 172.21.15.66:0/1475728891 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c037070 con 0x7efe2409aee0 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.100+0000 7efe289e4640 1 -- 172.21.15.66:0/1475728891 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c030710 con 0x7efe2409aee0 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.100+0000 7efe289e4640 1 -- 172.21.15.66:0/1475728891 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c030ae0 con 0x7efe2409aee0 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1475728891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 msgr2=0x7efe240d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1475728891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe240d1310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009720 tx=0x7efe1c02b730 comp rx=0 tx=0).stop 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1475728891 shutdown_connections 2024-03-30T13:10:41.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1475728891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe240d1310 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1475728891 >> 172.21.15.66:0/1475728891 conn(0x7efe240b3ce0 msgr2=0x7efe240b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1475728891 shutdown_connections 2024-03-30T13:10:41.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.101+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1475728891 wait complete. 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.102+0000 7efe2bc4a640 1 Processor -- start 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.102+0000 7efe2bc4a640 1 -- start start 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2bc4a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 0x7efe2414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2bc4a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2409aa30 0x7efe240b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2bc4a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe2414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2bc4a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efe2415ee70 con 0x7efe24071020 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2bc4a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe2415fa20 con 0x7efe2409aee0 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2bc4a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efe24071400 con 0x7efe2409aa30 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe299e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 0x7efe2414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe299e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 0x7efe2414c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43106/0 (socket says 172.21.15.66:43106) 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe299e6640 1 -- 172.21.15.66:0/1814442289 learned_addr learned my addr 172.21.15.66:0/1814442289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe291e5640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2409aa30 0x7efe240b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.103+0000 7efe2a1e7640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe2414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe299e6640 1 -- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2409aa30 msgr2=0x7efe240b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe299e6640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2409aa30 0x7efe240b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe299e6640 1 -- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 msgr2=0x7efe2414ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe299e6640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe2414ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe299e6640 1 -- 172.21.15.66:0/1814442289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe240c71b0 con 0x7efe24071020 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe299e6640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 0x7efe2414c730 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c780 tx=0x7efe1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14018070 con 0x7efe24071020 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1400a640 con 0x7efe24071020 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe240c7490 con 0x7efe24071020 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1400aab0 con 0x7efe24071020 2024-03-30T13:10:41.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.104+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe240c7a50 con 0x7efe24071020 2024-03-30T13:10:41.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.105+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe24075890 con 0x7efe24071020 2024-03-30T13:10:41.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.106+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efe1401b070 con 0x7efe24071020 2024-03-30T13:10:41.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.106+0000 7efe227fc640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efe1006c530 0x7efe1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.106+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7efe1408b090 con 0x7efe24071020 2024-03-30T13:10:41.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.106+0000 7efe291e5640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efe1006c530 0x7efe1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.106+0000 7efe291e5640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efe1006c530 0x7efe1006e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe24070770 tx=0x7efe18007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.110+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efe14058c40 con 0x7efe24071020 2024-03-30T13:10:41.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.295+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe240d16b0 con 0x7efe24071020 2024-03-30T13:10:41.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.295+0000 7efe227fc640 1 -- 172.21.15.66:0/1814442289 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7efe1405cf40 con 0x7efe24071020 2024-03-30T13:10:41.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:41.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efe1006c530 msgr2=0x7efe1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efe1006c530 0x7efe1006e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe24070770 tx=0x7efe18007de0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 msgr2=0x7efe2414c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 0x7efe2414c730 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c780 tx=0x7efe1400cc40 comp rx=0 tx=0).stop 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 shutdown_connections 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efe1006c530 0x7efe1006e9f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe2409aee0 0x7efe2414ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe2409aa30 0x7efe240b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 --2- 172.21.15.66:0/1814442289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe24071020 0x7efe2414c730 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 >> 172.21.15.66:0/1814442289 conn(0x7efe240b3ce0 msgr2=0x7efe240d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 shutdown_connections 2024-03-30T13:10:41.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.297+0000 7efe2bc4a640 1 -- 172.21.15.66:0/1814442289 wait complete. 2024-03-30T13:10:41.312 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:41.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.418+0000 7fcf049fe640 1 Processor -- start 2024-03-30T13:10:41.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcf049fe640 1 -- start start 2024-03-30T13:10:41.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcf049fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf000d0b90 0x7fcf000d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcf049fe640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf000b54b0 con 0x7fcf000b4920 2024-03-30T13:10:41.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcf049fe640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf000cfca0 con 0x7fcf000b4d00 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcf049fe640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcf000d0260 con 0x7fcf000d0b90 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fceff59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf000d0b90 0x7fcf000d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcefed9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcf000b4920 0x7fcf000b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:50474/0 (socket says 172.21.15.66:50474) 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.419+0000 7fcefed9d640 1 -- 172.21.15.66:0/818777425 learned_addr learned my addr 172.21.15.66:0/818777425 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.420+0000 7fceff59e640 1 -- 172.21.15.66:0/818777425 >> v1:172.21.15.117:6789/0 conn(0x7fcf000b4d00 legacy=0x7fcf000d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.420+0000 7fceff59e640 1 -- 172.21.15.66:0/818777425 >> v1:172.21.15.66:6789/0 conn(0x7fcf000b4920 legacy=0x7fcf000b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.420+0000 7fcefe59c640 1 -- 172.21.15.66:0/818777425 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988575841 0 0) 0x7fcf000cfca0 con 0x7fcf000b4d00 2024-03-30T13:10:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.420+0000 7fceff59e640 1 -- 172.21.15.66:0/818777425 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf000cff80 con 0x7fcf000d0b90 2024-03-30T13:10:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fceff59e640 1 --2- 172.21.15.66:0/818777425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf000d1040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef802bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76549b607ff90d04 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcefe59c640 1 -- 172.21.15.66:0/818777425 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8039070 con 0x7fcf000d0b90 2024-03-30T13:10:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcefe59c640 1 -- 172.21.15.66:0/818777425 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef802ca20 con 0x7fcf000d0b90 2024-03-30T13:10:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcefe59c640 1 -- 172.21.15.66:0/818777425 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef802cdf0 con 0x7fcf000d0b90 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 -- 172.21.15.66:0/818777425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 msgr2=0x7fcf000d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 --2- 172.21.15.66:0/818777425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf000d1040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef802bcf0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 -- 172.21.15.66:0/818777425 shutdown_connections 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 --2- 172.21.15.66:0/818777425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf000d1040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 -- 172.21.15.66:0/818777425 >> 172.21.15.66:0/818777425 conn(0x7fcf0009d4e0 msgr2=0x7fcf000d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 -- 172.21.15.66:0/818777425 shutdown_connections 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.421+0000 7fcf049fe640 1 -- 172.21.15.66:0/818777425 wait complete. 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 Processor -- start 2024-03-30T13:10:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 -- start start 2024-03-30T13:10:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf000b4920 0x7fcf000b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf000b4d00 0x7fcf001f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf001f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf000cf8c0 con 0x7fcf000d0b90 2024-03-30T13:10:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf000d2180 con 0x7fcf000b4d00 2024-03-30T13:10:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.423+0000 7fcf049fe640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcf000cfa40 con 0x7fcf000b4920 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf001f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf001f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43118/0 (socket says 172.21.15.66:43118) 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 -- 172.21.15.66:0/954921865 learned_addr learned my addr 172.21.15.66:0/954921865 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcef7fff640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf000b4d00 0x7fcf001f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcefed9d640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf000b4920 0x7fcf000b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 -- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf000b4920 msgr2=0x7fcf000b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf000b4920 0x7fcf000b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 -- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf000b4d00 msgr2=0x7fcf001f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf000b4d00 0x7fcf001f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 -- 172.21.15.66:0/954921865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf000b9fa0 con 0x7fcf000d0b90 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fceff59e640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf001f8030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcef8037040 tx=0x7fcef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8044020 con 0x7fcf000d0b90 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef803a040 con 0x7fcf000d0b90 2024-03-30T13:10:41.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8039050 con 0x7fcf000d0b90 2024-03-30T13:10:41.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf000cd060 con 0x7fcf000d0b90 2024-03-30T13:10:41.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.424+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf000cd620 con 0x7fcf000d0b90 2024-03-30T13:10:41.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.425+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf00075890 con 0x7fcf000d0b90 2024-03-30T13:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.426+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcef8003bd0 con 0x7fcf000d0b90 2024-03-30T13:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.426+0000 7fcef77fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcee006c530 0x7fcee006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.426+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fcef80adc20 con 0x7fcf000d0b90 2024-03-30T13:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.426+0000 7fcefed9d640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcee006c530 0x7fcee006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.427+0000 7fcefed9d640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcee006c530 0x7fcee006e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcef0000f60 tx=0x7fcef0007e20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.431+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcef804e060 con 0x7fcf000d0b90 2024-03-30T13:10:41.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.613+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf000d10c0 con 0x7fcf000d0b90 2024-03-30T13:10:41.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.614+0000 7fcef77fe640 1 -- 172.21.15.66:0/954921865 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fcf000b9fa0 con 0x7fcf000d0b90 2024-03-30T13:10:41.617 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:41.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.615+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcee006c530 msgr2=0x7fcee006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.615+0000 7fcf049fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcee006c530 0x7fcee006e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcef0000f60 tx=0x7fcef0007e20 comp rx=0 tx=0).stop 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.615+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 msgr2=0x7fcf001f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.615+0000 7fcf049fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf001f8030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcef8037040 tx=0x7fcef8004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 shutdown_connections 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcee006c530 0x7fcee006e9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf000d0b90 0x7fcf001f8030 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf000b4d00 0x7fcf001f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 --2- 172.21.15.66:0/954921865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf000b4920 0x7fcf000b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 >> 172.21.15.66:0/954921865 conn(0x7fcf0009d4e0 msgr2=0x7fcf000749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 shutdown_connections 2024-03-30T13:10:41.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.616+0000 7fcf049fe640 1 -- 172.21.15.66:0/954921865 wait complete. 2024-03-30T13:10:41.630 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:41.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 Processor -- start 2024-03-30T13:10:41.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 -- start start 2024-03-30T13:10:41.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb55409aa30 0x7fb55409aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb55415f180 con 0x7fb554071020 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb5540d0770 con 0x7fb55409aa30 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb55a3e8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb55415f460 con 0x7fb5540d0c00 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.737+0000 7fb5537fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb55409aa30 0x7fb55409aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb5537fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb55409aa30 0x7fb55409aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43126/0 (socket says 172.21.15.66:43126) 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb5537fe640 1 -- 172.21.15.66:0/4233027257 learned_addr learned my addr 172.21.15.66:0/4233027257 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb558985640 1 --2- 172.21.15.66:0/4233027257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb5537fe640 1 -- 172.21.15.66:0/4233027257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 msgr2=0x7fb5540d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb5537fe640 1 --2- 172.21.15.66:0/4233027257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb5537fe640 1 -- 172.21.15.66:0/4233027257 >> v1:172.21.15.117:6789/0 conn(0x7fb554071020 legacy=0x7fb55406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb552ffd640 1 -- 172.21.15.66:0/4233027257 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004944218 0 0) 0x7fb55415f180 con 0x7fb554071020 2024-03-30T13:10:41.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.738+0000 7fb5537fe640 1 -- 172.21.15.66:0/4233027257 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55415f740 con 0x7fb55409aa30 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb5537fe640 1 --2- 172.21.15.66:0/4233027257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb55409aee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb548009870 tx=0x7fb54802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc89775d77f8610 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb552ffd640 1 -- 172.21.15.66:0/4233027257 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548034070 con 0x7fb55409aa30 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb552ffd640 1 -- 172.21.15.66:0/4233027257 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb548030840 con 0x7fb55409aa30 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb552ffd640 1 -- 172.21.15.66:0/4233027257 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548030c10 con 0x7fb55409aa30 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 -- 172.21.15.66:0/4233027257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 msgr2=0x7fb55409aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/4233027257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb55409aee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb548009870 tx=0x7fb54802b980 comp rx=0 tx=0).stop 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 -- 172.21.15.66:0/4233027257 shutdown_connections 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/4233027257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/4233027257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb55409aee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 -- 172.21.15.66:0/4233027257 >> 172.21.15.66:0/4233027257 conn(0x7fb5540b3ce0 msgr2=0x7fb5540b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 -- 172.21.15.66:0/4233027257 shutdown_connections 2024-03-30T13:10:41.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.739+0000 7fb55a3e8640 1 -- 172.21.15.66:0/4233027257 wait complete. 2024-03-30T13:10:41.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 Processor -- start 2024-03-30T13:10:41.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 -- start start 2024-03-30T13:10:41.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb554071020 0x7fb554157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb5540b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb55406f620 con 0x7fb55409aa30 2024-03-30T13:10:41.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb554071400 con 0x7fb554071020 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.741+0000 7fb55a3e8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb5540d15d0 con 0x7fb5540d0c00 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb5540b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb5540b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43142/0 (socket says 172.21.15.66:43142) 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb553fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb554071020 0x7fb554157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 -- 172.21.15.66:0/2354942307 learned_addr learned my addr 172.21.15.66:0/2354942307 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb558985640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 -- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 msgr2=0x7fb5540b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 -- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb554071020 msgr2=0x7fb554157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb554071020 0x7fb554157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 -- 172.21.15.66:0/2354942307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5541583e0 con 0x7fb55409aa30 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb5537fe640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb5540b4c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb548009870 tx=0x7fb5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548034040 con 0x7fb55409aa30 2024-03-30T13:10:41.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb548030640 con 0x7fb55409aa30 2024-03-30T13:10:41.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb554157560 con 0x7fb55409aa30 2024-03-30T13:10:41.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548039d40 con 0x7fb55409aa30 2024-03-30T13:10:41.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.742+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5540c6050 con 0x7fb55409aa30 2024-03-30T13:10:41.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.743+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb554075890 con 0x7fb55409aa30 2024-03-30T13:10:41.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.744+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb54804b060 con 0x7fb55409aa30 2024-03-30T13:10:41.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.744+0000 7fb550ff9640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb54006c530 0x7fb54006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.745+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fb5480ad650 con 0x7fb55409aa30 2024-03-30T13:10:41.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.745+0000 7fb553fff640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb54006c530 0x7fb54006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.745+0000 7fb553fff640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb54006c530 0x7fb54006e9f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb544000f60 tx=0x7fb544009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.749+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb554157560 con 0x7fb55409aa30 2024-03-30T13:10:41.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.934+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb554157960 con 0x7fb55409aa30 2024-03-30T13:10:41.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.935+0000 7fb550ff9640 1 -- 172.21.15.66:0/2354942307 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fb5541583e0 con 0x7fb55409aa30 2024-03-30T13:10:41.939 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb54006c530 msgr2=0x7fb54006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb54006c530 0x7fb54006e9f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb544000f60 tx=0x7fb544009040 comp rx=0 tx=0).stop 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 msgr2=0x7fb5540b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb5540b4c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb548009870 tx=0x7fb5480046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 shutdown_connections 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb54006c530 0x7fb54006e9f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5540d0c00 0x7fb5540b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb55409aa30 0x7fb5540b4c20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 --2- 172.21.15.66:0/2354942307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb554071020 0x7fb554157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 >> 172.21.15.66:0/2354942307 conn(0x7fb5540b3ce0 msgr2=0x7fb55406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 shutdown_connections 2024-03-30T13:10:41.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:41.937+0000 7fb55a3e8640 1 -- 172.21.15.66:0/2354942307 wait complete. 2024-03-30T13:10:41.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:10:42.049 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:42.051 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:42.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.058+0000 7fc2b28a5640 1 Processor -- start 2024-03-30T13:10:42.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.058+0000 7fc2b28a5640 1 -- start start 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2b28a5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 0x7fc2ac0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2b28a5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc2ac0d3ee0 0x7fc2ac0d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2b28a5640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac0d3120 con 0x7fc2ac15e7a0 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2b28a5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac0cfc40 con 0x7fc2ac0cec60 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2b28a5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac0d3400 con 0x7fc2ac0d3ee0 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2ab7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 0x7fc2ac0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2abfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc2ac15e7a0 0x7fc2ac0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42794/0 (socket says 172.21.15.66:42794) 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2ab7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 0x7fc2ac0d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43158/0 (socket says 172.21.15.66:43158) 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2abfff640 1 -- 172.21.15.66:0/2099614885 learned_addr learned my addr 172.21.15.66:0/2099614885 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.059+0000 7fc2b0e42640 1 --2- 172.21.15.66:0/2099614885 >> v2:172.21.15.117:3300/0 conn(0x7fc2ac0d3ee0 0x7fc2ac0d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2aaffd640 1 -- 172.21.15.66:0/2099614885 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352784727 0 0) 0x7fc2ac0d3120 con 0x7fc2ac15e7a0 2024-03-30T13:10:42.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2aaffd640 1 -- 172.21.15.66:0/2099614885 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc294003520 con 0x7fc2ac15e7a0 2024-03-30T13:10:42.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2b0e42640 1 -- 172.21.15.66:0/2099614885 >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 msgr2=0x7fc2ac0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:42.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2b0e42640 1 --2- 172.21.15.66:0/2099614885 >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 0x7fc2ac0d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2b0e42640 1 -- 172.21.15.66:0/2099614885 >> v1:172.21.15.66:6789/0 conn(0x7fc2ac15e7a0 legacy=0x7fc2ac0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2b0e42640 1 -- 172.21.15.66:0/2099614885 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac0d36e0 con 0x7fc2ac0d3ee0 2024-03-30T13:10:42.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.060+0000 7fc2ab7fe640 1 --2- 172.21.15.66:0/2099614885 >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 0x7fc2ac0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:42.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b0e42640 1 --2- 172.21.15.66:0/2099614885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0d17f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0cf320 tx=0x7fc2a402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64e225a8e2cf570 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2aaffd640 1 -- 172.21.15.66:0/2099614885 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4039070 con 0x7fc2ac0d3ee0 2024-03-30T13:10:42.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2aaffd640 1 -- 172.21.15.66:0/2099614885 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a402cbf0 con 0x7fc2ac0d3ee0 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2aaffd640 1 -- 172.21.15.66:0/2099614885 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4034440 con 0x7fc2ac0d3ee0 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 -- 172.21.15.66:0/2099614885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 msgr2=0x7fc2ac0d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/2099614885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0d17f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0cf320 tx=0x7fc2a402be40 comp rx=0 tx=0).stop 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 -- 172.21.15.66:0/2099614885 shutdown_connections 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/2099614885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0d17f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/2099614885 >> v2:172.21.15.66:3300/0 conn(0x7fc2ac0cec60 0x7fc2ac0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 -- 172.21.15.66:0/2099614885 >> 172.21.15.66:0/2099614885 conn(0x7fc2ac0676b0 msgr2=0x7fc2ac067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 -- 172.21.15.66:0/2099614885 shutdown_connections 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.061+0000 7fc2b28a5640 1 -- 172.21.15.66:0/2099614885 wait complete. 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.062+0000 7fc2b28a5640 1 Processor -- start 2024-03-30T13:10:42.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 -- start start 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 0x7fc2ac14d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2ac15a330 0x7fc2ac160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0b7720 con 0x7fc2ac0cec60 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0b75a0 con 0x7fc2ac0d3ee0 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.063+0000 7fc2b28a5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac0b78a0 con 0x7fc2ac15a330 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 0x7fc2ac14d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 0x7fc2ac14d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43170/0 (socket says 172.21.15.66:43170) 2024-03-30T13:10:42.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 -- 172.21.15.66:0/808904223 learned_addr learned my addr 172.21.15.66:0/808904223 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2ab7fe640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2b0e42640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2ac15a330 0x7fc2ac160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 -- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2ac15a330 msgr2=0x7fc2ac160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2ac15a330 0x7fc2ac160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 -- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 msgr2=0x7fc2ac0c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 -- 172.21.15.66:0/808904223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac0b46e0 con 0x7fc2ac0cec60 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2ab7fe640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2abfff640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 0x7fc2ac14d830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c390 tx=0x7fc29c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c017070 con 0x7fc2ac0cec60 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29c005ce0 con 0x7fc2ac0cec60 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac0cb460 con 0x7fc2ac0cec60 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0072f0 con 0x7fc2ac0cec60 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.064+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac0cba20 con 0x7fc2ac0cec60 2024-03-30T13:10:42.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.066+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac075900 con 0x7fc2ac0cec60 2024-03-30T13:10:42.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.066+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc29c01a070 con 0x7fc2ac0cec60 2024-03-30T13:10:42.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.066+0000 7fc2a8ff9640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc29806c560 0x7fc29806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.066+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fc29c08adf0 con 0x7fc2ac0cec60 2024-03-30T13:10:42.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.067+0000 7fc2ab7fe640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc29806c560 0x7fc29806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.067+0000 7fc2ab7fe640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc29806c560 0x7fc29806ea20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0c9b10 tx=0x7fc2a0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.071+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc29c0589a0 con 0x7fc2ac0cec60 2024-03-30T13:10:42.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.270+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2ac06d190 con 0x7fc2ac0cec60 2024-03-30T13:10:42.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.270+0000 7fc2a8ff9640 1 -- 172.21.15.66:0/808904223 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fc29c05cca0 con 0x7fc2ac0cec60 2024-03-30T13:10:42.274 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.272+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc29806c560 msgr2=0x7fc29806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.272+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc29806c560 0x7fc29806ea20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0c9b10 tx=0x7fc2a0009190 comp rx=0 tx=0).stop 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.272+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 msgr2=0x7fc2ac14d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.272+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 0x7fc2ac14d830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c390 tx=0x7fc29c00c850 comp rx=0 tx=0).stop 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 shutdown_connections 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc29806c560 0x7fc29806ea20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2ac15a330 0x7fc2ac160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2ac0d3ee0 0x7fc2ac0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 --2- 172.21.15.66:0/808904223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2ac0cec60 0x7fc2ac14d830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 >> 172.21.15.66:0/808904223 conn(0x7fc2ac0676b0 msgr2=0x7fc2ac067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:42.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 shutdown_connections 2024-03-30T13:10:42.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.273+0000 7fc2b28a5640 1 -- 172.21.15.66:0/808904223 wait complete. 2024-03-30T13:10:42.287 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-30T13:10:42.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:42.288 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-30T13:10:42.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:10:42.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:10:42.317 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T13:10:42.372 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T13:10:42.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:42.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.478+0000 7f67b3ed9640 1 Processor -- start 2024-03-30T13:10:42.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.478+0000 7f67b3ed9640 1 -- start start 2024-03-30T13:10:42.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b3ed9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f67ac071c30 0x7f67ac160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b3ed9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f67ac160660 0x7f67ac0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b3ed9640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67ac146040 con 0x7f67ac145c60 2024-03-30T13:10:42.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b3ed9640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f67ac160e50 con 0x7f67ac160660 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b3ed9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f67ac15d8b0 con 0x7f67ac071c30 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f67ac160660 0x7f67ac0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f67ac160660 0x7f67ac0ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43174/0 (socket says 172.21.15.66:43174) 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b1474640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f67ac071c30 0x7f67ac160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 -- 172.21.15.66:0/3795851909 learned_addr learned my addr 172.21.15.66:0/3795851909 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 -- 172.21.15.66:0/3795851909 >> v2:172.21.15.117:3300/0 conn(0x7f67ac071c30 msgr2=0x7f67ac160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 --2- 172.21.15.66:0/3795851909 >> v2:172.21.15.117:3300/0 conn(0x7f67ac071c30 0x7f67ac160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 -- 172.21.15.66:0/3795851909 >> v1:172.21.15.117:6789/0 conn(0x7f67ac145c60 legacy=0x7f67ac071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.479+0000 7f67b2476640 1 -- 172.21.15.66:0/3795851909 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac15caf0 con 0x7f67ac160660 2024-03-30T13:10:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.480+0000 7f67b2476640 1 --2- 172.21.15.66:0/3795851909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac0ca830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67ac073b90 tx=0x7f67a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfe9c9ecbc2268c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.480+0000 7f67b0c73640 1 -- 172.21.15.66:0/3795851909 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a402ca80 con 0x7f67ac160660 2024-03-30T13:10:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.480+0000 7f67b0c73640 1 -- 172.21.15.66:0/3795851909 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a402cc20 con 0x7f67ac160660 2024-03-30T13:10:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.480+0000 7f67b0c73640 1 -- 172.21.15.66:0/3795851909 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40343f0 con 0x7f67ac160660 2024-03-30T13:10:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 -- 172.21.15.66:0/3795851909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 msgr2=0x7f67ac0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/3795851909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac0ca830 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67ac073b90 tx=0x7f67a402bee0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 -- 172.21.15.66:0/3795851909 shutdown_connections 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/3795851909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac0ca830 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/3795851909 >> v2:172.21.15.117:3300/0 conn(0x7f67ac071c30 0x7f67ac160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 -- 172.21.15.66:0/3795851909 >> 172.21.15.66:0/3795851909 conn(0x7f67ac0b1c90 msgr2=0x7f67ac0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 -- 172.21.15.66:0/3795851909 shutdown_connections 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.481+0000 7f67b3ed9640 1 -- 172.21.15.66:0/3795851909 wait complete. 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.482+0000 7f67b3ed9640 1 Processor -- start 2024-03-30T13:10:42.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.482+0000 7f67b3ed9640 1 -- start start 2024-03-30T13:10:42.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b3ed9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67ac071c30 0x7f67ac1f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b3ed9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67ac145c60 0x7f67ac0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b3ed9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b3ed9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac160a90 con 0x7f67ac160660 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b3ed9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac15d0b0 con 0x7f67ac071c30 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b3ed9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f67ac160c10 con 0x7f67ac145c60 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b2476640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b2476640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac13f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43176/0 (socket says 172.21.15.66:43176) 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b2476640 1 -- 172.21.15.66:0/2991269503 learned_addr learned my addr 172.21.15.66:0/2991269503 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b1474640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67ac145c60 0x7f67ac0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b1c75640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67ac071c30 0x7f67ac1f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b2476640 1 -- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67ac145c60 msgr2=0x7f67ac0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.483+0000 7f67b2476640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67ac145c60 0x7f67ac0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67b2476640 1 -- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67ac071c30 msgr2=0x7f67ac1f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67b2476640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67ac071c30 0x7f67ac1f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67b2476640 1 -- 172.21.15.66:0/2991269503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac0b2ed0 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67b2476640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac13f8a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009f40 tx=0x7f67a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4043020 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a403e450 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac0b31b0 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4009510 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.484+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac0b3770 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.485+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ac075a10 con 0x7f67ac160660 2024-03-30T13:10:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.486+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f67a4046020 con 0x7f67ac160660 2024-03-30T13:10:42.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.486+0000 7f67a27fc640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f679006c530 0x7f679006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.487+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f67a40ae1d0 con 0x7f67ac160660 2024-03-30T13:10:42.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.487+0000 7f67b1c75640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f679006c530 0x7f679006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.487+0000 7f67b1c75640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f679006c530 0x7f679006e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f67ac1f5300 tx=0x7f679c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.491+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f67a404d030 con 0x7f67ac160660 2024-03-30T13:10:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.715+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67ac0ca610 con 0x7f67ac160660 2024-03-30T13:10:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.715+0000 7f67a27fc640 1 -- 172.21.15.66:0/2991269503 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f67a407bd00 con 0x7f67ac160660 2024-03-30T13:10:42.719 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:42.719 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f679006c530 msgr2=0x7f679006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f679006c530 0x7f679006e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f67ac1f5300 tx=0x7f679c009040 comp rx=0 tx=0).stop 2024-03-30T13:10:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 msgr2=0x7f67ac13f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac13f8a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009f40 tx=0x7f67a4004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 shutdown_connections 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f679006c530 0x7f679006e9f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67ac160660 0x7f67ac13f8a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67ac145c60 0x7f67ac0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 --2- 172.21.15.66:0/2991269503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67ac071c30 0x7f67ac1f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 >> 172.21.15.66:0/2991269503 conn(0x7f67ac0b1c90 msgr2=0x7f67ac15e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 shutdown_connections 2024-03-30T13:10:42.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.717+0000 7f67b3ed9640 1 -- 172.21.15.66:0/2991269503 wait complete. 2024-03-30T13:10:42.735 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:42.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.842+0000 7ff32ebe2640 1 Processor -- start 2024-03-30T13:10:42.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.842+0000 7ff32ebe2640 1 -- start start 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32ebe2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff32815e950 0x7ff3280d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32ebe2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d48c0 0x7ff3280d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32ebe2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff3280d2f00 0x7ff3280cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32ebe2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff3280d3cb0 con 0x7ff32815e950 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32ebe2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff3280d4d00 con 0x7ff3280d2f00 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32ebe2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff3280d36f0 con 0x7ff3280d48c0 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32c97e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff32815e950 0x7ff3280d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32c97e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff32815e950 0x7ff3280d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43178/0 (socket says 172.21.15.66:43178) 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32c97e640 1 -- 172.21.15.66:0/4073170032 learned_addr learned my addr 172.21.15.66:0/4073170032 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff327fff640 1 --2- 172.21.15.66:0/4073170032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d48c0 0x7ff3280d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.843+0000 7ff32d17f640 1 --2- 172.21.15.66:0/4073170032 >> v2:172.21.15.117:3300/0 conn(0x7ff3280d2f00 0x7ff3280cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.844+0000 7ff32c97e640 1 -- 172.21.15.66:0/4073170032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d48c0 msgr2=0x7ff3280d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.844+0000 7ff32c97e640 1 --2- 172.21.15.66:0/4073170032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d48c0 0x7ff3280d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.844+0000 7ff32c97e640 1 -- 172.21.15.66:0/4073170032 >> v2:172.21.15.117:3300/0 conn(0x7ff3280d2f00 msgr2=0x7ff3280cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.844+0000 7ff32c97e640 1 --2- 172.21.15.66:0/4073170032 >> v2:172.21.15.117:3300/0 conn(0x7ff3280d2f00 0x7ff3280cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.844+0000 7ff32c97e640 1 -- 172.21.15.66:0/4073170032 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3280d39d0 con 0x7ff32815e950 2024-03-30T13:10:42.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.844+0000 7ff32d17f640 1 --2- 172.21.15.66:0/4073170032 >> v2:172.21.15.117:3300/0 conn(0x7ff3280d2f00 0x7ff3280cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.845+0000 7ff32c97e640 1 --2- 172.21.15.66:0/4073170032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32815e950 0x7ff3280d4380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff3280ce9f0 tx=0x7ff31c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d2933a9c3833a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.845+0000 7ff3277fe640 1 -- 172.21.15.66:0/4073170032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c034070 con 0x7ff32815e950 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.845+0000 7ff3277fe640 1 -- 172.21.15.66:0/4073170032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31c030860 con 0x7ff32815e950 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.845+0000 7ff3277fe640 1 -- 172.21.15.66:0/4073170032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c030c30 con 0x7ff32815e950 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 -- 172.21.15.66:0/4073170032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32815e950 msgr2=0x7ff3280d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/4073170032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32815e950 0x7ff3280d4380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff3280ce9f0 tx=0x7ff31c02b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 -- 172.21.15.66:0/4073170032 shutdown_connections 2024-03-30T13:10:42.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/4073170032 >> v2:172.21.15.117:3300/0 conn(0x7ff3280d2f00 0x7ff3280cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/4073170032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d48c0 0x7ff3280d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/4073170032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32815e950 0x7ff3280d4380 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 -- 172.21.15.66:0/4073170032 >> 172.21.15.66:0/4073170032 conn(0x7ff3280676e0 msgr2=0x7ff328067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:42.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 -- 172.21.15.66:0/4073170032 shutdown_connections 2024-03-30T13:10:42.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.846+0000 7ff32ebe2640 1 -- 172.21.15.66:0/4073170032 wait complete. 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.847+0000 7ff32ebe2640 1 Processor -- start 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.847+0000 7ff32ebe2640 1 -- start start 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32ebe2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d2f00 0x7ff3280aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32ebe2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 0x7ff3280ab330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32ebe2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff32815e950 0x7ff3280c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32ebe2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3280d3390 con 0x7ff3280d48c0 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32ebe2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3280b7da0 con 0x7ff32815e950 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32ebe2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff32815ed30 con 0x7ff3280d2f00 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 0x7ff3280ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 0x7ff3280ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43192/0 (socket says 172.21.15.66:43192) 2024-03-30T13:10:42.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 -- 172.21.15.66:0/2851016373 learned_addr learned my addr 172.21.15.66:0/2851016373 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:42.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32d17f640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff32815e950 0x7ff3280c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32c97e640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d2f00 0x7ff3280aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 -- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d2f00 msgr2=0x7ff3280aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d2f00 0x7ff3280aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 -- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff32815e950 msgr2=0x7ff3280c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff32815e950 0x7ff3280c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff327fff640 1 -- 172.21.15.66:0/2851016373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32814e290 con 0x7ff3280d48c0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.848+0000 7ff32c97e640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d2f00 0x7ff3280aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.849+0000 7ff327fff640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 0x7ff3280ab330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff31400e9e0 tx=0x7ff31400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.849+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314017070 con 0x7ff3280d48c0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.849+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31400c940 con 0x7ff3280d48c0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.849+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400cd30 con 0x7ff3280d48c0 2024-03-30T13:10:42.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.849+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff328160180 con 0x7ff3280d48c0 2024-03-30T13:10:42.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.849+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff328160740 con 0x7ff3280d48c0 2024-03-30T13:10:42.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.850+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff328075a10 con 0x7ff3280d48c0 2024-03-30T13:10:42.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.850+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff31401a070 con 0x7ff3280d48c0 2024-03-30T13:10:42.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.851+0000 7ff3257fa640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff30c06c530 0x7ff30c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:42.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.851+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7ff31408b950 con 0x7ff3280d48c0 2024-03-30T13:10:42.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.851+0000 7ff32c97e640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff30c06c530 0x7ff30c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:42.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.851+0000 7ff32c97e640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff30c06c530 0x7ff30c06e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff31c036000 tx=0x7ff31c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:42.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:42.855+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff314059be0 con 0x7ff3280d48c0 2024-03-30T13:10:43.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.041+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3280b9270 con 0x7ff3280d48c0 2024-03-30T13:10:43.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.043+0000 7ff3257fa640 1 -- 172.21.15.66:0/2851016373 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7ff3140594e0 con 0x7ff3280d48c0 2024-03-30T13:10:43.046 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:43.047 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:43.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.044+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff30c06c530 msgr2=0x7ff30c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.044+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff30c06c530 0x7ff30c06e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff31c036000 tx=0x7ff31c03b040 comp rx=0 tx=0).stop 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.044+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 msgr2=0x7ff3280ab330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.044+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 0x7ff3280ab330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff31400e9e0 tx=0x7ff31400ede0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 shutdown_connections 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff30c06c530 0x7ff30c06e9f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff32815e950 0x7ff3280c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3280d48c0 0x7ff3280ab330 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 --2- 172.21.15.66:0/2851016373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3280d2f00 0x7ff3280aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 >> 172.21.15.66:0/2851016373 conn(0x7ff3280676e0 msgr2=0x7ff3280cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 shutdown_connections 2024-03-30T13:10:43.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.045+0000 7ff32ebe2640 1 -- 172.21.15.66:0/2851016373 wait complete. 2024-03-30T13:10:43.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:09:40.767113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:09:41.426231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:09:45.120738+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:10:43.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.174+0000 7f2ef90eb640 1 Processor -- start 2024-03-30T13:10:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.174+0000 7f2ef90eb640 1 -- start start 2024-03-30T13:10:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef90eb640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2ef415aaf0 0x7f2ef40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef90eb640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef40b4d70 con 0x7f2ef40718c0 2024-03-30T13:10:43.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef90eb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef40b35c0 con 0x7f2ef415a710 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef90eb640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef40b41d0 con 0x7f2ef415aaf0 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef259c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2ef415aaf0 0x7f2ef40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef359e640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2ef40718c0 0x7f2ef406d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42796/0 (socket says 172.21.15.66:42796) 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef359e640 1 -- 172.21.15.66:0/1495646664 learned_addr learned my addr 172.21.15.66:0/1495646664 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef259c640 1 -- 172.21.15.66:0/1495646664 >> v1:172.21.15.117:6789/0 conn(0x7f2ef415a710 legacy=0x7f2ef40b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef1d9b640 1 -- 172.21.15.66:0/1495646664 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019477002 0 0) 0x7f2ef40b35c0 con 0x7f2ef415a710 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef259c640 1 -- 172.21.15.66:0/1495646664 >> v1:172.21.15.66:6789/0 conn(0x7f2ef40718c0 legacy=0x7f2ef406d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.176+0000 7f2ef259c640 1 -- 172.21.15.66:0/1495646664 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef40b38a0 con 0x7f2ef415aaf0 2024-03-30T13:10:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.177+0000 7f2ef259c640 1 --2- 172.21.15.66:0/1495646664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415aaf0 0x7f2ef40b2f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40b4350 tx=0x7f2edc02cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd2ad0b2373470b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef1d9b640 1 -- 172.21.15.66:0/1495646664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc03a070 con 0x7f2ef415aaf0 2024-03-30T13:10:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef1d9b640 1 -- 172.21.15.66:0/1495646664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2edc02d990 con 0x7f2ef415aaf0 2024-03-30T13:10:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef1d9b640 1 -- 172.21.15.66:0/1495646664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00b040 con 0x7f2ef415aaf0 2024-03-30T13:10:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1495646664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415aaf0 msgr2=0x7f2ef40b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1495646664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415aaf0 0x7f2ef40b2f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40b4350 tx=0x7f2edc02cc60 comp rx=0 tx=0).stop 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1495646664 shutdown_connections 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1495646664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415aaf0 0x7f2ef40b2f20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1495646664 >> 172.21.15.66:0/1495646664 conn(0x7f2ef40b5b00 msgr2=0x7f2ef406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1495646664 shutdown_connections 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.178+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1495646664 wait complete. 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.180+0000 7f2ef90eb640 1 Processor -- start 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.181+0000 7f2ef90eb640 1 -- start start 2024-03-30T13:10:43.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef90eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef40718c0 0x7f2ef414f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef90eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 0x7f2ef40ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef90eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef415aaf0 0x7f2ef40acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef90eb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4072c50 con 0x7f2ef415a710 2024-03-30T13:10:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef90eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef40b3e60 con 0x7f2ef40718c0 2024-03-30T13:10:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef90eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef406d8f0 con 0x7f2ef415aaf0 2024-03-30T13:10:43.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 0x7f2ef40ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 0x7f2ef40ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43214/0 (socket says 172.21.15.66:43214) 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 -- 172.21.15.66:0/1558439999 learned_addr learned my addr 172.21.15.66:0/1558439999 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef2d9d640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef40718c0 0x7f2ef414f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef359e640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef415aaf0 0x7f2ef40acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 -- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef415aaf0 msgr2=0x7f2ef40acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef415aaf0 0x7f2ef40acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 -- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef40718c0 msgr2=0x7f2ef414f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef40718c0 0x7f2ef414f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 -- 172.21.15.66:0/1558439999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef4150970 con 0x7f2ef415a710 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.182+0000 7f2ef259c640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 0x7f2ef40ac9f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.183+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc042020 con 0x7f2ef415a710 2024-03-30T13:10:43.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.183+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2edc035ac0 con 0x7f2ef415a710 2024-03-30T13:10:43.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.183+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef414fbb0 con 0x7f2ef415a710 2024-03-30T13:10:43.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.183+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc03a040 con 0x7f2ef415a710 2024-03-30T13:10:43.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.183+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef4150170 con 0x7f2ef415a710 2024-03-30T13:10:43.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.184+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4075890 con 0x7f2ef415a710 2024-03-30T13:10:43.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.184+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2edc045020 con 0x7f2ef415a710 2024-03-30T13:10:43.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.185+0000 7f2eeb7fe640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ed406c530 0x7f2ed406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.185+0000 7f2ef2d9d640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ed406c530 0x7f2ed406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.185+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f2edc0ad400 con 0x7f2ef415a710 2024-03-30T13:10:43.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.185+0000 7f2ef2d9d640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ed406c530 0x7f2ed406e9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000f60 tx=0x7f2ee4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.190+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2edc04c030 con 0x7f2ef415a710 2024-03-30T13:10:43.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.378+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ef406dde0 con 0x7f2ef415a710 2024-03-30T13:10:43.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.379+0000 7f2eeb7fe640 1 -- 172.21.15.66:0/1558439999 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f2edc07af10 con 0x7f2ef415a710 2024-03-30T13:10:43.383 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ed406c530 msgr2=0x7f2ed406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ed406c530 0x7f2ed406e9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000f60 tx=0x7f2ee4009040 comp rx=0 tx=0).stop 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 msgr2=0x7f2ef40ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 0x7f2ef40ac9f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc0046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 shutdown_connections 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ed406c530 0x7f2ed406e9f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef415aaf0 0x7f2ef40acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef415a710 0x7f2ef40ac9f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 --2- 172.21.15.66:0/1558439999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef40718c0 0x7f2ef414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 >> 172.21.15.66:0/1558439999 conn(0x7f2ef40b5b00 msgr2=0x7f2ef40b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:43.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 shutdown_connections 2024-03-30T13:10:43.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.381+0000 7f2ef90eb640 1 -- 172.21.15.66:0/1558439999 wait complete. 2024-03-30T13:10:43.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:10:43.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.504+0000 7f894e075640 1 Processor -- start 2024-03-30T13:10:43.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.504+0000 7f894e075640 1 -- start start 2024-03-30T13:10:43.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.505+0000 7f894e075640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89480b5dc0 0x7f89480b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.505+0000 7f894e075640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89480ba090 0x7f89480d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.505+0000 7f894e075640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89480b66e0 con 0x7f894806c620 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.505+0000 7f894e075640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f89480d2ff0 con 0x7f89480b5dc0 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.505+0000 7f894e075640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f89480d32d0 con 0x7f89480ba090 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f8947fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f894806c620 0x7f894806cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:51776/0 (socket says 172.21.15.66:51776) 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89480ba090 0x7f89480d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f8947fff640 1 -- 172.21.15.66:0/1547741526 learned_addr learned my addr 172.21.15.66:0/1547741526 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89477fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89480b5dc0 0x7f89480b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89467fc640 1 -- 172.21.15.66:0/1547741526 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984675759 0 0) 0x7f89480b66e0 con 0x7f894806c620 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89467fc640 1 -- 172.21.15.66:0/1547741526 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8934003520 con 0x7f894806c620 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89477fe640 1 -- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89480ba090 msgr2=0x7f89480d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:43.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89477fe640 1 --2- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89480ba090 0x7f89480d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89477fe640 1 -- 172.21.15.66:0/1547741526 >> v1:172.21.15.117:6789/0 conn(0x7f894806c620 legacy=0x7f894806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f89477fe640 1 -- 172.21.15.66:0/1547741526 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89480d35b0 con 0x7f89480b5dc0 2024-03-30T13:10:43.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.506+0000 7f8946ffd640 1 --2- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89480ba090 0x7f89480d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:43.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.507+0000 7f89477fe640 1 --2- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480b5dc0 0x7f89480b61a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89380097a0 tx=0x7f893802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=540246e172ef3e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.507+0000 7f89467fc640 1 -- 172.21.15.66:0/1547741526 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938035020 con 0x7f89480b5dc0 2024-03-30T13:10:43.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.507+0000 7f89467fc640 1 -- 172.21.15.66:0/1547741526 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8938031830 con 0x7f89480b5dc0 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.507+0000 7f89467fc640 1 -- 172.21.15.66:0/1547741526 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938031be0 con 0x7f89480b5dc0 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 -- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480b5dc0 msgr2=0x7f89480b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 --2- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480b5dc0 0x7f89480b61a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89380097a0 tx=0x7f893802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 -- 172.21.15.66:0/1547741526 shutdown_connections 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 --2- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89480ba090 0x7f89480d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 --2- 172.21.15.66:0/1547741526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480b5dc0 0x7f89480b61a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 -- 172.21.15.66:0/1547741526 >> 172.21.15.66:0/1547741526 conn(0x7f89480678f0 msgr2=0x7f89480d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 -- 172.21.15.66:0/1547741526 shutdown_connections 2024-03-30T13:10:43.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.508+0000 7f894e075640 1 -- 172.21.15.66:0/1547741526 wait complete. 2024-03-30T13:10:43.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.510+0000 7f894e075640 1 Processor -- start 2024-03-30T13:10:43.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.510+0000 7f894e075640 1 -- start start 2024-03-30T13:10:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f894e075640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 0x7f8948158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f894e075640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89480b5dc0 0x7f89480cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f894e075640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480ba090 0x7f8948160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f894e075640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89480d4090 con 0x7f89480b5dc0 2024-03-30T13:10:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f894e075640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89480d3b70 con 0x7f89480ba090 2024-03-30T13:10:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f894e075640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f89480b83e0 con 0x7f894806c620 2024-03-30T13:10:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89480b5dc0 0x7f89480cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f89477fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 0x7f8948158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f89477fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 0x7f8948158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:55662/0 (socket says 172.21.15.66:55662) 2024-03-30T13:10:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f89477fe640 1 -- 172.21.15.66:0/4067828952 learned_addr learned my addr 172.21.15.66:0/4067828952 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8947fff640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480ba090 0x7f8948160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8946ffd640 1 -- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 msgr2=0x7f8948158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8946ffd640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 0x7f8948158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8946ffd640 1 -- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480ba090 msgr2=0x7f8948160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8946ffd640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480ba090 0x7f8948160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.511+0000 7f8946ffd640 1 -- 172.21.15.66:0/4067828952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89480cbec0 con 0x7f89480b5dc0 2024-03-30T13:10:43.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f89477fe640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 0x7f8948158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:43.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f8946ffd640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89480b5dc0 0x7f89480cda70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f893000e870 tx=0x7f893000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930017070 con 0x7f89480b5dc0 2024-03-30T13:10:43.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893000c940 con 0x7f89480b5dc0 2024-03-30T13:10:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893000cd10 con 0x7f89480b5dc0 2024-03-30T13:10:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89480cc1a0 con 0x7f89480b5dc0 2024-03-30T13:10:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.512+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89480cc760 con 0x7f89480b5dc0 2024-03-30T13:10:43.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.513+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8948075890 con 0x7f89480b5dc0 2024-03-30T13:10:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.513+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f893001a070 con 0x7f89480b5dc0 2024-03-30T13:10:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.514+0000 7f893ffff640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f892806c530 0x7f892806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.514+0000 7f89477fe640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f892806c530 0x7f892806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.514+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f893008bb20 con 0x7f89480b5dc0 2024-03-30T13:10:43.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.514+0000 7f89477fe640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f892806c530 0x7f892806e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8938009560 tx=0x7f893803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.518+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8930059760 con 0x7f89480b5dc0 2024-03-30T13:10:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.703+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89480d2c00 con 0x7f89480b5dc0 2024-03-30T13:10:43.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.704+0000 7f893ffff640 1 -- 172.21.15.66:0/4067828952 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f893005da60 con 0x7f89480b5dc0 2024-03-30T13:10:43.707 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f892806c530 msgr2=0x7f892806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f892806c530 0x7f892806e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8938009560 tx=0x7f893803c040 comp rx=0 tx=0).stop 2024-03-30T13:10:43.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89480b5dc0 msgr2=0x7f89480cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89480b5dc0 0x7f89480cda70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f893000e870 tx=0x7f893000ed30 comp rx=0 tx=0).stop 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 shutdown_connections 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f892806c530 0x7f892806e9f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89480ba090 0x7f8948160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89480b5dc0 0x7f89480cda70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 --2- 172.21.15.66:0/4067828952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f894806c620 0x7f8948158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.705+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 >> 172.21.15.66:0/4067828952 conn(0x7f89480678f0 msgr2=0x7f89480cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.706+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 shutdown_connections 2024-03-30T13:10:43.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.706+0000 7f894e075640 1 -- 172.21.15.66:0/4067828952 wait complete. 2024-03-30T13:10:43.720 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:10:43.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.827+0000 7f748f89a640 1 Processor -- start 2024-03-30T13:10:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.827+0000 7f748f89a640 1 -- start start 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748f89a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7488162090 0x7f7488162500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748f89a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74880cc840 0x7f74880ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748f89a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74880cd190 con 0x7f748815e4b0 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748f89a640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f74880cfac0 con 0x7f7488162090 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748f89a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7488162c00 con 0x7f74880cc840 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7488162090 0x7f7488162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7488162090 0x7f7488162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43224/0 (socket says 172.21.15.66:43224) 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 -- 172.21.15.66:0/3330328459 learned_addr learned my addr 172.21.15.66:0/3330328459 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748de37640 1 --2- 172.21.15.66:0/3330328459 >> v2:172.21.15.117:3300/0 conn(0x7f74880cc840 0x7f74880ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 -- 172.21.15.66:0/3330328459 >> v2:172.21.15.117:3300/0 conn(0x7f74880cc840 msgr2=0x7f74880ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 --2- 172.21.15.66:0/3330328459 >> v2:172.21.15.117:3300/0 conn(0x7f74880cc840 0x7f74880ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 -- 172.21.15.66:0/3330328459 >> v1:172.21.15.117:6789/0 conn(0x7f748815e4b0 legacy=0x7f74880cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f747ffff640 1 -- 172.21.15.66:0/3330328459 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566113709 0 0) 0x7f7484001110 con 0x7f748815e4b0 2024-03-30T13:10:43.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.828+0000 7f748ce35640 1 -- 172.21.15.66:0/3330328459 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7488162ee0 con 0x7f7488162090 2024-03-30T13:10:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.829+0000 7f748ce35640 1 --2- 172.21.15.66:0/3330328459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7488162090 0x7f7488162500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f74880c5b20 tx=0x7f747802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d2e2a4571eda server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.829+0000 7f747ffff640 1 -- 172.21.15.66:0/3330328459 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478034070 con 0x7f7488162090 2024-03-30T13:10:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.829+0000 7f747ffff640 1 -- 172.21.15.66:0/3330328459 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7478030860 con 0x7f7488162090 2024-03-30T13:10:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.829+0000 7f747ffff640 1 -- 172.21.15.66:0/3330328459 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478030c30 con 0x7f7488162090 2024-03-30T13:10:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 -- 172.21.15.66:0/3330328459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7488162090 msgr2=0x7f7488162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 --2- 172.21.15.66:0/3330328459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7488162090 0x7f7488162500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f74880c5b20 tx=0x7f747802b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 -- 172.21.15.66:0/3330328459 shutdown_connections 2024-03-30T13:10:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 --2- 172.21.15.66:0/3330328459 >> v2:172.21.15.117:3300/0 conn(0x7f74880cc840 0x7f74880ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 --2- 172.21.15.66:0/3330328459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7488162090 0x7f7488162500 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 -- 172.21.15.66:0/3330328459 >> 172.21.15.66:0/3330328459 conn(0x7f74880b3d20 msgr2=0x7f74880b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 -- 172.21.15.66:0/3330328459 shutdown_connections 2024-03-30T13:10:43.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.830+0000 7f748f89a640 1 -- 172.21.15.66:0/3330328459 wait complete. 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.831+0000 7f748f89a640 1 Processor -- start 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.831+0000 7f748f89a640 1 -- start start 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748f89a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74880cc840 0x7f74880b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748f89a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 0x7f748814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748f89a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7488162090 0x7f748806f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748f89a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74880d2420 con 0x7f748815e4b0 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748f89a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74880cee90 con 0x7f74880cc840 2024-03-30T13:10:43.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748f89a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f748815f8a0 con 0x7f7488162090 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 0x7f748814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 0x7f748814b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43226/0 (socket says 172.21.15.66:43226) 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748d636640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74880cc840 0x7f74880b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 -- 172.21.15.66:0/3111455336 learned_addr learned my addr 172.21.15.66:0/3111455336 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748de37640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7488162090 0x7f748806f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 -- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7488162090 msgr2=0x7f748806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7488162090 0x7f748806f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 -- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74880cc840 msgr2=0x7f74880b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74880cc840 0x7f74880b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.832+0000 7f748ce35640 1 -- 172.21.15.66:0/3111455336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74880b50e0 con 0x7f748815e4b0 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.833+0000 7f748ce35640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 0x7f748814b720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74780090b0 tx=0x7f74780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.833+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478034040 con 0x7f748815e4b0 2024-03-30T13:10:43.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.833+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747803ab90 con 0x7f748815e4b0 2024-03-30T13:10:43.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.833+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74880b5350 con 0x7f748815e4b0 2024-03-30T13:10:43.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.833+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478004030 con 0x7f748815e4b0 2024-03-30T13:10:43.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.833+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74880b5910 con 0x7f748815e4b0 2024-03-30T13:10:43.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.834+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488075890 con 0x7f748815e4b0 2024-03-30T13:10:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.834+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7478037070 con 0x7f748815e4b0 2024-03-30T13:10:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.835+0000 7f747dffb640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f746c06c530 0x7f746c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.835+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f74780ad8d0 con 0x7f748815e4b0 2024-03-30T13:10:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.835+0000 7f748d636640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f746c06c530 0x7f746c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:43.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.835+0000 7f748d636640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f746c06c530 0x7f746c06e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7484000f60 tx=0x7f7484009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:43.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:43.839+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f747803f070 con 0x7f748815e4b0 2024-03-30T13:10:44.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.024+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f74880b8170 con 0x7f748815e4b0 2024-03-30T13:10:44.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.025+0000 7f747dffb640 1 -- 172.21.15.66:0/3111455336 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f747807b480 con 0x7f748815e4b0 2024-03-30T13:10:44.029 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f746c06c530 msgr2=0x7f746c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f746c06c530 0x7f746c06e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7484000f60 tx=0x7f7484009040 comp rx=0 tx=0).stop 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 msgr2=0x7f748814b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 0x7f748814b720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74780090b0 tx=0x7f74780046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 shutdown_connections 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f746c06c530 0x7f746c06e9f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7488162090 0x7f748806f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f748815e4b0 0x7f748814b720 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 --2- 172.21.15.66:0/3111455336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74880cc840 0x7f74880b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 >> 172.21.15.66:0/3111455336 conn(0x7f74880b3d20 msgr2=0x7f74880b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 shutdown_connections 2024-03-30T13:10:44.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.027+0000 7f748f89a640 1 -- 172.21.15.66:0/3111455336 wait complete. 2024-03-30T13:10:44.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:10:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.147+0000 7fb59db50640 1 Processor -- start 2024-03-30T13:10:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.147+0000 7fb59db50640 1 -- start start 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb59db50640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5980b5dc0 0x7fb5980b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb59db50640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb59db50640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5980b66e0 con 0x7fb5980ba090 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb59db50640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb5980d2d90 con 0x7fb5980b5dc0 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb59db50640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980d2ff0 con 0x7fb5980b73d0 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb5977fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5980b5dc0 0x7fb5980b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb5977fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5980b5dc0 0x7fb5980b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43228/0 (socket says 172.21.15.66:43228) 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb5977fe640 1 -- 172.21.15.66:0/3316002063 learned_addr learned my addr 172.21.15.66:0/3316002063 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.148+0000 7fb597fff640 1 --2- 172.21.15.66:0/3316002063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.149+0000 7fb5967fc640 1 -- 172.21.15.66:0/3316002063 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4129405044 0 0) 0x7fb5980b66e0 con 0x7fb5980ba090 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.149+0000 7fb5977fe640 1 -- 172.21.15.66:0/3316002063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 msgr2=0x7fb5980b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.149+0000 7fb5977fe640 1 --2- 172.21.15.66:0/3316002063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.149+0000 7fb5977fe640 1 -- 172.21.15.66:0/3316002063 >> v1:172.21.15.117:6789/0 conn(0x7fb5980ba090 legacy=0x7fb5980d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.149+0000 7fb5977fe640 1 -- 172.21.15.66:0/3316002063 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5980d32d0 con 0x7fb5980b5dc0 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.149+0000 7fb5977fe640 1 --2- 172.21.15.66:0/3316002063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980b5dc0 0x7fb5980b61a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb588009870 tx=0x7fb58802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254a635ad858b135 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb5967fc640 1 -- 172.21.15.66:0/3316002063 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588034070 con 0x7fb5980b5dc0 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb5967fc640 1 -- 172.21.15.66:0/3316002063 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb588030840 con 0x7fb5980b5dc0 2024-03-30T13:10:44.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb5967fc640 1 -- 172.21.15.66:0/3316002063 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588030c10 con 0x7fb5980b5dc0 2024-03-30T13:10:44.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 -- 172.21.15.66:0/3316002063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980b5dc0 msgr2=0x7fb5980b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 --2- 172.21.15.66:0/3316002063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980b5dc0 0x7fb5980b61a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb588009870 tx=0x7fb58802b980 comp rx=0 tx=0).stop 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 -- 172.21.15.66:0/3316002063 shutdown_connections 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 --2- 172.21.15.66:0/3316002063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 --2- 172.21.15.66:0/3316002063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980b5dc0 0x7fb5980b61a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 -- 172.21.15.66:0/3316002063 >> 172.21.15.66:0/3316002063 conn(0x7fb5980678f0 msgr2=0x7fb5980d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 -- 172.21.15.66:0/3316002063 shutdown_connections 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.150+0000 7fb59db50640 1 -- 172.21.15.66:0/3316002063 wait complete. 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.151+0000 7fb59db50640 1 Processor -- start 2024-03-30T13:10:44.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 -- start start 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5980b5dc0 0x7fb5981f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 0x7fb5980b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980d3b70 con 0x7fb5980ba090 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980d3890 con 0x7fb5980b5dc0 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb59db50640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980d4090 con 0x7fb5980b73d0 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.152+0000 7fb597fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 0x7fb5980b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 0x7fb5980b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43230/0 (socket says 172.21.15.66:43230) 2024-03-30T13:10:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 -- 172.21.15.66:0/3777986826 learned_addr learned my addr 172.21.15.66:0/3777986826 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb5977fe640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5980b5dc0 0x7fb5981f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb596ffd640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 -- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 msgr2=0x7fb5980ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 -- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5980b5dc0 msgr2=0x7fb5981f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5980b5dc0 0x7fb5981f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 -- 172.21.15.66:0/3777986826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981f8760 con 0x7fb5980ba090 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb597fff640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 0x7fb5980b2150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb59000efc0 tx=0x7fb59000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590018020 con 0x7fb5980ba090 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb590014480 con 0x7fb5980ba090 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590014870 con 0x7fb5980ba090 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5981f8a40 con 0x7fb5980ba090 2024-03-30T13:10:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.153+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59814b710 con 0x7fb5980ba090 2024-03-30T13:10:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.154+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb598075890 con 0x7fb5980ba090 2024-03-30T13:10:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.155+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb590005d10 con 0x7fb5980ba090 2024-03-30T13:10:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.155+0000 7fb57ffff640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb58406c530 0x7fb58406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.155+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7fb59008c590 con 0x7fb5980ba090 2024-03-30T13:10:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.155+0000 7fb5977fe640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb58406c530 0x7fb58406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.156+0000 7fb5977fe640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb58406c530 0x7fb58406e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb588036000 tx=0x7fb588008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.160+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb590022070 con 0x7fb5980ba090 2024-03-30T13:10:44.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.343+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb5980d01f0 con 0x7fb5980ba090 2024-03-30T13:10:44.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.343+0000 7fb57ffff640 1 -- 172.21.15.66:0/3777986826 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fb590010040 con 0x7fb5980ba090 2024-03-30T13:10:44.347 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb58406c530 msgr2=0x7fb58406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb58406c530 0x7fb58406e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb588036000 tx=0x7fb588008030 comp rx=0 tx=0).stop 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 msgr2=0x7fb5980b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 0x7fb5980b2150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb59000efc0 tx=0x7fb59000c490 comp rx=0 tx=0).stop 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 shutdown_connections 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb58406c530 0x7fb58406e9f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5980ba090 0x7fb5980b2150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5980b73d0 0x7fb5980ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 --2- 172.21.15.66:0/3777986826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5980b5dc0 0x7fb5981f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 >> 172.21.15.66:0/3777986826 conn(0x7fb5980678f0 msgr2=0x7fb5980b9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:44.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 shutdown_connections 2024-03-30T13:10:44.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.345+0000 7fb59db50640 1 -- 172.21.15.66:0/3777986826 wait complete. 2024-03-30T13:10:44.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.463+0000 7f573c40d640 1 Processor -- start 2024-03-30T13:10:44.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.463+0000 7f573c40d640 1 -- start start 2024-03-30T13:10:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f573c40d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5734071c30 0x7f5734160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f573c40d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734160660 0x7f57340ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f573c40d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5734146040 con 0x7f5734145c60 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f573c40d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5734160e50 con 0x7f5734071c30 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f573c40d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f573415d8b0 con 0x7f5734160660 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5734071c30 0x7f5734160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5734071c30 0x7f5734160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43234/0 (socket says 172.21.15.66:43234) 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 -- 172.21.15.66:0/1120411272 learned_addr learned my addr 172.21.15.66:0/1120411272 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f573a9aa640 1 --2- 172.21.15.66:0/1120411272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734160660 0x7f57340ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57391a7640 1 -- 172.21.15.66:0/1120411272 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825599430 0 0) 0x7f5734146040 con 0x7f5734145c60 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 -- 172.21.15.66:0/1120411272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734160660 msgr2=0x7f57340ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 --2- 172.21.15.66:0/1120411272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734160660 0x7f57340ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 -- 172.21.15.66:0/1120411272 >> v1:172.21.15.117:6789/0 conn(0x7f5734145c60 legacy=0x7f5734071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.464+0000 7f57399a8640 1 -- 172.21.15.66:0/1120411272 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573415caf0 con 0x7f5734071c30 2024-03-30T13:10:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.465+0000 7f57399a8640 1 --2- 172.21.15.66:0/1120411272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734071c30 0x7f5734160120 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5724009870 tx=0x7f572402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee3df59e92bfc21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.465+0000 7f57391a7640 1 -- 172.21.15.66:0/1120411272 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724035020 con 0x7f5734071c30 2024-03-30T13:10:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.465+0000 7f57391a7640 1 -- 172.21.15.66:0/1120411272 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724031640 con 0x7f5734071c30 2024-03-30T13:10:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.465+0000 7f57391a7640 1 -- 172.21.15.66:0/1120411272 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572402d040 con 0x7f5734071c30 2024-03-30T13:10:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 -- 172.21.15.66:0/1120411272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734071c30 msgr2=0x7f5734160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 --2- 172.21.15.66:0/1120411272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734071c30 0x7f5734160120 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5724009870 tx=0x7f572402b8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 -- 172.21.15.66:0/1120411272 shutdown_connections 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 --2- 172.21.15.66:0/1120411272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734160660 0x7f57340ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 --2- 172.21.15.66:0/1120411272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734071c30 0x7f5734160120 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 -- 172.21.15.66:0/1120411272 >> 172.21.15.66:0/1120411272 conn(0x7f57340b1c90 msgr2=0x7f57340b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 -- 172.21.15.66:0/1120411272 shutdown_connections 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.466+0000 7f573c40d640 1 -- 172.21.15.66:0/1120411272 wait complete. 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.467+0000 7f573c40d640 1 Processor -- start 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.467+0000 7f573c40d640 1 -- start start 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734071c30 0x7f573415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5734145c60 0x7f57340c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 0x7f57340c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5734160ad0 con 0x7f5734160660 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f573415d0b0 con 0x7f5734145c60 2024-03-30T13:10:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f57340cad70 con 0x7f5734071c30 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 0x7f57340c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 0x7f57340c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43248/0 (socket says 172.21.15.66:43248) 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 -- 172.21.15.66:0/1415828166 learned_addr learned my addr 172.21.15.66:0/1415828166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a1a9640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734071c30 0x7f573415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f57399a8640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5734145c60 0x7f57340c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 -- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734071c30 msgr2=0x7f573415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734071c30 0x7f573415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 -- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5734145c60 msgr2=0x7f57340c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5734145c60 0x7f57340c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 -- 172.21.15.66:0/1415828166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57340ce9a0 con 0x7f5734160660 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573a9aa640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 0x7f57340c9850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f572c00efc0 tx=0x7f572c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c018020 con 0x7f5734160660 2024-03-30T13:10:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c014480 con 0x7f5734160660 2024-03-30T13:10:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57340cec80 con 0x7f5734160660 2024-03-30T13:10:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c014870 con 0x7f5734160660 2024-03-30T13:10:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.468+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57340cf240 con 0x7f5734160660 2024-03-30T13:10:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.469+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734075a10 con 0x7f5734160660 2024-03-30T13:10:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.470+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f572c005d10 con 0x7f5734160660 2024-03-30T13:10:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.470+0000 7f572affd640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f571806c530 0x7f571806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.470+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f572c08c460 con 0x7f5734160660 2024-03-30T13:10:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.470+0000 7f573a1a9640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f571806c530 0x7f571806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.471+0000 7f573a1a9640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f571806c530 0x7f571806e9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f60 tx=0x7f5730009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.475+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f572c022070 con 0x7f5734160660 2024-03-30T13:10:44.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.698+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f573415f0e0 con 0x7f5734160660 2024-03-30T13:10:44.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.698+0000 7f572affd640 1 -- 172.21.15.66:0/1415828166 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f572c010040 con 0x7f5734160660 2024-03-30T13:10:44.702 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:44.703 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:44.703 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f571806c530 msgr2=0x7f571806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f571806c530 0x7f571806e9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f60 tx=0x7f5730009040 comp rx=0 tx=0).stop 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 msgr2=0x7f57340c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 0x7f57340c9850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f572c00efc0 tx=0x7f572c00c490 comp rx=0 tx=0).stop 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 shutdown_connections 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f571806c530 0x7f571806e9f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5734160660 0x7f57340c9850 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5734145c60 0x7f57340c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 --2- 172.21.15.66:0/1415828166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5734071c30 0x7f573415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.700+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 >> 172.21.15.66:0/1415828166 conn(0x7f57340b1c90 msgr2=0x7f5734142a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:44.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.701+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 shutdown_connections 2024-03-30T13:10:44.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.701+0000 7f573c40d640 1 -- 172.21.15.66:0/1415828166 wait complete. 2024-03-30T13:10:44.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:44.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.819+0000 7f1cc5dc3640 1 Processor -- start 2024-03-30T13:10:44.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.819+0000 7f1cc5dc3640 1 -- start start 2024-03-30T13:10:44.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cc5dc3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0150 0x7f1cc015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cc5dc3640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1cc00b43e0 0x7f1cc00ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cc5dc3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc006e220 con 0x7f1cc00d0530 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cc5dc3640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc006df80 con 0x7f1cc00b43e0 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cc5dc3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc015f530 con 0x7f1cc00d0150 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbffff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1cc00b43e0 0x7f1cc00ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbeffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1cc00d0530 0x7f1cc00b3cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42812/0 (socket says 172.21.15.66:42812) 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbeffd640 1 -- 172.21.15.66:0/2053959128 learned_addr learned my addr 172.21.15.66:0/2053959128 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbf7fe640 1 --2- 172.21.15.66:0/2053959128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0150 0x7f1cc015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbffff640 1 -- 172.21.15.66:0/2053959128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0150 msgr2=0x7f1cc015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:44.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbffff640 1 --2- 172.21.15.66:0/2053959128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0150 0x7f1cc015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbffff640 1 -- 172.21.15.66:0/2053959128 >> v1:172.21.15.66:6789/0 conn(0x7f1cc00d0530 legacy=0x7f1cc00b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbe7fc640 1 -- 172.21.15.66:0/2053959128 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206901470 0 0) 0x7f1cc006e220 con 0x7f1cc00d0530 2024-03-30T13:10:44.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.820+0000 7f1cbffff640 1 -- 172.21.15.66:0/2053959128 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc015f810 con 0x7f1cc00b43e0 2024-03-30T13:10:44.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.821+0000 7f1cbffff640 1 --2- 172.21.15.66:0/2053959128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc00ce8c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009d90 tx=0x7f1cb802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6cf844188d614 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.821+0000 7f1cbe7fc640 1 -- 172.21.15.66:0/2053959128 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8039070 con 0x7f1cc00b43e0 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.821+0000 7f1cbe7fc640 1 -- 172.21.15.66:0/2053959128 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb802cc50 con 0x7f1cc00b43e0 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.821+0000 7f1cbe7fc640 1 -- 172.21.15.66:0/2053959128 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8034440 con 0x7f1cc00b43e0 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2053959128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 msgr2=0x7f1cc00ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2053959128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc00ce8c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009d90 tx=0x7f1cb802bea0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2053959128 shutdown_connections 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2053959128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc00ce8c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2053959128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0150 0x7f1cc015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2053959128 >> 172.21.15.66:0/2053959128 conn(0x7f1cc006f820 msgr2=0x7f1cc006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2053959128 shutdown_connections 2024-03-30T13:10:44.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.822+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2053959128 wait complete. 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.823+0000 7f1cc5dc3640 1 Processor -- start 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.823+0000 7f1cc5dc3640 1 -- start start 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cc5dc3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc01f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cc5dc3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc00d0150 0x7f1cc013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cc5dc3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0530 0x7f1cc0140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cc5dc3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00b4950 con 0x7f1cc00b43e0 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cc5dc3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc006ca50 con 0x7f1cc00d0150 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cc5dc3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc00b4ad0 con 0x7f1cc00d0530 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc01f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc01f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43268/0 (socket says 172.21.15.66:43268) 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 -- 172.21.15.66:0/2955390523 learned_addr learned my addr 172.21.15.66:0/2955390523 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbffff640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0530 0x7f1cc0140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbeffd640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc00d0150 0x7f1cc013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 -- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0530 msgr2=0x7f1cc0140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0530 0x7f1cc0140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 -- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc00d0150 msgr2=0x7f1cc013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc00d0150 0x7f1cc013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.824+0000 7f1cbf7fe640 1 -- 172.21.15.66:0/2955390523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc00c8a50 con 0x7f1cc00b43e0 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.825+0000 7f1cbf7fe640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc01f4ec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9e0 tx=0x7f1cb400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.825+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4017070 con 0x7f1cc00b43e0 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.825+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb400c9d0 con 0x7f1cc00b43e0 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.825+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc01f5800 con 0x7f1cc00b43e0 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.825+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400cdc0 con 0x7f1cc00b43e0 2024-03-30T13:10:44.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.825+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01f5dc0 con 0x7f1cc00b43e0 2024-03-30T13:10:44.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.826+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc0075a10 con 0x7f1cc00b43e0 2024-03-30T13:10:44.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.826+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1cb401a070 con 0x7f1cc00b43e0 2024-03-30T13:10:44.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.827+0000 7f1ca7fff640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1cac06c530 0x7f1cac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:44.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.827+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f1cb408bd90 con 0x7f1cc00b43e0 2024-03-30T13:10:44.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.827+0000 7f1cbeffd640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1cac06c530 0x7f1cac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:44.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.827+0000 7f1cbeffd640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1cac06c530 0x7f1cac06e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00cf7d0 tx=0x7f1cb0007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:44.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:44.831+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1cb40599d0 con 0x7f1cc00b43e0 2024-03-30T13:10:45.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.018+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cc00b81d0 con 0x7f1cc00b43e0 2024-03-30T13:10:45.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.019+0000 7f1ca7fff640 1 -- 172.21.15.66:0/2955390523 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f1cb405dcd0 con 0x7f1cc00b43e0 2024-03-30T13:10:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:45.023 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1cac06c530 msgr2=0x7f1cac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1cac06c530 0x7f1cac06e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00cf7d0 tx=0x7f1cb0007d50 comp rx=0 tx=0).stop 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 msgr2=0x7f1cc01f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc01f4ec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9e0 tx=0x7f1cb400eea0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 shutdown_connections 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1cac06c530 0x7f1cac06e9f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc00d0530 0x7f1cc0140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc00d0150 0x7f1cc013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 --2- 172.21.15.66:0/2955390523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cc00b43e0 0x7f1cc01f4ec0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 >> 172.21.15.66:0/2955390523 conn(0x7f1cc006f820 msgr2=0x7f1cc00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:45.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 shutdown_connections 2024-03-30T13:10:45.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.021+0000 7f1cc5dc3640 1 -- 172.21.15.66:0/2955390523 wait complete. 2024-03-30T13:10:45.036 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T13:10:45.036 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.140+0000 7f9ea5c70640 1 Processor -- start 2024-03-30T13:10:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.140+0000 7f9ea5c70640 1 -- start start 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9ea5c70640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ea00d4a90 0x7f9ea00d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9ea5c70640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d2f00 0x7f9ea00b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9ea5c70640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea00d36f0 con 0x7f9ea015e950 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9ea5c70640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea00d3cb0 con 0x7f9ea00d4a90 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9ea5c70640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea00d39d0 con 0x7f9ea00d2f00 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ea00d4a90 0x7f9ea00d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ea00d4a90 0x7f9ea00d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43276/0 (socket says 172.21.15.66:43276) 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 -- 172.21.15.66:0/1192930739 learned_addr learned my addr 172.21.15.66:0/1192930739 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9ffff640 1 --2- 172.21.15.66:0/1192930739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d2f00 0x7f9ea00b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 -- 172.21.15.66:0/1192930739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d2f00 msgr2=0x7f9ea00b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 --2- 172.21.15.66:0/1192930739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d2f00 0x7f9ea00b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 -- 172.21.15.66:0/1192930739 >> v1:172.21.15.117:6789/0 conn(0x7f9ea015e950 legacy=0x7f9ea00d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9effd640 1 -- 172.21.15.66:0/1192930739 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea00b7ac0 con 0x7f9ea00d4a90 2024-03-30T13:10:45.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.141+0000 7f9e9e7fc640 1 -- 172.21.15.66:0/1192930739 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15714961 0 0) 0x7f9ea00d36f0 con 0x7f9ea015e950 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.142+0000 7f9e9effd640 1 --2- 172.21.15.66:0/1192930739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d4a90 0x7f9ea00d29c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009870 tx=0x7f9e9402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee79e2b84e43b996 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.142+0000 7f9e9e7fc640 1 -- 172.21.15.66:0/1192930739 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9402f840 con 0x7f9ea00d4a90 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.142+0000 7f9e9e7fc640 1 -- 172.21.15.66:0/1192930739 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9402f9e0 con 0x7f9ea00d4a90 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.142+0000 7f9e9e7fc640 1 -- 172.21.15.66:0/1192930739 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9402fdb0 con 0x7f9ea00d4a90 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1192930739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d4a90 msgr2=0x7f9ea00d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1192930739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d4a90 0x7f9ea00d29c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009870 tx=0x7f9e9402ba20 comp rx=0 tx=0).stop 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1192930739 shutdown_connections 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1192930739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d2f00 0x7f9ea00b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1192930739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d4a90 0x7f9ea00d29c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1192930739 >> 172.21.15.66:0/1192930739 conn(0x7f9ea00676e0 msgr2=0x7f9ea0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1192930739 shutdown_connections 2024-03-30T13:10:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.143+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1192930739 wait complete. 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.144+0000 7f9ea5c70640 1 Processor -- start 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.144+0000 7f9ea5c70640 1 -- start start 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9ea5c70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 0x7f9ea00b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9ea5c70640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d4a90 0x7f9ea00ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9ea5c70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea015e950 0x7f9ea01f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9ea5c70640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea015ed30 con 0x7f9ea00d2f00 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9ea5c70640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea00d33b0 con 0x7f9ea015e950 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9ea5c70640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea015eeb0 con 0x7f9ea00d4a90 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 0x7f9ea00b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 0x7f9ea00b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43282/0 (socket says 172.21.15.66:43282) 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 -- 172.21.15.66:0/1857191449 learned_addr learned my addr 172.21.15.66:0/1857191449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9ffff640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea015e950 0x7f9ea01f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9effd640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d4a90 0x7f9ea00ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 -- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d4a90 msgr2=0x7f9ea00ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d4a90 0x7f9ea00ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 -- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea015e950 msgr2=0x7f9ea01f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea015e950 0x7f9ea01f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 -- 172.21.15.66:0/1857191449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea00aadf0 con 0x7f9ea00d2f00 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9ffff640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea015e950 0x7f9ea01f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e9f7fe640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 0x7f9ea00b58c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c4b0 tx=0x7f9e9000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.145+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90017070 con 0x7f9ea00d2f00 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.146+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e90005480 con 0x7f9ea00d2f00 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.146+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea00ab0d0 con 0x7f9ea00d2f00 2024-03-30T13:10:45.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.146+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90003830 con 0x7f9ea00d2f00 2024-03-30T13:10:45.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.146+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea00ab690 con 0x7f9ea00d2f00 2024-03-30T13:10:45.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.146+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea0075a10 con 0x7f9ea00d2f00 2024-03-30T13:10:45.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.147+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9e9001a070 con 0x7f9ea00d2f00 2024-03-30T13:10:45.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.147+0000 7f9e87fff640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9e8c06c530 0x7f9e8c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.148+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f9e9008aee0 con 0x7f9ea00d2f00 2024-03-30T13:10:45.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.148+0000 7f9e9effd640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9e8c06c530 0x7f9e8c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.148+0000 7f9e9effd640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9e8c06c530 0x7f9e8c06e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e94036000 tx=0x7f9e94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.152+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9e900590f0 con 0x7f9ea00d2f00 2024-03-30T13:10:45.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.374+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ea0075000 con 0x7f9ea00d2f00 2024-03-30T13:10:45.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.374+0000 7f9e87fff640 1 -- 172.21.15.66:0/1857191449 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f9e900589f0 con 0x7f9ea00d2f00 2024-03-30T13:10:45.378 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:45.378 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:45.379 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:45.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9e8c06c530 msgr2=0x7f9e8c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9e8c06c530 0x7f9e8c06e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e94036000 tx=0x7f9e94008030 comp rx=0 tx=0).stop 2024-03-30T13:10:45.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 msgr2=0x7f9ea00b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 0x7f9ea00b58c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c4b0 tx=0x7f9e9000c970 comp rx=0 tx=0).stop 2024-03-30T13:10:45.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 shutdown_connections 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9e8c06c530 0x7f9e8c06e9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea015e950 0x7f9ea01f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ea00d4a90 0x7f9ea00ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 --2- 172.21.15.66:0/1857191449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ea00d2f00 0x7f9ea00b58c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 >> 172.21.15.66:0/1857191449 conn(0x7f9ea00676e0 msgr2=0x7f9ea00b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 shutdown_connections 2024-03-30T13:10:45.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.376+0000 7f9ea5c70640 1 -- 172.21.15.66:0/1857191449 wait complete. 2024-03-30T13:10:45.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:45.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.494+0000 7f3be78c1640 1 Processor -- start 2024-03-30T13:10:45.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.495+0000 7f3be78c1640 1 -- start start 2024-03-30T13:10:45.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.495+0000 7f3be78c1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be0162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.495+0000 7f3be78c1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3be00b3cb0 0x7f3be00b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.495+0000 7f3be78c1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be00d0860 con 0x7f3be00cf8e0 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.495+0000 7f3be78c1640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3be00d21e0 con 0x7f3be00b3cb0 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.495+0000 7f3be78c1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3be00b4620 con 0x7f3be00d0480 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3be00b3cb0 0x7f3be00b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3be00b3cb0 0x7f3be00b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43290/0 (socket says 172.21.15.66:43290) 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 -- 172.21.15.66:0/4223337811 learned_addr learned my addr 172.21.15.66:0/4223337811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be565d640 1 --2- 172.21.15.66:0/4223337811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be0162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 -- 172.21.15.66:0/4223337811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 msgr2=0x7f3be0162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 --2- 172.21.15.66:0/4223337811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be0162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 -- 172.21.15.66:0/4223337811 >> v1:172.21.15.117:6789/0 conn(0x7f3be00cf8e0 legacy=0x7f3be0141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3bdffff640 1 -- 172.21.15.66:0/4223337811 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414195991 0 0) 0x7f3be00d0860 con 0x7f3be00cf8e0 2024-03-30T13:10:45.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.496+0000 7f3be5e5e640 1 -- 172.21.15.66:0/4223337811 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be0162b80 con 0x7f3be00b3cb0 2024-03-30T13:10:45.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be5e5e640 1 --2- 172.21.15.66:0/4223337811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be00b40e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009870 tx=0x7f3bd002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedd7a6c81aec06d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3bdffff640 1 -- 172.21.15.66:0/4223337811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0034070 con 0x7f3be00b3cb0 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3bdffff640 1 -- 172.21.15.66:0/4223337811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd0030840 con 0x7f3be00b3cb0 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3bdffff640 1 -- 172.21.15.66:0/4223337811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0030c10 con 0x7f3be00b3cb0 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 -- 172.21.15.66:0/4223337811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 msgr2=0x7f3be00b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 --2- 172.21.15.66:0/4223337811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be00b40e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009870 tx=0x7f3bd002b980 comp rx=0 tx=0).stop 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 -- 172.21.15.66:0/4223337811 shutdown_connections 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 --2- 172.21.15.66:0/4223337811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be00b40e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 --2- 172.21.15.66:0/4223337811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be0162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 -- 172.21.15.66:0/4223337811 >> 172.21.15.66:0/4223337811 conn(0x7f3be00672c0 msgr2=0x7f3be00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 -- 172.21.15.66:0/4223337811 shutdown_connections 2024-03-30T13:10:45.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.497+0000 7f3be78c1640 1 -- 172.21.15.66:0/4223337811 wait complete. 2024-03-30T13:10:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 Processor -- start 2024-03-30T13:10:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 -- start start 2024-03-30T13:10:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be0147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be00b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be01f7230 0x7f3be00c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3be00d1e70 con 0x7f3be00b3cb0 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0163140 con 0x7f3be01f7230 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be78c1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3be01424b0 con 0x7f3be00d0480 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be565d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be0147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be565d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be0147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43298/0 (socket says 172.21.15.66:43298) 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be565d640 1 -- 172.21.15.66:0/2999390541 learned_addr learned my addr 172.21.15.66:0/2999390541 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be5e5e640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be01f7230 0x7f3be00c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.499+0000 7f3be4e5c640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be00b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be565d640 1 -- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 msgr2=0x7f3be00b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be565d640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be00b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be565d640 1 -- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be01f7230 msgr2=0x7f3be00c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be565d640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be01f7230 0x7f3be00c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be565d640 1 -- 172.21.15.66:0/2999390541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be00b6510 con 0x7f3be00b3cb0 2024-03-30T13:10:45.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be565d640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be0147cb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800efc0 tx=0x7f3bd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8018020 con 0x7f3be00b3cb0 2024-03-30T13:10:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd8014480 con 0x7f3be00b3cb0 2024-03-30T13:10:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be00b67f0 con 0x7f3be00b3cb0 2024-03-30T13:10:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8014850 con 0x7f3be00b3cb0 2024-03-30T13:10:45.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.500+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be00b6db0 con 0x7f3be00b3cb0 2024-03-30T13:10:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.501+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0075a10 con 0x7f3be00b3cb0 2024-03-30T13:10:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.502+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3bd8005d10 con 0x7f3be00b3cb0 2024-03-30T13:10:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.502+0000 7f3bddffb640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bcc06c530 0x7f3bcc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.502+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f3bd808c460 con 0x7f3be00b3cb0 2024-03-30T13:10:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.502+0000 7f3be4e5c640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bcc06c530 0x7f3bcc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.503+0000 7f3be4e5c640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bcc06c530 0x7f3bcc06e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3be00c5940 tx=0x7f3bd4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.506+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3bd8022070 con 0x7f3be00b3cb0 2024-03-30T13:10:45.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.730+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3be00cf5a0 con 0x7f3be00b3cb0 2024-03-30T13:10:45.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.731+0000 7f3bddffb640 1 -- 172.21.15.66:0/2999390541 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f3bd8010040 con 0x7f3be00b3cb0 2024-03-30T13:10:45.735 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:45.735 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1230168196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1230168196},{"type":"v1","addr":"172.21.15.117:6834","nonce":1230168196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1248991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1248991767},{"type":"v1","addr":"172.21.15.66:6835","nonce":1248991767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1605941326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1605941326},{"type":"v1","addr":"172.21.15.66:6837","nonce":1605941326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:09:42.480275+0000","modified":"2024-03-30T13:10:04.521329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/4158016355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":4158016355},{"type":"v1","addr":"172.21.15.117:6835","nonce":4158016355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-30T13:10:45.735 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-03-30T13:10:45.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bcc06c530 msgr2=0x7f3bcc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bcc06c530 0x7f3bcc06e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3be00c5940 tx=0x7f3bd4009040 comp rx=0 tx=0).stop 2024-03-30T13:10:45.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 msgr2=0x7f3be0147cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be0147cb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800efc0 tx=0x7f3bd800c490 comp rx=0 tx=0).stop 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 shutdown_connections 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bcc06c530 0x7f3bcc06e9f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be01f7230 0x7f3be00c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be00d0480 0x7f3be00b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 --2- 172.21.15.66:0/2999390541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3be00b3cb0 0x7f3be0147cb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 >> 172.21.15.66:0/2999390541 conn(0x7f3be00672c0 msgr2=0x7f3be00ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.733+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 shutdown_connections 2024-03-30T13:10:45.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.734+0000 7f3be78c1640 1 -- 172.21.15.66:0/2999390541 wait complete. 2024-03-30T13:10:45.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:45.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.852+0000 7f54f080d640 1 Processor -- start 2024-03-30T13:10:45.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.852+0000 7f54f080d640 1 -- start start 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.853+0000 7f54f080d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54ec15b270 0x7f54ec15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.853+0000 7f54f080d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54ec072280 0x7f54ec0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.853+0000 7f54f080d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54ec15b650 con 0x7f54ec0ccf10 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.853+0000 7f54f080d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f54ec0cd380 con 0x7f54ec072280 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.853+0000 7f54f080d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f54ec072ab0 con 0x7f54ec15b270 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54ec072280 0x7f54ec0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54ec072280 0x7f54ec0cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43314/0 (socket says 172.21.15.66:43314) 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3986154013 learned_addr learned my addr 172.21.15.66:0/3986154013 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54ea59c640 1 --2- 172.21.15.66:0/3986154013 >> v2:172.21.15.117:3300/0 conn(0x7f54ec15b270 0x7f54ec15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3986154013 >> v2:172.21.15.117:3300/0 conn(0x7f54ec15b270 msgr2=0x7f54ec15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 --2- 172.21.15.66:0/3986154013 >> v2:172.21.15.117:3300/0 conn(0x7f54ec15b270 0x7f54ec15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3986154013 >> v1:172.21.15.117:6789/0 conn(0x7f54ec0ccf10 legacy=0x7f54ec0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3986154013 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec072d90 con 0x7f54ec072280 2024-03-30T13:10:45.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.854+0000 7f54e959a640 1 -- 172.21.15.66:0/3986154013 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877764434 0 0) 0x7f54ec15b650 con 0x7f54ec0ccf10 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54e9d9b640 1 --2- 172.21.15.66:0/3986154013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec072280 0x7f54ec0cc9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f54ec06fa10 tx=0x7f54dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90e420aa9ed3122 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54e959a640 1 -- 172.21.15.66:0/3986154013 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc034070 con 0x7f54ec072280 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54e959a640 1 -- 172.21.15.66:0/3986154013 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc030860 con 0x7f54ec072280 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54e959a640 1 -- 172.21.15.66:0/3986154013 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc030c30 con 0x7f54ec072280 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 -- 172.21.15.66:0/3986154013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec072280 msgr2=0x7f54ec0cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 --2- 172.21.15.66:0/3986154013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec072280 0x7f54ec0cc9d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f54ec06fa10 tx=0x7f54dc02b9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 -- 172.21.15.66:0/3986154013 shutdown_connections 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 --2- 172.21.15.66:0/3986154013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec072280 0x7f54ec0cc9d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 --2- 172.21.15.66:0/3986154013 >> v2:172.21.15.117:3300/0 conn(0x7f54ec15b270 0x7f54ec15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 -- 172.21.15.66:0/3986154013 >> 172.21.15.66:0/3986154013 conn(0x7f54ec0b1c90 msgr2=0x7f54ec0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 -- 172.21.15.66:0/3986154013 shutdown_connections 2024-03-30T13:10:45.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.855+0000 7f54f080d640 1 -- 172.21.15.66:0/3986154013 wait complete. 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 Processor -- start 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 -- start start 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec072280 0x7f54ec14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 0x7f54ec0b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b270 0x7f54ec0cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec073630 con 0x7f54ec0ccf10 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec073350 con 0x7f54ec15b270 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.857+0000 7f54f080d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec0726e0 con 0x7f54ec072280 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 0x7f54ec0b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 0x7f54ec0b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43324/0 (socket says 172.21.15.66:43324) 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3059955091 learned_addr learned my addr 172.21.15.66:0/3059955091 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:45.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54ea59c640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec072280 0x7f54ec14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54ead9d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b270 0x7f54ec0cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec072280 msgr2=0x7f54ec14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec072280 0x7f54ec14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b270 msgr2=0x7f54ec0cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b270 0x7f54ec0cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 -- 172.21.15.66:0/3059955091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec14ae10 con 0x7f54ec0ccf10 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54ea59c640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec072280 0x7f54ec14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e9d9b640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 0x7f54ec0b5300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f54dc02b9e0 tx=0x7f54dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc048070 con 0x7f54ec0ccf10 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc030660 con 0x7f54ec0ccf10 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec0b2e40 con 0x7f54ec0ccf10 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc034040 con 0x7f54ec0ccf10 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.858+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec0b3400 con 0x7f54ec0ccf10 2024-03-30T13:10:45.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.859+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ec0cee00 con 0x7f54ec0ccf10 2024-03-30T13:10:45.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.860+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f54dc04b070 con 0x7f54ec0ccf10 2024-03-30T13:10:45.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.860+0000 7f54e2ffd640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f54cc06c530 0x7f54cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:45.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.860+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f54dc0ad9a0 con 0x7f54ec0ccf10 2024-03-30T13:10:45.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.860+0000 7f54ea59c640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f54cc06c530 0x7f54cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:45.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.861+0000 7f54ea59c640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f54cc06c530 0x7f54cc06e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009e90 tx=0x7f54d4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:45.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:45.865+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f54ec0cee00 con 0x7f54ec0ccf10 2024-03-30T13:10:46.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.050+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54ec075a10 con 0x7f54ec0ccf10 2024-03-30T13:10:46.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.051+0000 7f54e2ffd640 1 -- 172.21.15.66:0/3059955091 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18938 (secure 0 0 0) 0x7f54dc07b550 con 0x7f54ec0ccf10 2024-03-30T13:10:46.055 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:46.055 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:24.220860+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:09:40.767113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:09:41.426231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:09:45.120738+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f54cc06c530 msgr2=0x7f54cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f54cc06c530 0x7f54cc06e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009e90 tx=0x7f54d4009190 comp rx=0 tx=0).stop 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 msgr2=0x7f54ec0b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 0x7f54ec0b5300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f54dc02b9e0 tx=0x7f54dc004730 comp rx=0 tx=0).stop 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 shutdown_connections 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f54cc06c530 0x7f54cc06e9f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b270 0x7f54ec0cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54ec0ccf10 0x7f54ec0b5300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 --2- 172.21.15.66:0/3059955091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec072280 0x7f54ec14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 >> 172.21.15.66:0/3059955091 conn(0x7f54ec0b1c90 msgr2=0x7f54ec0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:46.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 shutdown_connections 2024-03-30T13:10:46.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.053+0000 7f54f080d640 1 -- 172.21.15.66:0/3059955091 wait complete. 2024-03-30T13:10:46.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T13:10:46.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.173+0000 7f5570870640 1 Processor -- start 2024-03-30T13:10:46.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.173+0000 7f5570870640 1 -- start start 2024-03-30T13:10:46.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.174+0000 7f5570870640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f556c0d1300 0x7f556c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.174+0000 7f5570870640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f556c0d3ee0 0x7f556c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.174+0000 7f5570870640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f556c0cfc00 con 0x7f556c0b38f0 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.174+0000 7f5570870640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f556c0d1c20 con 0x7f556c0d3ee0 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.174+0000 7f5570870640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f556c0cf930 con 0x7f556c0d1300 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.174+0000 7f5569d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f556c0d3ee0 0x7f556c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556ad9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f556c0b38f0 0x7f556c0b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42826/0 (socket says 172.21.15.66:42826) 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556ad9d640 1 -- 172.21.15.66:0/1555471139 learned_addr learned my addr 172.21.15.66:0/1555471139 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556a59c640 1 --2- 172.21.15.66:0/1555471139 >> v2:172.21.15.117:3300/0 conn(0x7f556c0d1300 0x7f556c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556a59c640 1 -- 172.21.15.66:0/1555471139 >> v2:172.21.15.66:3300/0 conn(0x7f556c0d3ee0 msgr2=0x7f556c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556a59c640 1 --2- 172.21.15.66:0/1555471139 >> v2:172.21.15.66:3300/0 conn(0x7f556c0d3ee0 0x7f556c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556a59c640 1 -- 172.21.15.66:0/1555471139 >> v1:172.21.15.66:6789/0 conn(0x7f556c0b38f0 legacy=0x7f556c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f556a59c640 1 -- 172.21.15.66:0/1555471139 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c0d3120 con 0x7f556c0d1300 2024-03-30T13:10:46.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.175+0000 7f5569d9b640 1 --2- 172.21.15.66:0/1555471139 >> v2:172.21.15.66:3300/0 conn(0x7f556c0d3ee0 0x7f556c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:46.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f556a59c640 1 --2- 172.21.15.66:0/1555471139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d1300 0x7f556c0d16e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f556c06d5f0 tx=0x7f556402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f155d2f8addf86b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:46.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f556959a640 1 -- 172.21.15.66:0/1555471139 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564035020 con 0x7f556c0d1300 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f556959a640 1 -- 172.21.15.66:0/1555471139 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5564031830 con 0x7f556c0d1300 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f556959a640 1 -- 172.21.15.66:0/1555471139 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564031c00 con 0x7f556c0d1300 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 -- 172.21.15.66:0/1555471139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d1300 msgr2=0x7f556c0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 --2- 172.21.15.66:0/1555471139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d1300 0x7f556c0d16e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f556c06d5f0 tx=0x7f556402b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 -- 172.21.15.66:0/1555471139 shutdown_connections 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 --2- 172.21.15.66:0/1555471139 >> v2:172.21.15.66:3300/0 conn(0x7f556c0d3ee0 0x7f556c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 --2- 172.21.15.66:0/1555471139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d1300 0x7f556c0d16e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 -- 172.21.15.66:0/1555471139 >> 172.21.15.66:0/1555471139 conn(0x7f556c0676e0 msgr2=0x7f556c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 -- 172.21.15.66:0/1555471139 shutdown_connections 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.176+0000 7f5570870640 1 -- 172.21.15.66:0/1555471139 wait complete. 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 Processor -- start 2024-03-30T13:10:46.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 -- start start 2024-03-30T13:10:46.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c0b38f0 0x7f556c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 0x7f556c15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d3ee0 0x7f556c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0d39c0 con 0x7f556c0d1300 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0d36e0 con 0x7f556c0d3ee0 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.178+0000 7f5570870640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f556c15e960 con 0x7f556c0b38f0 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 0x7f556c15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 0x7f556c15ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43348/0 (socket says 172.21.15.66:43348) 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 -- 172.21.15.66:0/748875695 learned_addr learned my addr 172.21.15.66:0/748875695 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f556a59c640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c0b38f0 0x7f556c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f556ad9d640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d3ee0 0x7f556c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 -- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c0b38f0 msgr2=0x7f556c14b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c0b38f0 0x7f556c14b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 -- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d3ee0 msgr2=0x7f556c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d3ee0 0x7f556c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 -- 172.21.15.66:0/748875695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c0ccc50 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5569d9b640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 0x7f556c15ec60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f555400ebe0 tx=0x7f555400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554017070 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555400cc30 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c0ccf30 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554013470 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.179+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c0cd4f0 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.181+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c075900 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.181+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f555401a070 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.181+0000 7f5562ffd640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f554c06c560 0x7f554c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.181+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6764+0+0 (secure 0 0 0) 0x7f555408bd00 con 0x7f556c0d1300 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.181+0000 7f556a59c640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f554c06c560 0x7f554c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.182+0000 7f556a59c640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f554c06c560 0x7f554c06ea20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5564036000 tx=0x7f5564008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.186+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f55540598b0 con 0x7f556c0d1300 2024-03-30T13:10:46.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.407+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f556c0cf1d0 con 0x7f556c0d1300 2024-03-30T13:10:46.411 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T13:10:46.408+0000 7f1a04250640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T13:10:46.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.409+0000 7f5562ffd640 1 -- 172.21.15.66:0/748875695 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f55540600d0 con 0x7f556c0d1300 2024-03-30T13:10:46.413 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T13:10:46.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f554c06c560 msgr2=0x7f554c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f554c06c560 0x7f554c06ea20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5564036000 tx=0x7f5564008030 comp rx=0 tx=0).stop 2024-03-30T13:10:46.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 msgr2=0x7f556c15ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 0x7f556c15ec60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f555400ebe0 tx=0x7f555400c520 comp rx=0 tx=0).stop 2024-03-30T13:10:46.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 shutdown_connections 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f554c06c560 0x7f554c06ea20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c0d3ee0 0x7f556c0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c0d1300 0x7f556c15ec60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 --2- 172.21.15.66:0/748875695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c0b38f0 0x7f556c14b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 >> 172.21.15.66:0/748875695 conn(0x7f556c0676e0 msgr2=0x7f556c0c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 shutdown_connections 2024-03-30T13:10:46.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.411+0000 7f5570870640 1 -- 172.21.15.66:0/748875695 wait complete. 2024-03-30T13:10:46.417 INFO:tasks.ceph.mds.d.smithi117.stderr: -5506> 2024-03-30T13:09:47.222+0000 7f2d77caa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T13:10:46.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T13:10:46.449 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-30T13:10:46.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.532+0000 7f5022700640 1 Processor -- start 2024-03-30T13:10:46.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.532+0000 7f5022700640 1 -- start start 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f5022700640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f501c0b5dc0 0x7f501c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f5022700640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f501c0b73d0 0x7f501c0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f5022700640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f501c0b66e0 con 0x7f501c0ba090 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f5022700640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f501c0d2d90 con 0x7f501c0b5dc0 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f5022700640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f501c0d2ff0 con 0x7f501c0b73d0 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f501bfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f501c0b5dc0 0x7f501c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f501b7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f501c0ba090 0x7f501c0d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42832/0 (socket says 172.21.15.66:42832) 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.533+0000 7f501b7fe640 1 -- 172.21.15.66:0/482712271 learned_addr learned my addr 172.21.15.66:0/482712271 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.534+0000 7f5020c9d640 1 --2- 172.21.15.66:0/482712271 >> v2:172.21.15.117:3300/0 conn(0x7f501c0b73d0 0x7f501c0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.534+0000 7f501bfff640 1 -- 172.21.15.66:0/482712271 >> v2:172.21.15.117:3300/0 conn(0x7f501c0b73d0 msgr2=0x7f501c0b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.534+0000 7f501bfff640 1 --2- 172.21.15.66:0/482712271 >> v2:172.21.15.117:3300/0 conn(0x7f501c0b73d0 0x7f501c0b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.534+0000 7f501bfff640 1 -- 172.21.15.66:0/482712271 >> v1:172.21.15.66:6789/0 conn(0x7f501c0ba090 legacy=0x7f501c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.534+0000 7f501bfff640 1 -- 172.21.15.66:0/482712271 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c0d32d0 con 0x7f501c0b5dc0 2024-03-30T13:10:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.534+0000 7f501affd640 1 -- 172.21.15.66:0/482712271 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569596027 0 0) 0x7f501c0b66e0 con 0x7f501c0ba090 2024-03-30T13:10:46.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f501bfff640 1 --2- 172.21.15.66:0/482712271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c0b61a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f501c0d5420 tx=0x7f500c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840ca3d6a380727f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f501affd640 1 -- 172.21.15.66:0/482712271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c035020 con 0x7f501c0b5dc0 2024-03-30T13:10:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f501affd640 1 -- 172.21.15.66:0/482712271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c031830 con 0x7f501c0b5dc0 2024-03-30T13:10:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f501affd640 1 -- 172.21.15.66:0/482712271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c031c00 con 0x7f501c0b5dc0 2024-03-30T13:10:46.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 -- 172.21.15.66:0/482712271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 msgr2=0x7f501c0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 --2- 172.21.15.66:0/482712271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c0b61a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f501c0d5420 tx=0x7f500c02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 -- 172.21.15.66:0/482712271 shutdown_connections 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 --2- 172.21.15.66:0/482712271 >> v2:172.21.15.117:3300/0 conn(0x7f501c0b73d0 0x7f501c0b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 --2- 172.21.15.66:0/482712271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c0b61a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 -- 172.21.15.66:0/482712271 >> 172.21.15.66:0/482712271 conn(0x7f501c0678f0 msgr2=0x7f501c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 -- 172.21.15.66:0/482712271 shutdown_connections 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.535+0000 7f5022700640 1 -- 172.21.15.66:0/482712271 wait complete. 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.536+0000 7f5022700640 1 Processor -- start 2024-03-30T13:10:46.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 -- start start 2024-03-30T13:10:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c1f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f501c0b73d0 0x7f501c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f501c0ba090 0x7f501c0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0d3b70 con 0x7f501c0b5dc0 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0d3890 con 0x7f501c0ba090 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f5022700640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f501c0d4090 con 0x7f501c0b73d0 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.537+0000 7f501bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c1f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c1f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43372/0 (socket says 172.21.15.66:43372) 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 -- 172.21.15.66:0/3318378814 learned_addr learned my addr 172.21.15.66:0/3318378814 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f5020c9d640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f501c0ba090 0x7f501c0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501b7fe640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f501c0b73d0 0x7f501c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 -- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f501c0b73d0 msgr2=0x7f501c1603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f501c0b73d0 0x7f501c1603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 -- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f501c0ba090 msgr2=0x7f501c0ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f501c0ba090 0x7f501c0ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:46.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 -- 172.21.15.66:0/3318378814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c0ab180 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f501bfff640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c1f4730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f500c009890 tx=0x7f500c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c041070 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c03b920 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c0ab460 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c035040 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.538+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c0aba20 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.539+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c075890 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.540+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f500c038020 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.540+0000 7f5018ff9640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f500806c530 0x7f500806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.540+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6850+0+0 (secure 0 0 0) 0x7f500c0ad3b0 con 0x7f501c0b5dc0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.540+0000 7f501b7fe640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f500806c530 0x7f500806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:46.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.540+0000 7f501b7fe640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f500806c530 0x7f500806e9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f501c0d5390 tx=0x7f5010007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:46.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.544+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f500c031800 con 0x7f501c0b5dc0 2024-03-30T13:10:46.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:46.766+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f501c0d2890 con 0x7f501c0b5dc0 2024-03-30T13:10:47.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.414+0000 7f5018ff9640 1 -- 172.21.15.66:0/3318378814 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f500c07af00 con 0x7f501c0b5dc0 2024-03-30T13:10:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f500806c530 msgr2=0x7f500806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f500806c530 0x7f500806e9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f501c0d5390 tx=0x7f5010007de0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 msgr2=0x7f501c1f4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c1f4730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f500c009890 tx=0x7f500c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 shutdown_connections 2024-03-30T13:10:47.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f500806c530 0x7f500806e9f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f501c0ba090 0x7f501c0ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f501c0b73d0 0x7f501c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 --2- 172.21.15.66:0/3318378814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f501c0b5dc0 0x7f501c1f4730 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.416+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 >> 172.21.15.66:0/3318378814 conn(0x7f501c0678f0 msgr2=0x7f501c15d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.417+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 shutdown_connections 2024-03-30T13:10:47.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.417+0000 7f5022700640 1 -- 172.21.15.66:0/3318378814 wait complete. 2024-03-30T13:10:47.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T13:10:47.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.544+0000 7f1ef518d640 1 Processor -- start 2024-03-30T13:10:47.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.544+0000 7f1ef518d640 1 -- start start 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1ef518d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1ef518d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ef006d4a0 0x7f1ef00d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1ef518d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef00d2910 con 0x7f1ef00d3ef0 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1ef518d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef00d3520 con 0x7f1ef006d4a0 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1ef518d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef00d2bf0 con 0x7f1ef015e440 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1eee59c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ef006d4a0 0x7f1ef00d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:47.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1eee59c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ef006d4a0 0x7f1ef00d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43388/0 (socket says 172.21.15.66:43388) 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.545+0000 7f1eee59c640 1 -- 172.21.15.66:0/31696160 learned_addr learned my addr 172.21.15.66:0/31696160 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.546+0000 7f1eeed9d640 1 --2- 172.21.15.66:0/31696160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.546+0000 7f1eee59c640 1 -- 172.21.15.66:0/31696160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 msgr2=0x7f1ef00b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.546+0000 7f1eee59c640 1 --2- 172.21.15.66:0/31696160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.546+0000 7f1eee59c640 1 -- 172.21.15.66:0/31696160 >> v1:172.21.15.117:6789/0 conn(0x7f1ef00d3ef0 legacy=0x7f1ef00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.546+0000 7f1eee59c640 1 -- 172.21.15.66:0/31696160 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef00d2ed0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.546+0000 7f1eee59c640 1 --2- 172.21.15.66:0/31696160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef00d39b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009870 tx=0x7f1ed802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3928a0754fdb4065 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1eedd9b640 1 -- 172.21.15.66:0/31696160 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed802f830 con 0x7f1ef006d4a0 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1eedd9b640 1 -- 172.21.15.66:0/31696160 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed802f9d0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1eedd9b640 1 -- 172.21.15.66:0/31696160 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed802fda0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 -- 172.21.15.66:0/31696160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 msgr2=0x7f1ef00d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:47.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 --2- 172.21.15.66:0/31696160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef00d39b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009870 tx=0x7f1ed802ba10 comp rx=0 tx=0).stop 2024-03-30T13:10:47.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 -- 172.21.15.66:0/31696160 shutdown_connections 2024-03-30T13:10:47.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 --2- 172.21.15.66:0/31696160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef00d39b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 --2- 172.21.15.66:0/31696160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 -- 172.21.15.66:0/31696160 >> 172.21.15.66:0/31696160 conn(0x7f1ef00676b0 msgr2=0x7f1ef0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:47.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 -- 172.21.15.66:0/31696160 shutdown_connections 2024-03-30T13:10:47.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.547+0000 7f1ef518d640 1 -- 172.21.15.66:0/31696160 wait complete. 2024-03-30T13:10:47.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 Processor -- start 2024-03-30T13:10:47.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 -- start start 2024-03-30T13:10:47.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef01611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ef015d730 0x7f1ef015f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef015e820 con 0x7f1ef006d4a0 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef00ba220 con 0x7f1ef015d730 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1ef518d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef015e9a0 con 0x7f1ef015e440 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1eeed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef01611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1eeed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef01611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43398/0 (socket says 172.21.15.66:43398) 2024-03-30T13:10:47.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.549+0000 7f1eeed9d640 1 -- 172.21.15.66:0/786572021 learned_addr learned my addr 172.21.15.66:0/786572021 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eee59c640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eeed9d640 1 -- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 msgr2=0x7f1ef00cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eeed9d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eeed9d640 1 -- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ef015d730 msgr2=0x7f1ef015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eef59e640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ef015d730 0x7f1ef015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eeed9d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ef015d730 0x7f1ef015f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eeed9d640 1 -- 172.21.15.66:0/786572021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef01618b0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1eeed9d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef01611f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d5f0 tx=0x7f1ee800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80125f0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8019070 con 0x7f1ef006d4a0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef00cd0b0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80129d0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.550+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef015dbc0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.551+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef00cfd30 con 0x7f1ef006d4a0 2024-03-30T13:10:47.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.552+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1ee8012b70 con 0x7f1ef006d4a0 2024-03-30T13:10:47.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.552+0000 7f1ee77fe640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ed006c560 0x7f1ed006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:47.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.552+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6850+0+0 (secure 0 0 0) 0x7f1ee808bfa0 con 0x7f1ef006d4a0 2024-03-30T13:10:47.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.552+0000 7f1eee59c640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ed006c560 0x7f1ed006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:47.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.553+0000 7f1eee59c640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ed006c560 0x7f1ed006ea20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8036000 tx=0x7f1ed8039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:47.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.556+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1ef00cfd30 con 0x7f1ef006d4a0 2024-03-30T13:10:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:47.740+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ef0068d50 con 0x7f1ef006d4a0 2024-03-30T13:10:48.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.416+0000 7f1ee77fe640 1 -- 172.21.15.66:0/786572021 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f1ef0068d50 con 0x7f1ef006d4a0 2024-03-30T13:10:48.420 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T13:10:48.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ed006c560 msgr2=0x7f1ed006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ed006c560 0x7f1ed006ea20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8036000 tx=0x7f1ed8039040 comp rx=0 tx=0).stop 2024-03-30T13:10:48.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 msgr2=0x7f1ef01611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef01611f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d5f0 tx=0x7f1ee800dab0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 shutdown_connections 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ed006c560 0x7f1ed006ea20 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8036000 tx=0x7f1ed8039040 comp rx=0 tx=0).stop 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ef015d730 0x7f1ef015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ef015e440 0x7f1ef00cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 --2- 172.21.15.66:0/786572021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef006d4a0 0x7f1ef01611f0 secure :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d5f0 tx=0x7f1ee800dab0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.419+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 >> 172.21.15.66:0/786572021 conn(0x7f1ef00676b0 msgr2=0x7f1ef00b7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.420+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 shutdown_connections 2024-03-30T13:10:48.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.420+0000 7f1ef518d640 1 -- 172.21.15.66:0/786572021 wait complete. 2024-03-30T13:10:48.439 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T13:10:48.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.554+0000 7fc61bbe2640 1 Processor -- start 2024-03-30T13:10:48.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.554+0000 7fc61bbe2640 1 -- start start 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61bbe2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc614071070 0x7fc614071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61bbe2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc6140ce760 0x7fc6141624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61bbe2640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc614071990 con 0x7fc6140d1b80 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61bbe2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140cc900 con 0x7fc6140ce760 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61bbe2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140ccbe0 con 0x7fc614071070 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61917d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc6140ce760 0x7fc6141624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61a17f640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc6140d1b80 0x7fc61415e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42834/0 (socket says 172.21.15.66:42834) 2024-03-30T13:10:48.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61a17f640 1 -- 172.21.15.66:0/63797877 learned_addr learned my addr 172.21.15.66:0/63797877 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:48.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61997e640 1 --2- 172.21.15.66:0/63797877 >> v2:172.21.15.117:3300/0 conn(0x7fc614071070 0x7fc614071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61917d640 1 -- 172.21.15.66:0/63797877 >> v2:172.21.15.117:3300/0 conn(0x7fc614071070 msgr2=0x7fc614071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:48.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61917d640 1 --2- 172.21.15.66:0/63797877 >> v2:172.21.15.117:3300/0 conn(0x7fc614071070 0x7fc614071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61917d640 1 -- 172.21.15.66:0/63797877 >> v1:172.21.15.66:6789/0 conn(0x7fc6140d1b80 legacy=0x7fc61415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61897c640 1 -- 172.21.15.66:0/63797877 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775420923 0 0) 0x7fc614071990 con 0x7fc6140d1b80 2024-03-30T13:10:48.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.555+0000 7fc61917d640 1 -- 172.21.15.66:0/63797877 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140ccec0 con 0x7fc6140ce760 2024-03-30T13:10:48.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.556+0000 7fc61917d640 1 --2- 172.21.15.66:0/63797877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6140ce760 0x7fc6141624d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc6140757a0 tx=0x7fc60402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89034e58cc0e472a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:48.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.556+0000 7fc61897c640 1 -- 172.21.15.66:0/63797877 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604035020 con 0x7fc6140ce760 2024-03-30T13:10:48.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.556+0000 7fc61897c640 1 -- 172.21.15.66:0/63797877 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604031830 con 0x7fc6140ce760 2024-03-30T13:10:48.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.556+0000 7fc61897c640 1 -- 172.21.15.66:0/63797877 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604031c00 con 0x7fc6140ce760 2024-03-30T13:10:48.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 -- 172.21.15.66:0/63797877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6140ce760 msgr2=0x7fc6141624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/63797877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6140ce760 0x7fc6141624d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc6140757a0 tx=0x7fc60402b8d0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 -- 172.21.15.66:0/63797877 shutdown_connections 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/63797877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6140ce760 0x7fc6141624d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/63797877 >> v2:172.21.15.117:3300/0 conn(0x7fc614071070 0x7fc614071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 -- 172.21.15.66:0/63797877 >> 172.21.15.66:0/63797877 conn(0x7fc6140d2110 msgr2=0x7fc6140d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 -- 172.21.15.66:0/63797877 shutdown_connections 2024-03-30T13:10:48.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.557+0000 7fc61bbe2640 1 -- 172.21.15.66:0/63797877 wait complete. 2024-03-30T13:10:48.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.558+0000 7fc61bbe2640 1 Processor -- start 2024-03-30T13:10:48.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.558+0000 7fc61bbe2640 1 -- start start 2024-03-30T13:10:48.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61bbe2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 0x7fc6140c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61bbe2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6140ce760 0x7fc614151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61bbe2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6140d1b80 0x7fc6141516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61bbe2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc61415ee80 con 0x7fc614071070 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61bbe2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140704a0 con 0x7fc6140d1b80 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61bbe2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc61415f000 con 0x7fc6140ce760 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 0x7fc6140c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 0x7fc6140c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43424/0 (socket says 172.21.15.66:43424) 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 -- 172.21.15.66:0/4020759775 learned_addr learned my addr 172.21.15.66:0/4020759775 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61917d640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6140ce760 0x7fc614151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61a17f640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6140d1b80 0x7fc6141516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 -- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6140ce760 msgr2=0x7fc614151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6140ce760 0x7fc614151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 -- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6140d1b80 msgr2=0x7fc6141516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6140d1b80 0x7fc6141516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.559+0000 7fc61997e640 1 -- 172.21.15.66:0/4020759775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6141f4ef0 con 0x7fc614071070 2024-03-30T13:10:48.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.560+0000 7fc61997e640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 0x7fc6140c7ea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00d830 tx=0x7fc60c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.560+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00b6c0 con 0x7fc614071070 2024-03-30T13:10:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.560+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c019070 con 0x7fc614071070 2024-03-30T13:10:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.560+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6141f51d0 con 0x7fc614071070 2024-03-30T13:10:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.560+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00ba80 con 0x7fc614071070 2024-03-30T13:10:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.560+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6141f5790 con 0x7fc614071070 2024-03-30T13:10:48.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.561+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614075900 con 0x7fc614071070 2024-03-30T13:10:48.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.561+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc60c00bc20 con 0x7fc614071070 2024-03-30T13:10:48.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.562+0000 7fc60a7fc640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc5f806c530 0x7fc5f806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.562+0000 7fc61917d640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc5f806c530 0x7fc5f806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.562+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6412+0+0 (secure 0 0 0) 0x7fc60c08be90 con 0x7fc614071070 2024-03-30T13:10:48.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.562+0000 7fc61917d640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc5f806c530 0x7fc5f806e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc604036000 tx=0x7fc604008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:48.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.566+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc60c05a1f0 con 0x7fc614071070 2024-03-30T13:10:48.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:48.754+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc614071e20 con 0x7fc614071070 2024-03-30T13:10:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.420+0000 7fc60a7fc640 1 -- 172.21.15.66:0/4020759775 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7fc60c059af0 con 0x7fc614071070 2024-03-30T13:10:49.424 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T13:10:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.426+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc5f806c530 msgr2=0x7fc5f806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.426+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc5f806c530 0x7fc5f806e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc604036000 tx=0x7fc604008030 comp rx=0 tx=0).stop 2024-03-30T13:10:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.426+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 msgr2=0x7fc6140c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.426+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 0x7fc6140c7ea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00d830 tx=0x7fc60c00dcf0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.428+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 shutdown_connections 2024-03-30T13:10:49.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.428+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc5f806c530 0x7fc5f806e9f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.428+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6140d1b80 0x7fc6141516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.428+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6140ce760 0x7fc614151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.428+0000 7fc61bbe2640 1 --2- 172.21.15.66:0/4020759775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc614071070 0x7fc6140c7ea0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.428+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 >> 172.21.15.66:0/4020759775 conn(0x7fc6140d2110 msgr2=0x7fc6140b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.429+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 shutdown_connections 2024-03-30T13:10:49.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.429+0000 7fc61bbe2640 1 -- 172.21.15.66:0/4020759775 wait complete. 2024-03-30T13:10:49.447 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-30T13:10:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.557+0000 7f1ca59fd640 1 Processor -- start 2024-03-30T13:10:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.557+0000 7f1ca59fd640 1 -- start start 2024-03-30T13:10:49.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1ca59fd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 0x7f1ca00cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1ca59fd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00cc3a0 con 0x7f1ca00cfa20 2024-03-30T13:10:49.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1ca59fd640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00cfee0 con 0x7f1ca00d10b0 2024-03-30T13:10:49.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1ca59fd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca00ce0d0 con 0x7f1ca00ccd90 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9f7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1ca00cfa20 0x7f1ca00c7450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42840/0 (socket says 172.21.15.66:42840) 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9f7fe640 1 -- 172.21.15.66:0/356233392 learned_addr learned my addr 172.21.15.66:0/356233392 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9effd640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 0x7f1ca00cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1183164408 0 0) 0x7f1c98001490 con 0x7f1ca00cfa20 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c8c003520 con 0x7f1ca00cfa20 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289677525 0 0) 0x7f1ca00cfee0 con 0x7f1ca00d10b0 2024-03-30T13:10:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c98001490 con 0x7f1ca00d10b0 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1376589462 0 0) 0x7f1c8c003520 con 0x7f1ca00cfa20 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 msgr2=0x7f1ca00cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 0x7f1ca00cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 >> v1:172.21.15.117:6789/0 conn(0x7f1ca00d10b0 legacy=0x7f1ca00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca00ce3b0 con 0x7f1ca00cfa20 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f1c98001bc0 con 0x7f1ca00cfa20 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 >> v1:172.21.15.66:6789/0 conn(0x7f1ca00cfa20 legacy=0x7f1ca00c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1c8c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.558+0000 7f1c9e7fc640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 0x7f1c8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9e7fc640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c004d40 0x7f1c8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca00ce0d0 con 0x7f1c8c003b70 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c00ba70 con 0x7f1c8c004d40 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8c00bc70 con 0x7f1ca00ccd90 2024-03-30T13:10:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9effd640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1c8c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c97fff640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 0x7f1c8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9f7fe640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c004d40 0x7f1c8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9effd640 1 -- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 msgr2=0x7f1c8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9effd640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca00ccd90 0x7f1c8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9effd640 1 -- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c004d40 msgr2=0x7f1c8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9effd640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c004d40 0x7f1c8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9effd640 1 -- 172.21.15.66:0/356233392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c003520 con 0x7f1c8c003b70 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.559+0000 7f1c9f7fe640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c004d40 0x7f1c8c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1c9effd640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1c8c003f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009ac0 tx=0x7f1c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdec9590f6f04583 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90039070 con 0x7f1c8c003b70 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9002c6b0 con 0x7f1c8c003b70 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1c9e7fc640 1 -- 172.21.15.66:0/356233392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90009690 con 0x7f1c8c003b70 2024-03-30T13:10:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1ca59fd640 1 -- 172.21.15.66:0/356233392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 msgr2=0x7f1c8c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/356233392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1c8c003f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009ac0 tx=0x7f1c8c005830 comp rx=0 tx=0).stop 2024-03-30T13:10:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1c9f7fe640 1 -- 172.21.15.66:0/356233392 reap_dead start 2024-03-30T13:10:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1ca59fd640 1 -- 172.21.15.66:0/356233392 shutdown_connections 2024-03-30T13:10:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1ca59fd640 1 -- 172.21.15.66:0/356233392 >> 172.21.15.66:0/356233392 conn(0x7f1ca00d0380 msgr2=0x7f1ca00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1ca59fd640 1 -- 172.21.15.66:0/356233392 shutdown_connections 2024-03-30T13:10:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.560+0000 7f1ca59fd640 1 -- 172.21.15.66:0/356233392 wait complete. 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 Processor -- start 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 -- start start 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1ca0160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca01f52f0 0x7f1ca0099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca006ecd0 0x7f1ca006f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca00cca00 con 0x7f1c8c003b70 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca00cc880 con 0x7f1ca006ecd0 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1ca59fd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca00cd170 con 0x7f1ca01f52f0 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1c9effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1ca0160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1c9effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1ca0160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43444/0 (socket says 172.21.15.66:43444) 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.562+0000 7f1c9effd640 1 -- 172.21.15.66:0/3017553057 learned_addr learned my addr 172.21.15.66:0/3017553057 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:49.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9f7fe640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca006ecd0 0x7f1ca006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c97fff640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca01f52f0 0x7f1ca0099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9effd640 1 -- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca01f52f0 msgr2=0x7f1ca0099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9effd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca01f52f0 0x7f1ca0099850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9effd640 1 -- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca006ecd0 msgr2=0x7f1ca006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9effd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca006ecd0 0x7f1ca006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9effd640 1 -- 172.21.15.66:0/3017553057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca01f5770 con 0x7f1c8c003b70 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c97fff640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca01f52f0 0x7f1ca0099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9effd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1ca0160530 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c90000c00 tx=0x7f1c90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90042020 con 0x7f1c8c003b70 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9003a040 con 0x7f1c8c003b70 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca0160c10 con 0x7f1c8c003b70 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90039070 con 0x7f1c8c003b70 2024-03-30T13:10:49.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.563+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca0159430 con 0x7f1c8c003b70 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.564+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca0075900 con 0x7f1c8c003b70 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.565+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1c90045020 con 0x7f1c8c003b70 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.565+0000 7f1c9cff9640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1c8006c530 0x7f1c8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.565+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7f1c900acbb0 con 0x7f1c8c003b70 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.565+0000 7f1c97fff640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1c8006c530 0x7f1c8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.566+0000 7f1c97fff640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1c8006c530 0x7f1c8006e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ca006f870 tx=0x7f1c88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.569+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1c9004c070 con 0x7f1c8c003b70 2024-03-30T13:10:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:49.754+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ca006e270 con 0x7f1c8c003b70 2024-03-30T13:10:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.422+0000 7f1c9cff9640 1 -- 172.21.15.66:0/3017553057 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f1c9007b140 con 0x7f1c8c003b70 2024-03-30T13:10:50.426 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' removed 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1c8006c530 msgr2=0x7f1c8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1c8006c530 0x7f1c8006e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ca006f870 tx=0x7f1c88009040 comp rx=0 tx=0).stop 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 msgr2=0x7f1ca0160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1ca0160530 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c90000c00 tx=0x7f1c90004730 comp rx=0 tx=0).stop 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 shutdown_connections 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1c8006c530 0x7f1c8006e9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ca006ecd0 0x7f1ca006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ca01f52f0 0x7f1ca0099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 --2- 172.21.15.66:0/3017553057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c8c003b70 0x7f1ca0160530 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 >> 172.21.15.66:0/3017553057 conn(0x7f1ca00d0380 msgr2=0x7f1ca006db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 shutdown_connections 2024-03-30T13:10:50.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.427+0000 7f1ca59fd640 1 -- 172.21.15.66:0/3017553057 wait complete. 2024-03-30T13:10:50.448 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T13:10:50.448 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T13:10:50.448 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T13:10:50.448 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:10:50.449 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T13:10:50.449 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T13:10:50.449 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T13:10:50.449 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:10:50.449 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T13:10:50.449 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T13:10:50.450 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T13:10:50.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:10:50.450 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T13:10:50.450 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T13:10:50.450 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T13:10:50.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:10:50.459 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T13:10:50.459 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T13:10:50.460 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T13:10:50.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T13:10:50.461 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T13:10:50.461 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T13:10:50.462 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T13:10:50.462 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T13:10:50.502 INFO:tasks.ceph.mds.a:Started 2024-03-30T13:10:50.502 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:50.503 INFO:tasks.ceph.mds.c:Started 2024-03-30T13:10:50.503 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:50.504 INFO:tasks.ceph.mds.b:Started 2024-03-30T13:10:50.505 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:50.505 INFO:tasks.ceph.mds.d:Started 2024-03-30T13:10:50.506 DEBUG:teuthology.parallel:result is None 2024-03-30T13:10:50.506 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T13:10:50.587 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-03-30T13:10:50.587 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-03-30T13:10:50.589 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-03-30T13:10:50.589 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T13:10:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.616+0000 7f3bd0c0d640 1 Processor -- start 2024-03-30T13:10:50.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.617+0000 7f3bd0c0d640 1 -- start start 2024-03-30T13:10:50.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.617+0000 7f3bd0c0d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15fe10 0x7f3bcc0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.617+0000 7f3bd0c0d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bcc0b4130 0x7f3bcc15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bd0c0d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bcc0cced0 con 0x7f3bcc15c260 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bcc0b4130 0x7f3bcc15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3bcc0b4130 0x7f3bcc15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43480/0 (socket says 172.21.15.66:43480) 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 -- 172.21.15.66:0/626518349 learned_addr learned my addr 172.21.15.66:0/626518349 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3bcc160270 con 0x7f3bcc0b4130 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc0b4580 con 0x7f3bcc15fe10 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bc9d9b640 1 --2- 172.21.15.66:0/626518349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15fe10 0x7f3bcc0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 -- 172.21.15.66:0/626518349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15fe10 msgr2=0x7f3bcc0b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 --2- 172.21.15.66:0/626518349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15fe10 0x7f3bcc0b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 -- 172.21.15.66:0/626518349 >> v1:172.21.15.117:6789/0 conn(0x7f3bcc15c260 legacy=0x7f3bcc0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:50.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.618+0000 7f3bcad9d640 1 -- 172.21.15.66:0/626518349 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc071110 con 0x7f3bcc0b4130 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.619+0000 7f3bcad9d640 1 --2- 172.21.15.66:0/626518349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc15c9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009e50 tx=0x7f3bc402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7d7703102eb495 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.619+0000 7f3bc959a640 1 -- 172.21.15.66:0/626518349 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc402ca70 con 0x7f3bcc0b4130 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.619+0000 7f3bc959a640 1 -- 172.21.15.66:0/626518349 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc402cc10 con 0x7f3bcc0b4130 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.619+0000 7f3bc959a640 1 -- 172.21.15.66:0/626518349 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40343f0 con 0x7f3bcc0b4130 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.619+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc0b4130 msgr2=0x7f3bcc15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.619+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/626518349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc15c9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009e50 tx=0x7f3bc402bed0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.620+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 shutdown_connections 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.620+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/626518349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc15c9f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.620+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/626518349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15fe10 0x7f3bcc0b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.620+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 >> 172.21.15.66:0/626518349 conn(0x7f3bcc0cebf0 msgr2=0x7f3bcc0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:50.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.620+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 shutdown_connections 2024-03-30T13:10:50.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.620+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/626518349 wait complete. 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.621+0000 7f3bd0c0d640 1 Processor -- start 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.621+0000 7f3bd0c0d640 1 -- start start 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bd0c0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc0a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bd0c0d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15c260 0x7f3bcc0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bd0c0d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 0x7f3bcc1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bd0c0d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc0719b0 con 0x7f3bcc15fe10 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bd0c0d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc0716d0 con 0x7f3bcc0b4130 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bd0c0d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc071c90 con 0x7f3bcc15c260 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 0x7f3bcc1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:50.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 0x7f3bcc1f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43492/0 (socket says 172.21.15.66:43492) 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 -- 172.21.15.66:0/3009037020 learned_addr learned my addr 172.21.15.66:0/3009037020 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bca59c640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc0a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bc9d9b640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15c260 0x7f3bcc0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 -- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15c260 msgr2=0x7f3bcc0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15c260 0x7f3bcc0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 -- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0b4130 msgr2=0x7f3bcc0a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc0a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 -- 172.21.15.66:0/3009037020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc0c9f30 con 0x7f3bcc15fe10 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.622+0000 7f3bcad9d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 0x7f3bcc1f5ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4036040 tx=0x7f3bc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.623+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4043020 con 0x7f3bcc15fe10 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.623+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc0ca210 con 0x7f3bcc15fe10 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.623+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc403e450 con 0x7f3bcc15fe10 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.623+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc403e840 con 0x7f3bcc15fe10 2024-03-30T13:10:50.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.623+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc0ca7d0 con 0x7f3bcc15fe10 2024-03-30T13:10:50.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.624+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc075900 con 0x7f3bcc15fe10 2024-03-30T13:10:50.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.624+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3bc4046020 con 0x7f3bcc15fe10 2024-03-30T13:10:50.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.625+0000 7f3bbaffd640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bac06c530 0x7f3bac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:50.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.625+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f3bc40add90 con 0x7f3bcc15fe10 2024-03-30T13:10:50.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.625+0000 7f3bca59c640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bac06c530 0x7f3bac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:50.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.625+0000 7f3bca59c640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bac06c530 0x7f3bac06e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc004ea0 tx=0x7f3bbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:50.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.629+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3bc404d030 con 0x7f3bcc15fe10 2024-03-30T13:10:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:50.826+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3bcc15f4d0 con 0x7f3bcc15fe10 2024-03-30T13:10:51.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.456+0000 7f3bbaffd640 1 -- 172.21.15.66:0/3009037020 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f3bc407c560 con 0x7f3bcc15fe10 2024-03-30T13:10:51.460 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bac06c530 msgr2=0x7f3bac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bac06c530 0x7f3bac06e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc004ea0 tx=0x7f3bbc009040 comp rx=0 tx=0).stop 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 msgr2=0x7f3bcc1f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 0x7f3bcc1f5ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4036040 tx=0x7f3bc4004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 shutdown_connections 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3bac06c530 0x7f3bac06e9f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc15fe10 0x7f3bcc1f5ef0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bcc15c260 0x7f3bcc0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 --2- 172.21.15.66:0/3009037020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bcc0b4130 0x7f3bcc0a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 >> 172.21.15.66:0/3009037020 conn(0x7f3bcc0cebf0 msgr2=0x7f3bcc15d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:51.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 shutdown_connections 2024-03-30T13:10:51.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.459+0000 7f3bd0c0d640 1 -- 172.21.15.66:0/3009037020 wait complete. 2024-03-30T13:10:51.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T13:10:51.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.587+0000 7f8cbb52f640 1 Processor -- start 2024-03-30T13:10:51.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.587+0000 7f8cbb52f640 1 -- start start 2024-03-30T13:10:51.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cbb52f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb4155990 0x7f8cb406c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cbb52f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cb40b4050 0x7f8cb40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cbb52f640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb41624f0 con 0x7f8cb40d0300 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cbb52f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f8cb41622a0 con 0x7f8cb40b4050 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cbb52f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb41627d0 con 0x7f8cb4155990 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cb9acc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8cb40d0300 0x7f8cb415eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:42856/0 (socket says 172.21.15.66:42856) 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cb9acc640 1 -- 172.21.15.66:0/1184836463 learned_addr learned my addr 172.21.15.66:0/1184836463 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cb92cb640 1 --2- 172.21.15.66:0/1184836463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb4155990 0x7f8cb406c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cb8aca640 1 --2- 172.21.15.66:0/1184836463 >> v2:172.21.15.117:3300/0 conn(0x7f8cb40b4050 0x7f8cb40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cabfff640 1 -- 172.21.15.66:0/1184836463 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097608388 0 0) 0x7f8cb41624f0 con 0x7f8cb40d0300 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cabfff640 1 -- 172.21.15.66:0/1184836463 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ca0003520 con 0x7f8cb40d0300 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cabfff640 1 -- 172.21.15.66:0/1184836463 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2511231836 0 0) 0x7f8ca0003520 con 0x7f8cb40d0300 2024-03-30T13:10:51.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cb92cb640 1 -- 172.21.15.66:0/1184836463 >> v2:172.21.15.117:3300/0 conn(0x7f8cb40b4050 msgr2=0x7f8cb40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:51.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.588+0000 7f8cb92cb640 1 --2- 172.21.15.66:0/1184836463 >> v2:172.21.15.117:3300/0 conn(0x7f8cb40b4050 0x7f8cb40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.589+0000 7f8cb92cb640 1 -- 172.21.15.66:0/1184836463 >> v1:172.21.15.66:6789/0 conn(0x7f8cb40d0300 legacy=0x7f8cb415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.589+0000 7f8cb92cb640 1 -- 172.21.15.66:0/1184836463 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb4162ab0 con 0x7f8cb4155990 2024-03-30T13:10:51.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.589+0000 7f8cb8aca640 1 --2- 172.21.15.66:0/1184836463 >> v2:172.21.15.117:3300/0 conn(0x7f8cb40b4050 0x7f8cb40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.589+0000 7f8cb92cb640 1 --2- 172.21.15.66:0/1184836463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb4155990 0x7f8cb406c690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097a0 tx=0x7f8cb002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f4e987344be1e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cabfff640 1 -- 172.21.15.66:0/1184836463 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0037070 con 0x7f8cb4155990 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cabfff640 1 -- 172.21.15.66:0/1184836463 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb0030510 con 0x7f8cb4155990 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cabfff640 1 -- 172.21.15.66:0/1184836463 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0019f0 con 0x7f8cb4155990 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1184836463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb4155990 msgr2=0x7f8cb406c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1184836463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb4155990 0x7f8cb406c690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097a0 tx=0x7f8cb002b710 comp rx=0 tx=0).stop 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1184836463 shutdown_connections 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1184836463 >> v2:172.21.15.117:3300/0 conn(0x7f8cb40b4050 0x7f8cb40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1184836463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb4155990 0x7f8cb406c690 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1184836463 >> 172.21.15.66:0/1184836463 conn(0x7f8cb40672c0 msgr2=0x7f8cb40676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:51.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1184836463 shutdown_connections 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.590+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1184836463 wait complete. 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.591+0000 7f8cbb52f640 1 Processor -- start 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.591+0000 7f8cbb52f640 1 -- start start 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cbb52f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb40b4050 0x7f8cb4154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cbb52f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb40d0300 0x7f8cb41f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cbb52f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 0x7f8cb415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cbb52f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb40cfb20 con 0x7f8cb4155990 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cbb52f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb40cf9a0 con 0x7f8cb40d0300 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cbb52f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb40b44d0 con 0x7f8cb40b4050 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 0x7f8cb415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 0x7f8cb415e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43504/0 (socket says 172.21.15.66:43504) 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 -- 172.21.15.66:0/1707387066 learned_addr learned my addr 172.21.15.66:0/1707387066 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb92cb640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb40b4050 0x7f8cb4154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb8aca640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb40d0300 0x7f8cb41f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 -- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb40b4050 msgr2=0x7f8cb4154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb40b4050 0x7f8cb4154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 -- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb40d0300 msgr2=0x7f8cb41f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb40d0300 0x7f8cb41f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb9acc640 1 -- 172.21.15.66:0/1707387066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb41f90e0 con 0x7f8cb4155990 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.592+0000 7f8cb8aca640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb40d0300 0x7f8cb41f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.593+0000 7f8cb9acc640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 0x7f8cb415e070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00ccc0 tx=0x7f8cac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.593+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac012400 con 0x7f8cb4155990 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.593+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cac0125c0 con 0x7f8cb4155990 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.593+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb4154a60 con 0x7f8cb4155990 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.593+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0129b0 con 0x7f8cb4155990 2024-03-30T13:10:51.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.593+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb415f050 con 0x7f8cb4155990 2024-03-30T13:10:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.594+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb4068fd0 con 0x7f8cb4155990 2024-03-30T13:10:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.594+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8cac012b50 con 0x7f8cb4155990 2024-03-30T13:10:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.595+0000 7f8ca9ffb640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8c9806c530 0x7f8c9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.595+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f8cac08bc70 con 0x7f8cb4155990 2024-03-30T13:10:51.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.595+0000 7f8cb92cb640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8c9806c530 0x7f8c9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.595+0000 7f8cb92cb640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8c9806c530 0x7f8c9806e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0034000 tx=0x7f8cb0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:51.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.599+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8cb4068fd0 con 0x7f8cb4155990 2024-03-30T13:10:51.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.853+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8cb40b1b20 con 0x7f8cb4155990 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.853+0000 7f8ca9ffb640 1 -- 172.21.15.66:0/1707387066 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f8cb40b1b20 con 0x7f8cb4155990 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghmDX8pGxAAXUG3Yk8OMBMyc01hkoZZUA==", 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.857 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghmpUGNKBAA+EXwkCzjb8uf4IZ1U5QtYg==", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.858 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghm84NPIxAAca1wZeauGlQMHS54SzhqIw==", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghm210LMBAALVzbm4DxE7Nkdt8Xy3Cj4A==", 2024-03-30T13:10:51.859 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDgDghmz9qhFxAA358Jcrx/J+fQZ7i9GGHxsw==", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.860 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDjDghmpXB1FhAASU2j0mAAH7HASAgaR0ZhFg==", 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.861 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDmDghmZnV/FRAA+G0QiFaHxEweK3xSIr1MYQ==", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDpDghma3JUFBAAD9kkiYh+/ew0XBFnsOPAJQ==", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.862 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDvDghmyvoYHxAARaVqrylp8Xl1Sw72nVydWw==", 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-03-30T13:10:51.863 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDyDghmgkkKHRAAIhxMaiQYTe2MlrvNZH/5uA==", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD1DghmRRx/GxAAWlX4LBu1lvjj0y8I5jmFjw==", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.864 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD4DghmM0RbGhAAX4q8eHJEih+NUBxEoPlXkw==", 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghmupudNBAAFaQaghMGgLisibrdNbX0TQ==", 2024-03-30T13:10:51.865 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDdDghmYAp9ABAANUw0ahZSYbh8lGdQvtXebA==", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T13:10:51.866 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghmbu2BOBAAvr83E6ExRX1dKKXuanZj6w==", 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDdDghmI35QBhAA26SuLDVaBa9wDDqx3JW1Og==", 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDbDghm/4fRNBAAA6kb1Bq9vj8lQo2wKYvo7A==", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mds", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD+DghmMYFHFxAAKOEG8amMM5kBeKzdKeaW9A==", 2024-03-30T13:10:51.869 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mds" 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mgr", 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD+DghmRIpHFxAAYO7OucDyrvuc5ErbiShT6w==", 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-osd", 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD+DghmhJFHFxAAmIakGlT6vUuwHMQAyuUaow==", 2024-03-30T13:10:51.870 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-osd" 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd", 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD+DghmtZhHFxAAKsTmH34XEM1vGVw0QCXYyg==", 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD+DghmHqBHFxAAJKkEP91Mbp6Ey9V4vzE5DQ==", 2024-03-30T13:10:51.871 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rgw", 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQD+Dghm26lHFxAAu86xcQWXdCFF0tBMLZ895A==", 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-30T13:10:51.872 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghm8/M0FxAA8he7CSb1rjXQZy2E4x/3IQ==", 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:10:51.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQDcDghm2nh1ExAA/llAPiVUrYlS3mxLR+rPZQ==", 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-30T13:10:51.874 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:10:51.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8c9806c530 msgr2=0x7f8c9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8c9806c530 0x7f8c9806e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0034000 tx=0x7f8cb0008030 comp rx=0 tx=0).stop 2024-03-30T13:10:51.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 msgr2=0x7f8cb415e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 0x7f8cb415e070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00ccc0 tx=0x7f8cac00a430 comp rx=0 tx=0).stop 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 shutdown_connections 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8c9806c530 0x7f8c9806e9f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb4155990 0x7f8cb415e070 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb40d0300 0x7f8cb41f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 --2- 172.21.15.66:0/1707387066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cb40b4050 0x7f8cb4154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 >> 172.21.15.66:0/1707387066 conn(0x7f8cb40672c0 msgr2=0x7f8cb406d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 shutdown_connections 2024-03-30T13:10:51.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.855+0000 7f8cbb52f640 1 -- 172.21.15.66:0/1707387066 wait complete. 2024-03-30T13:10:51.878 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-30T13:10:51.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.982+0000 7ff667ff4640 1 Processor -- start 2024-03-30T13:10:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.982+0000 7ff667ff4640 1 -- start start 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff667ff4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff6600cbed0 0x7ff66015f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff667ff4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff66015f6c0 0x7ff660151c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff667ff4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66015feb0 con 0x7ff6600cb050 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff667ff4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff66015c480 con 0x7ff6600cbed0 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff667ff4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff66015c910 con 0x7ff66015f6c0 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff6600cbed0 0x7ff66015f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff6600cbed0 0x7ff66015f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50104/0 (socket says 172.21.15.66:50104) 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 -- 172.21.15.66:0/1147910122 learned_addr learned my addr 172.21.15.66:0/1147910122 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff666591640 1 --2- 172.21.15.66:0/1147910122 >> v2:172.21.15.117:3300/0 conn(0x7ff66015f6c0 0x7ff660151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 -- 172.21.15.66:0/1147910122 >> v2:172.21.15.117:3300/0 conn(0x7ff66015f6c0 msgr2=0x7ff660151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 --2- 172.21.15.66:0/1147910122 >> v2:172.21.15.117:3300/0 conn(0x7ff66015f6c0 0x7ff660151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 -- 172.21.15.66:0/1147910122 >> v1:172.21.15.117:6789/0 conn(0x7ff6600cb050 legacy=0x7ff6600cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.983+0000 7ff66558f640 1 -- 172.21.15.66:0/1147910122 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66015bb50 con 0x7ff6600cbed0 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.984+0000 7ff66558f640 1 --2- 172.21.15.66:0/1147910122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cbed0 0x7ff66015f180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6600b1bb0 tx=0x7ff65002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0677b76b21aba03 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.984+0000 7ff664d8e640 1 -- 172.21.15.66:0/1147910122 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650035020 con 0x7ff6600cbed0 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.984+0000 7ff664d8e640 1 -- 172.21.15.66:0/1147910122 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff650031870 con 0x7ff6600cbed0 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.984+0000 7ff664d8e640 1 -- 172.21.15.66:0/1147910122 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650031c40 con 0x7ff6600cbed0 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 -- 172.21.15.66:0/1147910122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cbed0 msgr2=0x7ff66015f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 --2- 172.21.15.66:0/1147910122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cbed0 0x7ff66015f180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6600b1bb0 tx=0x7ff65002b910 comp rx=0 tx=0).stop 2024-03-30T13:10:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 -- 172.21.15.66:0/1147910122 shutdown_connections 2024-03-30T13:10:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 --2- 172.21.15.66:0/1147910122 >> v2:172.21.15.117:3300/0 conn(0x7ff66015f6c0 0x7ff660151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 --2- 172.21.15.66:0/1147910122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cbed0 0x7ff66015f180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 -- 172.21.15.66:0/1147910122 >> 172.21.15.66:0/1147910122 conn(0x7ff66006d610 msgr2=0x7ff66006da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 -- 172.21.15.66:0/1147910122 shutdown_connections 2024-03-30T13:10:51.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.985+0000 7ff667ff4640 1 -- 172.21.15.66:0/1147910122 wait complete. 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.986+0000 7ff667ff4640 1 Processor -- start 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.986+0000 7ff667ff4640 1 -- start start 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 0x7ff6600ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6600cbed0 0x7ff6600af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66015f6c0 0x7ff6600cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff66015fb50 con 0x7ff6600cb050 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff66015c110 con 0x7ff66015f6c0 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600cb430 con 0x7ff6600cbed0 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 0x7ff6600ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 0x7ff6600ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50112/0 (socket says 172.21.15.66:50112) 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 -- 172.21.15.66:0/538309883 learned_addr learned my addr 172.21.15.66:0/538309883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff66558f640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6600cbed0 0x7ff6600af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff666591640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66015f6c0 0x7ff6600cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 -- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6600cbed0 msgr2=0x7ff6600af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6600cbed0 0x7ff6600af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 -- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66015f6c0 msgr2=0x7ff6600cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66015f6c0 0x7ff6600cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 -- 172.21.15.66:0/538309883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600addf0 con 0x7ff6600cb050 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff665d90640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 0x7ff6600ad8b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00b3f0 tx=0x7ff65c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:51.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c015020 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65c011830 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6600ae0c0 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.987+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c011c20 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.988+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6600ae680 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.988+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff660075900 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.989+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff65c018020 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.989+0000 7ff6567fc640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff64406c530 0x7ff64406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.989+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff65c08af70 con 0x7ff6600cb050 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.989+0000 7ff66558f640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff64406c530 0x7ff64406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:51.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.990+0000 7ff66558f640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff64406c530 0x7ff64406e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff650036000 tx=0x7ff65003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:51.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:51.994+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff65c01f030 con 0x7ff6600cb050 2024-03-30T13:10:52.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.248+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff66006de30 con 0x7ff6600cb050 2024-03-30T13:10:52.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.250+0000 7ff6567fc640 1 -- 172.21.15.66:0/538309883 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7ff65c059190 con 0x7ff6600cb050 2024-03-30T13:10:52.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff64406c530 msgr2=0x7ff64406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff64406c530 0x7ff64406e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff650036000 tx=0x7ff65003c040 comp rx=0 tx=0).stop 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 msgr2=0x7ff6600ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 0x7ff6600ad8b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00b3f0 tx=0x7ff65c00b8b0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 shutdown_connections 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff64406c530 0x7ff64406e9f0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff650036000 tx=0x7ff65003c040 comp rx=0 tx=0).stop 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66015f6c0 0x7ff6600cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6600cbed0 0x7ff6600af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 --2- 172.21.15.66:0/538309883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6600cb050 0x7ff6600ad8b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 >> 172.21.15.66:0/538309883 conn(0x7ff66006d610 msgr2=0x7ff6600cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 shutdown_connections 2024-03-30T13:10:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.252+0000 7ff667ff4640 1 -- 172.21.15.66:0/538309883 wait complete. 2024-03-30T13:10:52.266 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-30T13:10:52.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.371+0000 7ff8bdc75640 1 Processor -- start 2024-03-30T13:10:52.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.371+0000 7ff8bdc75640 1 -- start start 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8bdc75640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff8b80d0560 0x7ff8b8162310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8bdc75640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8b80d1f20 0x7ff8b80b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8bdc75640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b81628f0 con 0x7ff8b815e770 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8bdc75640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b80b5200 con 0x7ff8b80d1f20 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8bdc75640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b8162bd0 con 0x7ff8b80d0560 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8b80d1f20 0x7ff8b80b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8b80d1f20 0x7ff8b80b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50126/0 (socket says 172.21.15.66:50126) 2024-03-30T13:10:52.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 -- 172.21.15.66:0/39189295 learned_addr learned my addr 172.21.15.66:0/39189295 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:52.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b6ffd640 1 --2- 172.21.15.66:0/39189295 >> v2:172.21.15.117:3300/0 conn(0x7ff8b80d0560 0x7ff8b8162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 -- 172.21.15.66:0/39189295 >> v2:172.21.15.117:3300/0 conn(0x7ff8b80d0560 msgr2=0x7ff8b8162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 --2- 172.21.15.66:0/39189295 >> v2:172.21.15.117:3300/0 conn(0x7ff8b80d0560 0x7ff8b8162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 -- 172.21.15.66:0/39189295 >> v1:172.21.15.117:6789/0 conn(0x7ff8b815e770 legacy=0x7ff8b80cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.372+0000 7ff8b7fff640 1 -- 172.21.15.66:0/39189295 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b8162eb0 con 0x7ff8b80d1f20 2024-03-30T13:10:52.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.373+0000 7ff8b7fff640 1 --2- 172.21.15.66:0/39189295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b80d1f20 0x7ff8b80b5690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80711e0 tx=0x7ff8b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c7ae3ba7e381cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.373+0000 7ff8b67fc640 1 -- 172.21.15.66:0/39189295 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b002ca80 con 0x7ff8b80d1f20 2024-03-30T13:10:52.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.373+0000 7ff8b67fc640 1 -- 172.21.15.66:0/39189295 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b002cc20 con 0x7ff8b80d1f20 2024-03-30T13:10:52.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.373+0000 7ff8b67fc640 1 -- 172.21.15.66:0/39189295 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00343f0 con 0x7ff8b80d1f20 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 -- 172.21.15.66:0/39189295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b80d1f20 msgr2=0x7ff8b80b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/39189295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b80d1f20 0x7ff8b80b5690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80711e0 tx=0x7ff8b002bee0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 -- 172.21.15.66:0/39189295 shutdown_connections 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/39189295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b80d1f20 0x7ff8b80b5690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/39189295 >> v2:172.21.15.117:3300/0 conn(0x7ff8b80d0560 0x7ff8b8162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 -- 172.21.15.66:0/39189295 >> 172.21.15.66:0/39189295 conn(0x7ff8b806f8f0 msgr2=0x7ff8b806fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 -- 172.21.15.66:0/39189295 shutdown_connections 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.374+0000 7ff8bdc75640 1 -- 172.21.15.66:0/39189295 wait complete. 2024-03-30T13:10:52.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.375+0000 7ff8bdc75640 1 Processor -- start 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.375+0000 7ff8bdc75640 1 -- start start 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8bdc75640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80d0560 0x7ff8b815fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8bdc75640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b80d1f20 0x7ff8b8146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8bdc75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 0x7ff8b8147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8bdc75640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b806c790 con 0x7ff8b815e770 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8bdc75640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b815eb50 con 0x7ff8b80d1f20 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8bdc75640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b806c910 con 0x7ff8b80d0560 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 0x7ff8b8147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 0x7ff8b8147160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50130/0 (socket says 172.21.15.66:50130) 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 -- 172.21.15.66:0/256551030 learned_addr learned my addr 172.21.15.66:0/256551030 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b77fe640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80d0560 0x7ff8b815fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b6ffd640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b80d1f20 0x7ff8b8146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 -- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80d0560 msgr2=0x7ff8b815fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80d0560 0x7ff8b815fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 -- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b80d1f20 msgr2=0x7ff8b8146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b80d1f20 0x7ff8b8146c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b7fff640 1 -- 172.21.15.66:0/256551030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b80c80d0 con 0x7ff8b815e770 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.376+0000 7ff8b6ffd640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b80d1f20 0x7ff8b8146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.377+0000 7ff8b7fff640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 0x7ff8b8147160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff8b002c4f0 tx=0x7ff8b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.377+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0043020 con 0x7ff8b815e770 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.377+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b003e450 con 0x7ff8b815e770 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.377+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0009510 con 0x7ff8b815e770 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.377+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b80c8370 con 0x7ff8b815e770 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.377+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b80c8930 con 0x7ff8b815e770 2024-03-30T13:10:52.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.378+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b8075900 con 0x7ff8b815e770 2024-03-30T13:10:52.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.378+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff8b0046020 con 0x7ff8b815e770 2024-03-30T13:10:52.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.378+0000 7ff89ffff640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff8a406c530 0x7ff8a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.379+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff8b00adcd0 con 0x7ff8b815e770 2024-03-30T13:10:52.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.379+0000 7ff8b77fe640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff8a406c530 0x7ff8a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.379+0000 7ff8b77fe640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff8a406c530 0x7ff8a406e9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800a750 tx=0x7ff8a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.383+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff8b004d030 con 0x7ff8b815e770 2024-03-30T13:10:52.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.640+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff8b80b8200 con 0x7ff8b815e770 2024-03-30T13:10:52.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.642+0000 7ff89ffff640 1 -- 172.21.15.66:0/256551030 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7ff8b007c540 con 0x7ff8b815e770 2024-03-30T13:10:52.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff8a406c530 msgr2=0x7ff8a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff8a406c530 0x7ff8a406e9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800a750 tx=0x7ff8a8008040 comp rx=0 tx=0).stop 2024-03-30T13:10:52.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 msgr2=0x7ff8b8147160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 0x7ff8b8147160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff8b002c4f0 tx=0x7ff8b0004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:52.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 shutdown_connections 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff8a406c530 0x7ff8a406e9f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b815e770 0x7ff8b8147160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b80d1f20 0x7ff8b8146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 --2- 172.21.15.66:0/256551030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8b80d0560 0x7ff8b815fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 >> 172.21.15.66:0/256551030 conn(0x7ff8b806f8f0 msgr2=0x7ff8b80b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 shutdown_connections 2024-03-30T13:10:52.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.644+0000 7ff8bdc75640 1 -- 172.21.15.66:0/256551030 wait complete. 2024-03-30T13:10:52.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-30T13:10:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.763+0000 7f91a024e640 1 Processor -- start 2024-03-30T13:10:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.763+0000 7f91a024e640 1 -- start start 2024-03-30T13:10:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f91a024e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f919815e540 0x7f91980734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f91a024e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 0x7f919806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f91a024e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91980b4470 con 0x7f919806b580 2024-03-30T13:10:52.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f91a024e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f91980b4a30 con 0x7f919815e540 2024-03-30T13:10:52.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f91a024e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f91980b4220 con 0x7f9198072e00 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f919d7e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 0x7f919806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f919d7e9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 0x7f919806ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:57276/0 (socket says 172.21.15.66:57276) 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.764+0000 7f919d7e9640 1 -- 172.21.15.66:0/934708401 learned_addr learned my addr 172.21.15.66:0/934708401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.765+0000 7f919dfea640 1 --2- 172.21.15.66:0/934708401 >> v2:172.21.15.117:3300/0 conn(0x7f919815e540 0x7f91980734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.765+0000 7f919d7e9640 1 -- 172.21.15.66:0/934708401 >> v2:172.21.15.117:3300/0 conn(0x7f919815e540 msgr2=0x7f91980734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.765+0000 7f919d7e9640 1 --2- 172.21.15.66:0/934708401 >> v2:172.21.15.117:3300/0 conn(0x7f919815e540 0x7f91980734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.765+0000 7f919d7e9640 1 -- 172.21.15.66:0/934708401 >> v1:172.21.15.117:6789/0 conn(0x7f919806b580 legacy=0x7f91980ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.765+0000 7f919d7e9640 1 -- 172.21.15.66:0/934708401 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91980b4750 con 0x7f9198072e00 2024-03-30T13:10:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f919d7e9640 1 --2- 172.21.15.66:0/934708401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 0x7f919806ef90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91980b43a0 tx=0x7f918c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c1fdafb0e78be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f919cfe8640 1 -- 172.21.15.66:0/934708401 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c034070 con 0x7f9198072e00 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f919cfe8640 1 -- 172.21.15.66:0/934708401 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c030840 con 0x7f9198072e00 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f919cfe8640 1 -- 172.21.15.66:0/934708401 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c030c10 con 0x7f9198072e00 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f91a024e640 1 -- 172.21.15.66:0/934708401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 msgr2=0x7f919806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f91a024e640 1 --2- 172.21.15.66:0/934708401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 0x7f919806ef90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91980b43a0 tx=0x7f918c02b980 comp rx=0 tx=0).stop 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f91a024e640 1 -- 172.21.15.66:0/934708401 shutdown_connections 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f91a024e640 1 --2- 172.21.15.66:0/934708401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9198072e00 0x7f919806ef90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f91a024e640 1 --2- 172.21.15.66:0/934708401 >> v2:172.21.15.117:3300/0 conn(0x7f919815e540 0x7f91980734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.766+0000 7f91a024e640 1 -- 172.21.15.66:0/934708401 >> 172.21.15.66:0/934708401 conn(0x7f91980d1170 msgr2=0x7f91980d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.767+0000 7f91a024e640 1 -- 172.21.15.66:0/934708401 shutdown_connections 2024-03-30T13:10:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.767+0000 7f91a024e640 1 -- 172.21.15.66:0/934708401 wait complete. 2024-03-30T13:10:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 Processor -- start 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 -- start start 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f919806b580 0x7f9198155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9198072e00 0x7f91980b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 0x7f91980b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f919806f4d0 con 0x7f919815e540 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f919815e920 con 0x7f9198072e00 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.768+0000 7f91a024e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9198160a40 con 0x7f919806b580 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 0x7f91980b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 0x7f91980b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50140/0 (socket says 172.21.15.66:50140) 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 -- 172.21.15.66:0/3425682046 learned_addr learned my addr 172.21.15.66:0/3425682046 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919dfea640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f919806b580 0x7f9198155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919d7e9640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9198072e00 0x7f91980b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 -- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f919806b580 msgr2=0x7f9198155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f919806b580 0x7f9198155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 -- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9198072e00 msgr2=0x7f91980b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9198072e00 0x7f91980b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 -- 172.21.15.66:0/3425682046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9198160d20 con 0x7f919815e540 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919d7e9640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9198072e00 0x7f91980b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f919e7eb640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 0x7f91980b5370 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f918800c6e0 tx=0x7f918800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188003b70 con 0x7f919815e540 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9188003d30 con 0x7f919815e540 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188005310 con 0x7f919815e540 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9198156630 con 0x7f919815e540 2024-03-30T13:10:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.769+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919815fd80 con 0x7f919815e540 2024-03-30T13:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.770+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198072970 con 0x7f919815e540 2024-03-30T13:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.771+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f91880054b0 con 0x7f919815e540 2024-03-30T13:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.771+0000 7f91967fc640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918406c530 0x7f918406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.771+0000 7f919dfea640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918406c530 0x7f918406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.771+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f918808b230 con 0x7f919815e540 2024-03-30T13:10:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.772+0000 7f919dfea640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918406c530 0x7f918406e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f91980d0ea0 tx=0x7f9190009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:52.776+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9198072970 con 0x7f919815e540 2024-03-30T13:10:53.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.036+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9198075900 con 0x7f919815e540 2024-03-30T13:10:53.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.038+0000 7f91967fc640 1 -- 172.21.15.66:0/3425682046 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f91880593c0 con 0x7f919815e540 2024-03-30T13:10:53.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918406c530 msgr2=0x7f918406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918406c530 0x7f918406e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f91980d0ea0 tx=0x7f9190009190 comp rx=0 tx=0).stop 2024-03-30T13:10:53.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 msgr2=0x7f91980b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 0x7f91980b5370 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f918800c6e0 tx=0x7f918800cba0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 shutdown_connections 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f918406c530 0x7f918406e9f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f919815e540 0x7f91980b5370 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9198072e00 0x7f91980b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 --2- 172.21.15.66:0/3425682046 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f919806b580 0x7f9198155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 >> 172.21.15.66:0/3425682046 conn(0x7f91980d1170 msgr2=0x7f91980d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 shutdown_connections 2024-03-30T13:10:53.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.040+0000 7f91a024e640 1 -- 172.21.15.66:0/3425682046 wait complete. 2024-03-30T13:10:53.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-30T13:10:53.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.166+0000 7f1eda196640 1 Processor -- start 2024-03-30T13:10:53.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.166+0000 7f1eda196640 1 -- start start 2024-03-30T13:10:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.166+0000 7f1eda196640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ed415e450 0x7f1ed40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1eda196640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ed40d38d0 0x7f1ed40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1eda196640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed40b9d20 con 0x7f1ed4162a80 2024-03-30T13:10:53.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1eda196640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed40cea50 con 0x7f1ed415e450 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1eda196640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed40b9fc0 con 0x7f1ed40d38d0 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed37fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ed415e450 0x7f1ed40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed2ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1ed4162a80 0x7f1ed40d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60660/0 (socket says 172.21.15.66:60660) 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed2ffd640 1 -- 172.21.15.66:0/3568961876 learned_addr learned my addr 172.21.15.66:0/3568961876 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed3fff640 1 --2- 172.21.15.66:0/3568961876 >> v2:172.21.15.117:3300/0 conn(0x7f1ed40d38d0 0x7f1ed40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed37fe640 1 -- 172.21.15.66:0/3568961876 >> v2:172.21.15.117:3300/0 conn(0x7f1ed40d38d0 msgr2=0x7f1ed40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed37fe640 1 --2- 172.21.15.66:0/3568961876 >> v2:172.21.15.117:3300/0 conn(0x7f1ed40d38d0 0x7f1ed40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed37fe640 1 -- 172.21.15.66:0/3568961876 >> v1:172.21.15.66:6789/0 conn(0x7f1ed4162a80 legacy=0x7f1ed40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.167+0000 7f1ed37fe640 1 -- 172.21.15.66:0/3568961876 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed4162ee0 con 0x7f1ed415e450 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1ed37fe640 1 --2- 172.21.15.66:0/3568961876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed415e450 0x7f1ed40cf3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40c7e40 tx=0x7f1ec802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d842b652a806ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1ed27fc640 1 -- 172.21.15.66:0/3568961876 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80315a0 con 0x7f1ed415e450 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1ed27fc640 1 -- 172.21.15.66:0/3568961876 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec8037070 con 0x7f1ed415e450 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1ed27fc640 1 -- 172.21.15.66:0/3568961876 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8031960 con 0x7f1ed415e450 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1eda196640 1 -- 172.21.15.66:0/3568961876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed415e450 msgr2=0x7f1ed40cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1eda196640 1 --2- 172.21.15.66:0/3568961876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed415e450 0x7f1ed40cf3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40c7e40 tx=0x7f1ec802b840 comp rx=0 tx=0).stop 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1eda196640 1 -- 172.21.15.66:0/3568961876 shutdown_connections 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1eda196640 1 --2- 172.21.15.66:0/3568961876 >> v2:172.21.15.117:3300/0 conn(0x7f1ed40d38d0 0x7f1ed40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1eda196640 1 --2- 172.21.15.66:0/3568961876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed415e450 0x7f1ed40cf3c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.168+0000 7f1eda196640 1 -- 172.21.15.66:0/3568961876 >> 172.21.15.66:0/3568961876 conn(0x7f1ed40678f0 msgr2=0x7f1ed40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.169+0000 7f1eda196640 1 -- 172.21.15.66:0/3568961876 shutdown_connections 2024-03-30T13:10:53.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.169+0000 7f1eda196640 1 -- 172.21.15.66:0/3568961876 wait complete. 2024-03-30T13:10:53.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.170+0000 7f1eda196640 1 Processor -- start 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.170+0000 7f1eda196640 1 -- start start 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 0x7f1ed40aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ed415e450 0x7f1ed40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ed4162a80 0x7f1ed40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed415e830 con 0x7f1ed40d38d0 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed40cef30 con 0x7f1ed415e450 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed415e9b0 con 0x7f1ed4162a80 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 0x7f1ed40aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 0x7f1ed40aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50154/0 (socket says 172.21.15.66:50154) 2024-03-30T13:10:53.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 -- 172.21.15.66:0/321062473 learned_addr learned my addr 172.21.15.66:0/321062473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 -- 172.21.15.66:0/321062473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ed4162a80 msgr2=0x7f1ed40b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ed4162a80 0x7f1ed40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 -- 172.21.15.66:0/321062473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ed415e450 msgr2=0x7f1ed40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ed415e450 0x7f1ed40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 -- 172.21.15.66:0/321062473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41e9460 con 0x7f1ed40d38d0 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ed37fe640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 0x7f1ed40aad50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8032840 con 0x7f1ed40d38d0 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec80329e0 con 0x7f1ed40d38d0 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed41e9740 con 0x7f1ed40d38d0 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8032dd0 con 0x7f1ed40d38d0 2024-03-30T13:10:53.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.171+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41e9d00 con 0x7f1ed40d38d0 2024-03-30T13:10:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.172+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed4075900 con 0x7f1ed40d38d0 2024-03-30T13:10:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.173+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1ec804c020 con 0x7f1ed40d38d0 2024-03-30T13:10:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.173+0000 7f1ebbfff640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ec006c530 0x7f1ec006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.173+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1ec802f020 con 0x7f1ed40d38d0 2024-03-30T13:10:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.173+0000 7f1ed2ffd640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ec006c530 0x7f1ec006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.174+0000 7f1ed2ffd640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ec006c530 0x7f1ec006e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005880 tx=0x7f1ec4007aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.178+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1ec8040070 con 0x7f1ed40d38d0 2024-03-30T13:10:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.433+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1ed40b4d90 con 0x7f1ed40d38d0 2024-03-30T13:10:53.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.436+0000 7f1ebbfff640 1 -- 172.21.15.66:0/321062473 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f1ec807bbb0 con 0x7f1ed40d38d0 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ec006c530 msgr2=0x7f1ec006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ec006c530 0x7f1ec006e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005880 tx=0x7f1ec4007aa0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 msgr2=0x7f1ed40aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 0x7f1ed40aad50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec80046c0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 shutdown_connections 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f1ec006c530 0x7f1ec006e9f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ed4162a80 0x7f1ed40b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ed415e450 0x7f1ed40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 --2- 172.21.15.66:0/321062473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ed40d38d0 0x7f1ed40aad50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 >> 172.21.15.66:0/321062473 conn(0x7f1ed40678f0 msgr2=0x7f1ed40b8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 shutdown_connections 2024-03-30T13:10:53.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.438+0000 7f1eda196640 1 -- 172.21.15.66:0/321062473 wait complete. 2024-03-30T13:10:53.452 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-30T13:10:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.557+0000 7f2fc763c640 1 Processor -- start 2024-03-30T13:10:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.557+0000 7f2fc763c640 1 -- start start 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc763c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2fc0155990 0x7f2fc006c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc763c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc763c640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc01624f0 con 0x7f2fc00d0300 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc763c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc01622a0 con 0x7f2fc0155990 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc763c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc01627d0 con 0x7f2fc00b4050 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc53d8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2fc0155990 0x7f2fc006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc5bd9640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2fc00d0300 0x7f2fc015eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60666/0 (socket says 172.21.15.66:60666) 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc5bd9640 1 -- 172.21.15.66:0/1149949689 learned_addr learned my addr 172.21.15.66:0/1149949689 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc4bd7640 1 --2- 172.21.15.66:0/1149949689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc53d8640 1 -- 172.21.15.66:0/1149949689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 msgr2=0x7f2fc00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc53d8640 1 --2- 172.21.15.66:0/1149949689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc53d8640 1 -- 172.21.15.66:0/1149949689 >> v1:172.21.15.66:6789/0 conn(0x7f2fc00d0300 legacy=0x7f2fc015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fc53d8640 1 -- 172.21.15.66:0/1149949689 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc0162ab0 con 0x7f2fc0155990 2024-03-30T13:10:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.558+0000 7f2fb7fff640 1 -- 172.21.15.66:0/1149949689 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282617648 0 0) 0x7f2fc01624f0 con 0x7f2fc00d0300 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.559+0000 7f2fc53d8640 1 --2- 172.21.15.66:0/1149949689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc0155990 0x7f2fc006c690 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2963ef501a85632 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.559+0000 7f2fb7fff640 1 -- 172.21.15.66:0/1149949689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0315a0 con 0x7f2fc0155990 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.559+0000 7f2fb7fff640 1 -- 172.21.15.66:0/1149949689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc037070 con 0x7f2fc0155990 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.559+0000 7f2fb7fff640 1 -- 172.21.15.66:0/1149949689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc031960 con 0x7f2fc0155990 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 -- 172.21.15.66:0/1149949689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc0155990 msgr2=0x7f2fc006c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 --2- 172.21.15.66:0/1149949689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc0155990 0x7f2fc006c690 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc02b840 comp rx=0 tx=0).stop 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 -- 172.21.15.66:0/1149949689 shutdown_connections 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 --2- 172.21.15.66:0/1149949689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 --2- 172.21.15.66:0/1149949689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc0155990 0x7f2fc006c690 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 -- 172.21.15.66:0/1149949689 >> 172.21.15.66:0/1149949689 conn(0x7f2fc00672c0 msgr2=0x7f2fc00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 -- 172.21.15.66:0/1149949689 shutdown_connections 2024-03-30T13:10:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.560+0000 7f2fc763c640 1 -- 172.21.15.66:0/1149949689 wait complete. 2024-03-30T13:10:53.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.561+0000 7f2fc763c640 1 Processor -- start 2024-03-30T13:10:53.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.561+0000 7f2fc763c640 1 -- start start 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc763c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc0158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc763c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 0x7f2fc00c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc763c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc0155990 0x7f2fc00c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc763c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00cfb20 con 0x7f2fc00d0300 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc763c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00cf9a0 con 0x7f2fc0155990 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc763c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc00b44d0 con 0x7f2fc00b4050 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 0x7f2fc00c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 0x7f2fc00c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50176/0 (socket says 172.21.15.66:50176) 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 -- 172.21.15.66:0/2047120389 learned_addr learned my addr 172.21.15.66:0/2047120389 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc53d8640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc0158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 -- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 msgr2=0x7f2fc0158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc0158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 -- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc0155990 msgr2=0x7f2fc00c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc5bd9640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc0155990 0x7f2fc00c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc0155990 0x7f2fc00c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 -- 172.21.15.66:0/2047120389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc00b5ad0 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc5bd9640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc0155990 0x7f2fc00c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.562+0000 7f2fc4bd7640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 0x7f2fc00c7820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000d5f0 tx=0x7f2fb000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.563+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00125f0 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.563+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb0019070 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.563+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc0158fe0 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.563+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00129d0 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.563+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc0161250 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.564+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc0075a10 con 0x7f2fc00d0300 2024-03-30T13:10:53.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.564+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2fb0012b70 con 0x7f2fc00d0300 2024-03-30T13:10:53.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.564+0000 7f2fb5ffb640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fa406c530 0x7f2fa406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.565+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f2fb008b890 con 0x7f2fc00d0300 2024-03-30T13:10:53.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.565+0000 7f2fc53d8640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fa406c530 0x7f2fa406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.565+0000 7f2fc53d8640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fa406c530 0x7f2fa406e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc036000 tx=0x7f2fbc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.571+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2fc00b5ad0 con 0x7f2fc00d0300 2024-03-30T13:10:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.830+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f2fc00cf5c0 con 0x7f2fc00d0300 2024-03-30T13:10:53.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.831+0000 7f2fb5ffb640 1 -- 172.21.15.66:0/2047120389 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f2fb005a100 con 0x7f2fc00d0300 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fa406c530 msgr2=0x7f2fa406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fa406c530 0x7f2fa406e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc036000 tx=0x7f2fbc008030 comp rx=0 tx=0).stop 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 msgr2=0x7f2fc00c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 0x7f2fc00c7820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000d5f0 tx=0x7f2fb000dab0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 shutdown_connections 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2fa406c530 0x7f2fa406e9f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc0155990 0x7f2fc00c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2fc00d0300 0x7f2fc00c7820 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 --2- 172.21.15.66:0/2047120389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2fc00b4050 0x7f2fc0158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 >> 172.21.15.66:0/2047120389 conn(0x7f2fc00672c0 msgr2=0x7f2fc00b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 shutdown_connections 2024-03-30T13:10:53.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.834+0000 7f2fc763c640 1 -- 172.21.15.66:0/2047120389 wait complete. 2024-03-30T13:10:53.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-30T13:10:53.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.954+0000 7f279d838640 1 Processor -- start 2024-03-30T13:10:53.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.954+0000 7f279d838640 1 -- start start 2024-03-30T13:10:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f279d838640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f279815e770 0x7f2798162310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f279d838640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f27980d0560 0x7f27980b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f279d838640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27981628f0 con 0x7f27980d1f20 2024-03-30T13:10:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f279d838640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f27980b5200 con 0x7f27980d0560 2024-03-30T13:10:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f279d838640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f2798162bd0 con 0x7f279815e770 2024-03-30T13:10:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f27977fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f27980d0560 0x7f27980b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f278ffff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f27980d1f20 0x7f27980cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60674/0 (socket says 172.21.15.66:60674) 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f278ffff640 1 -- 172.21.15.66:0/2026348704 learned_addr learned my addr 172.21.15.66:0/2026348704 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f2796ffd640 1 --2- 172.21.15.66:0/2026348704 >> v2:172.21.15.117:3300/0 conn(0x7f279815e770 0x7f2798162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f27977fe640 1 -- 172.21.15.66:0/2026348704 >> v2:172.21.15.117:3300/0 conn(0x7f279815e770 msgr2=0x7f2798162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f27977fe640 1 --2- 172.21.15.66:0/2026348704 >> v2:172.21.15.117:3300/0 conn(0x7f279815e770 0x7f2798162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f27977fe640 1 -- 172.21.15.66:0/2026348704 >> v1:172.21.15.66:6789/0 conn(0x7f27980d1f20 legacy=0x7f27980cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f27977fe640 1 -- 172.21.15.66:0/2026348704 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2798162eb0 con 0x7f27980d0560 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f27967fc640 1 -- 172.21.15.66:0/2026348704 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080949848 0 0) 0x7f27981628f0 con 0x7f27980d1f20 2024-03-30T13:10:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.955+0000 7f2796ffd640 1 --2- 172.21.15.66:0/2026348704 >> v2:172.21.15.117:3300/0 conn(0x7f279815e770 0x7f2798162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:53.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.956+0000 7f27977fe640 1 --2- 172.21.15.66:0/2026348704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27980d0560 0x7f27980b5690 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f27980b5380 tx=0x7f279002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ce66fb0e5f9e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.956+0000 7f27967fc640 1 -- 172.21.15.66:0/2026348704 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790039070 con 0x7f27980d0560 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.956+0000 7f27967fc640 1 -- 172.21.15.66:0/2026348704 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279002cc60 con 0x7f27980d0560 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.956+0000 7f27967fc640 1 -- 172.21.15.66:0/2026348704 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790034660 con 0x7f27980d0560 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 -- 172.21.15.66:0/2026348704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27980d0560 msgr2=0x7f27980b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 --2- 172.21.15.66:0/2026348704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27980d0560 0x7f27980b5690 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f27980b5380 tx=0x7f279002beb0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 -- 172.21.15.66:0/2026348704 shutdown_connections 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 --2- 172.21.15.66:0/2026348704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27980d0560 0x7f27980b5690 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 --2- 172.21.15.66:0/2026348704 >> v2:172.21.15.117:3300/0 conn(0x7f279815e770 0x7f2798162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 -- 172.21.15.66:0/2026348704 >> 172.21.15.66:0/2026348704 conn(0x7f279806f8f0 msgr2=0x7f279806fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 -- 172.21.15.66:0/2026348704 shutdown_connections 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.957+0000 7f279d838640 1 -- 172.21.15.66:0/2026348704 wait complete. 2024-03-30T13:10:53.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.958+0000 7f279d838640 1 Processor -- start 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.958+0000 7f279d838640 1 -- start start 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27980d0560 0x7f27980c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27980d1f20 0x7f2798153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 0x7f27980c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f279806c790 con 0x7f279815e770 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f279815eb50 con 0x7f27980d0560 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f279806c910 con 0x7f27980d1f20 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 0x7f27980c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 0x7f27980c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50196/0 (socket says 172.21.15.66:50196) 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 -- 172.21.15.66:0/2262909759 learned_addr learned my addr 172.21.15.66:0/2262909759 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f2796ffd640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27980d0560 0x7f27980c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f278ffff640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27980d1f20 0x7f2798153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 -- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27980d1f20 msgr2=0x7f2798153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27980d1f20 0x7f2798153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 -- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27980d0560 msgr2=0x7f27980c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27980d0560 0x7f27980c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 -- 172.21.15.66:0/2262909759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27980cc950 con 0x7f279815e770 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f27977fe640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 0x7f27980c77d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f279002c4c0 tx=0x7f2790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790039050 con 0x7f279815e770 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2790045070 con 0x7f279815e770 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27980ccc30 con 0x7f279815e770 2024-03-30T13:10:53.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279003e830 con 0x7f279815e770 2024-03-30T13:10:53.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.959+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27980c8250 con 0x7f279815e770 2024-03-30T13:10:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.960+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2798075900 con 0x7f279815e770 2024-03-30T13:10:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.961+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f279003e9d0 con 0x7f279815e770 2024-03-30T13:10:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.961+0000 7f2794ff9640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f277806c530 0x7f277806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.961+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f27900ae260 con 0x7f279815e770 2024-03-30T13:10:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.961+0000 7f2796ffd640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f277806c530 0x7f277806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.962+0000 7f2796ffd640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f277806c530 0x7f277806e9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f278800a930 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:53.966+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f279007cad0 con 0x7f279815e770 2024-03-30T13:10:54.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.228+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f27980b8200 con 0x7f279815e770 2024-03-30T13:10:54.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.230+0000 7f2794ff9640 1 -- 172.21.15.66:0/2262909759 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f279007c3d0 con 0x7f279815e770 2024-03-30T13:10:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f277806c530 msgr2=0x7f277806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f277806c530 0x7f277806e9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f278800a930 tx=0x7f2788008040 comp rx=0 tx=0).stop 2024-03-30T13:10:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 msgr2=0x7f27980c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 0x7f27980c77d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f279002c4c0 tx=0x7f2790004a40 comp rx=0 tx=0).stop 2024-03-30T13:10:54.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 shutdown_connections 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f277806c530 0x7f277806e9f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f279815e770 0x7f27980c77d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27980d1f20 0x7f2798153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 --2- 172.21.15.66:0/2262909759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27980d0560 0x7f27980c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 >> 172.21.15.66:0/2262909759 conn(0x7f279806f8f0 msgr2=0x7f27980749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.232+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 shutdown_connections 2024-03-30T13:10:54.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.233+0000 7f279d838640 1 -- 172.21.15.66:0/2262909759 wait complete. 2024-03-30T13:10:54.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.350+0000 7fe93d437640 1 Processor -- start 2024-03-30T13:10:54.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.350+0000 7fe93d437640 1 -- start start 2024-03-30T13:10:54.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe93d437640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe9380cedb0 0x7fe9380cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe93d437640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9380b63e0 0x7fe9380b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe93d437640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9380b7f50 con 0x7fe93815e9c0 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe93d437640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe9380b6d90 con 0x7fe9380cedb0 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe93d437640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe9380b3c80 con 0x7fe9380b63e0 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe9367fc640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe9380cedb0 0x7fe9380cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe9367fc640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe9380cedb0 0x7fe9380cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50202/0 (socket says 172.21.15.66:50202) 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.351+0000 7fe9367fc640 1 -- 172.21.15.66:0/2871192581 learned_addr learned my addr 172.21.15.66:0/2871192581 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.352+0000 7fe9377fe640 1 --2- 172.21.15.66:0/2871192581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9380b63e0 0x7fe9380b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.352+0000 7fe9367fc640 1 -- 172.21.15.66:0/2871192581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9380b63e0 msgr2=0x7fe9380b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.352+0000 7fe9367fc640 1 --2- 172.21.15.66:0/2871192581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9380b63e0 0x7fe9380b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.352+0000 7fe9367fc640 1 -- 172.21.15.66:0/2871192581 >> v1:172.21.15.117:6789/0 conn(0x7fe93815e9c0 legacy=0x7fe9380b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:54.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.352+0000 7fe9367fc640 1 -- 172.21.15.66:0/2871192581 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9380cf970 con 0x7fe9380cedb0 2024-03-30T13:10:54.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.352+0000 7fe9367fc640 1 --2- 172.21.15.66:0/2871192581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe9380cf220 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009870 tx=0x7fe92c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda803718c57c8cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe935ffb640 1 -- 172.21.15.66:0/2871192581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c034070 con 0x7fe9380cedb0 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe935ffb640 1 -- 172.21.15.66:0/2871192581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c030840 con 0x7fe9380cedb0 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe935ffb640 1 -- 172.21.15.66:0/2871192581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c030c10 con 0x7fe9380cedb0 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 -- 172.21.15.66:0/2871192581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 msgr2=0x7fe9380cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 --2- 172.21.15.66:0/2871192581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe9380cf220 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009870 tx=0x7fe92c02b980 comp rx=0 tx=0).stop 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 -- 172.21.15.66:0/2871192581 shutdown_connections 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 --2- 172.21.15.66:0/2871192581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9380b63e0 0x7fe9380b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 --2- 172.21.15.66:0/2871192581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe9380cf220 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 -- 172.21.15.66:0/2871192581 >> 172.21.15.66:0/2871192581 conn(0x7fe9380676e0 msgr2=0x7fe938067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 -- 172.21.15.66:0/2871192581 shutdown_connections 2024-03-30T13:10:54.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.353+0000 7fe93d437640 1 -- 172.21.15.66:0/2871192581 wait complete. 2024-03-30T13:10:54.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 Processor -- start 2024-03-30T13:10:54.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 -- start start 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9380b63e0 0x7fe93815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe938154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe93815e9c0 0x7fe938156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380d27a0 con 0x7fe9380cedb0 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380d2560 con 0x7fe9380b63e0 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe93d437640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe9380d2a80 con 0x7fe93815e9c0 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe9367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe938154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe9367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe938154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50206/0 (socket says 172.21.15.66:50206) 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.355+0000 7fe9367fc640 1 -- 172.21.15.66:0/3686933304 learned_addr learned my addr 172.21.15.66:0/3686933304 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe936ffd640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9380b63e0 0x7fe93815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9377fe640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe93815e9c0 0x7fe938156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9367fc640 1 -- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe93815e9c0 msgr2=0x7fe938156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9367fc640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe93815e9c0 0x7fe938156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9367fc640 1 -- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9380b63e0 msgr2=0x7fe93815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9367fc640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9380b63e0 0x7fe93815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9367fc640 1 -- 172.21.15.66:0/3686933304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9381573c0 con 0x7fe9380cedb0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe936ffd640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9380b63e0 0x7fe93815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9377fe640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe93815e9c0 0x7fe938156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe9367fc640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe938154480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe92c000c00 tx=0x7fe92c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c048070 con 0x7fe9380cedb0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c030640 con 0x7fe9380cedb0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93815f4a0 con 0x7fe9380cedb0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c034040 con 0x7fe9380cedb0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.356+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93815fa60 con 0x7fe9380cedb0 2024-03-30T13:10:54.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.357+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938075a10 con 0x7fe9380cedb0 2024-03-30T13:10:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.358+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe92c04b070 con 0x7fe9380cedb0 2024-03-30T13:10:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.358+0000 7fe91f7fe640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe92406c530 0x7fe92406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.358+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fe92c0ad3a0 con 0x7fe9380cedb0 2024-03-30T13:10:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.358+0000 7fe936ffd640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe92406c530 0x7fe92406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.359+0000 7fe936ffd640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe92406c530 0x7fe92406e9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe928001660 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.363+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe92c07bd30 con 0x7fe9380cedb0 2024-03-30T13:10:54.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.549+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9380b8f60 con 0x7fe9380cedb0 2024-03-30T13:10:54.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.549+0000 7fe91f7fe640 1 -- 172.21.15.66:0/3686933304 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13152 (secure 0 0 0) 0x7fe92c07b630 con 0x7fe9380cedb0 2024-03-30T13:10:54.553 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:54.553 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:51.464094+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe92406c530 msgr2=0x7fe92406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe92406c530 0x7fe92406e9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe928001660 tx=0x7fe928009040 comp rx=0 tx=0).stop 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 msgr2=0x7fe938154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe938154480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe92c000c00 tx=0x7fe92c004730 comp rx=0 tx=0).stop 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 shutdown_connections 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe92406c530 0x7fe92406e9f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe93815e9c0 0x7fe938156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9380cedb0 0x7fe938154480 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 --2- 172.21.15.66:0/3686933304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9380b63e0 0x7fe93815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 >> 172.21.15.66:0/3686933304 conn(0x7fe9380676e0 msgr2=0x7fe9380d3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 shutdown_connections 2024-03-30T13:10:54.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.552+0000 7fe93d437640 1 -- 172.21.15.66:0/3686933304 wait complete. 2024-03-30T13:10:54.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:10:54.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.674+0000 7f1607fe1640 1 Processor -- start 2024-03-30T13:10:54.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.674+0000 7f1607fe1640 1 -- start start 2024-03-30T13:10:54.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f1607fe1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f160015e7c0 0x7f160006c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f1607fe1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f160015eba0 0x7f160006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f1607fe1640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16000d1170 con 0x7f16000b4800 2024-03-30T13:10:54.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f1607fe1640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f16000cc840 con 0x7f160015eba0 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f1607fe1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f16000d1450 con 0x7f160015e7c0 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f160557c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f160015eba0 0x7f160006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f160657e640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f16000b4800 0x7f160009aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60684/0 (socket says 172.21.15.66:60684) 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f160657e640 1 -- 172.21.15.66:0/4238595487 learned_addr learned my addr 172.21.15.66:0/4238595487 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f1605d7d640 1 --2- 172.21.15.66:0/4238595487 >> v2:172.21.15.117:3300/0 conn(0x7f160015e7c0 0x7f160006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f160557c640 1 -- 172.21.15.66:0/4238595487 >> v2:172.21.15.117:3300/0 conn(0x7f160015e7c0 msgr2=0x7f160006c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f160557c640 1 --2- 172.21.15.66:0/4238595487 >> v2:172.21.15.117:3300/0 conn(0x7f160015e7c0 0x7f160006c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.675+0000 7f160557c640 1 -- 172.21.15.66:0/4238595487 >> v1:172.21.15.66:6789/0 conn(0x7f16000b4800 legacy=0x7f160009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.676+0000 7f160557c640 1 -- 172.21.15.66:0/4238595487 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16000d1730 con 0x7f160015eba0 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.676+0000 7f160557c640 1 --2- 172.21.15.66:0/4238595487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f160006cda0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f160009bc30 tx=0x7f15f002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a89e590bc55ebca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.676+0000 7f1604d7b640 1 -- 172.21.15.66:0/4238595487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0037070 con 0x7f160015eba0 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.676+0000 7f1604d7b640 1 -- 172.21.15.66:0/4238595487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0030710 con 0x7f160015eba0 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.676+0000 7f1604d7b640 1 -- 172.21.15.66:0/4238595487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0030ae0 con 0x7f160015eba0 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 -- 172.21.15.66:0/4238595487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 msgr2=0x7f160006cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 --2- 172.21.15.66:0/4238595487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f160006cda0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f160009bc30 tx=0x7f15f002b710 comp rx=0 tx=0).stop 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 -- 172.21.15.66:0/4238595487 shutdown_connections 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 --2- 172.21.15.66:0/4238595487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f160006cda0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 --2- 172.21.15.66:0/4238595487 >> v2:172.21.15.117:3300/0 conn(0x7f160015e7c0 0x7f160006c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 -- 172.21.15.66:0/4238595487 >> 172.21.15.66:0/4238595487 conn(0x7f16000b3ce0 msgr2=0x7f16000b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 -- 172.21.15.66:0/4238595487 shutdown_connections 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.677+0000 7f1607fe1640 1 -- 172.21.15.66:0/4238595487 wait complete. 2024-03-30T13:10:54.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.678+0000 7f1607fe1640 1 Processor -- start 2024-03-30T13:10:54.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.678+0000 7f1607fe1640 1 -- start start 2024-03-30T13:10:54.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1607fe1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16000b4800 0x7f16000b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1607fe1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160015e7c0 0x7f16000b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1607fe1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f16000b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1607fe1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16000cde60 con 0x7f160015eba0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1607fe1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16000cdce0 con 0x7f16000b4800 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1607fe1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f16000cdfe0 con 0x7f160015e7c0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f16000b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f16000b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50226/0 (socket says 172.21.15.66:50226) 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 -- 172.21.15.66:0/3310957883 learned_addr learned my addr 172.21.15.66:0/3310957883 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f1605d7d640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16000b4800 0x7f16000b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160557c640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160015e7c0 0x7f16000b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 -- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160015e7c0 msgr2=0x7f16000b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160015e7c0 0x7f16000b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 -- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16000b4800 msgr2=0x7f16000b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16000b4800 0x7f16000b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.679+0000 7f160657e640 1 -- 172.21.15.66:0/3310957883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160015efe0 con 0x7f160015eba0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.680+0000 7f160657e640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f16000b6100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00cac0 tx=0x7f15fc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.680+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018070 con 0x7f160015eba0 2024-03-30T13:10:54.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.680+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc00a640 con 0x7f160015eba0 2024-03-30T13:10:54.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.680+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160015f280 con 0x7f160015eba0 2024-03-30T13:10:54.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.680+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00a9f0 con 0x7f160015eba0 2024-03-30T13:10:54.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.680+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160015f840 con 0x7f160015eba0 2024-03-30T13:10:54.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.681+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1600075890 con 0x7f160015eba0 2024-03-30T13:10:54.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.681+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f15fc01b070 con 0x7f160015eba0 2024-03-30T13:10:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.682+0000 7f15f67fc640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f15e406c530 0x7f15e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.682+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f15fc08ae20 con 0x7f160015eba0 2024-03-30T13:10:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.682+0000 7f1605d7d640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f15e406c530 0x7f15e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.682+0000 7f1605d7d640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f15e406c530 0x7f15e406e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15f8006fd0 tx=0x7f15f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.686+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f15fc058f20 con 0x7f160015eba0 2024-03-30T13:10:54.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.869+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f160006cbd0 con 0x7f160015eba0 2024-03-30T13:10:54.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.870+0000 7f15f67fc640 1 -- 172.21.15.66:0/3310957883 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f15fc05d220 con 0x7f160015eba0 2024-03-30T13:10:54.874 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:10:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f15e406c530 msgr2=0x7f15e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f15e406c530 0x7f15e406e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15f8006fd0 tx=0x7f15f8008040 comp rx=0 tx=0).stop 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 msgr2=0x7f16000b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f16000b6100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00cac0 tx=0x7f15fc00cf80 comp rx=0 tx=0).stop 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 shutdown_connections 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f15e406c530 0x7f15e406e9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160015eba0 0x7f16000b6100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160015e7c0 0x7f16000b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 --2- 172.21.15.66:0/3310957883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16000b4800 0x7f16000b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 >> 172.21.15.66:0/3310957883 conn(0x7f16000b3ce0 msgr2=0x7f16000cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 shutdown_connections 2024-03-30T13:10:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.872+0000 7f1607fe1640 1 -- 172.21.15.66:0/3310957883 wait complete. 2024-03-30T13:10:54.887 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T13:10:54.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 Processor -- start 2024-03-30T13:10:54.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 -- start start 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc15e950 0x7fd5cc0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 0x7fd5cc0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5cc0d36f0 con 0x7fd5cc0d48c0 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd5cc0d3cb0 con 0x7fd5cc0d30d0 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.992+0000 7fd5d320d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc0d4d00 con 0x7fd5cc15e950 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d17aa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 0x7fd5cc0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d0fa9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc15e950 0x7fd5cc0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5cbfff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd5cc0d48c0 0x7fd5cc0d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:59582/0 (socket says 172.21.15.66:59582) 2024-03-30T13:10:54.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d17aa640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 0x7fd5cc0b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:43774/0 (socket says 172.21.15.66:43774) 2024-03-30T13:10:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5cbfff640 1 -- 172.21.15.66:0/51515873 learned_addr learned my addr 172.21.15.66:0/51515873 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d0fa9640 1 -- 172.21.15.66:0/51515873 >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 msgr2=0x7fd5cc0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d0fa9640 1 --2- 172.21.15.66:0/51515873 >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 0x7fd5cc0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d0fa9640 1 -- 172.21.15.66:0/51515873 >> v1:172.21.15.117:6789/0 conn(0x7fd5cc0d48c0 legacy=0x7fd5cc0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d0fa9640 1 -- 172.21.15.66:0/51515873 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc0d39d0 con 0x7fd5cc15e950 2024-03-30T13:10:54.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.993+0000 7fd5d17aa640 1 --2- 172.21.15.66:0/51515873 >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 0x7fd5cc0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.994+0000 7fd5d0fa9640 1 --2- 172.21.15.66:0/51515873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc15e950 0x7fd5cc0d4380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00097a0 tx=0x7fd5c002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47f2f82c21791cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.994+0000 7fd5cb7fe640 1 -- 172.21.15.66:0/51515873 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0034070 con 0x7fd5cc15e950 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.994+0000 7fd5cb7fe640 1 -- 172.21.15.66:0/51515873 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c0030610 con 0x7fd5cc15e950 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.994+0000 7fd5cb7fe640 1 -- 172.21.15.66:0/51515873 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00309e0 con 0x7fd5cc15e950 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 -- 172.21.15.66:0/51515873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc15e950 msgr2=0x7fd5cc0d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 --2- 172.21.15.66:0/51515873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc15e950 0x7fd5cc0d4380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00097a0 tx=0x7fd5c002b960 comp rx=0 tx=0).stop 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 -- 172.21.15.66:0/51515873 shutdown_connections 2024-03-30T13:10:55.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 --2- 172.21.15.66:0/51515873 >> v2:172.21.15.117:3300/0 conn(0x7fd5cc0d30d0 0x7fd5cc0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 --2- 172.21.15.66:0/51515873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc15e950 0x7fd5cc0d4380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 -- 172.21.15.66:0/51515873 >> 172.21.15.66:0/51515873 conn(0x7fd5cc0676e0 msgr2=0x7fd5cc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 -- 172.21.15.66:0/51515873 shutdown_connections 2024-03-30T13:10:55.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.995+0000 7fd5d320d640 1 -- 172.21.15.66:0/51515873 wait complete. 2024-03-30T13:10:55.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.996+0000 7fd5d320d640 1 Processor -- start 2024-03-30T13:10:55.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.996+0000 7fd5d320d640 1 -- start start 2024-03-30T13:10:55.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d320d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc0d30d0 0x7fd5cc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d320d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5cc0d48c0 0x7fd5cc155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d320d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 0x7fd5cc15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d320d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc15ed30 con 0x7fd5cc15e950 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d320d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc0b7da0 con 0x7fd5cc0d48c0 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d320d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc15eeb0 con 0x7fd5cc0d30d0 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 0x7fd5cc15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 0x7fd5cc15f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50230/0 (socket says 172.21.15.66:50230) 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 -- 172.21.15.66:0/2181576063 learned_addr learned my addr 172.21.15.66:0/2181576063 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d0fa9640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc0d30d0 0x7fd5cc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5cbfff640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5cc0d48c0 0x7fd5cc155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 -- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc0d30d0 msgr2=0x7fd5cc155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc0d30d0 0x7fd5cc155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 -- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5cc0d48c0 msgr2=0x7fd5cc155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5cc0d48c0 0x7fd5cc155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d17aa640 1 -- 172.21.15.66:0/2181576063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc160440 con 0x7fd5cc15e950 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.997+0000 7fd5d0fa9640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc0d30d0 0x7fd5cc155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.998+0000 7fd5d17aa640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 0x7fd5cc15f5e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009f00 tx=0x7fd5c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.998+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4018030 con 0x7fd5cc15e950 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.998+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c400ce50 con 0x7fd5cc15e950 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.998+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc1616f0 con 0x7fd5cc15e950 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.998+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4003bb0 con 0x7fd5cc15e950 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.998+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc161cb0 con 0x7fd5cc15e950 2024-03-30T13:10:55.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.999+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc075a10 con 0x7fd5cc15e950 2024-03-30T13:10:55.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.999+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd5c4005d10 con 0x7fd5cc15e950 2024-03-30T13:10:55.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:54.999+0000 7fd5c97fa640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5b006c530 0x7fd5b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.000+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fd5c408bda0 con 0x7fd5cc15e950 2024-03-30T13:10:55.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.000+0000 7fd5d0fa9640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5b006c530 0x7fd5b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.000+0000 7fd5d0fa9640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5b006c530 0x7fd5b006e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0036000 tx=0x7fd5c003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.004+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd5c4022070 con 0x7fd5cc15e950 2024-03-30T13:10:55.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.190+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd5cc067ac0 con 0x7fd5cc15e950 2024-03-30T13:10:55.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.190+0000 7fd5c97fa640 1 -- 172.21.15.66:0/2181576063 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7fd5c4010040 con 0x7fd5cc15e950 2024-03-30T13:10:55.194 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:55.195 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-03-30T13:10:55.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:10:55.195 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-03-30T13:10:55.195 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-03-30T13:10:55.195 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:10:55.195 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.192+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5b006c530 msgr2=0x7fd5b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.192+0000 7fd5d320d640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5b006c530 0x7fd5b006e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0036000 tx=0x7fd5c003b040 comp rx=0 tx=0).stop 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.192+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 msgr2=0x7fd5cc15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.192+0000 7fd5d320d640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 0x7fd5cc15f5e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009f00 tx=0x7fd5c400c490 comp rx=0 tx=0).stop 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 shutdown_connections 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5b006c530 0x7fd5b006e9f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5cc15e950 0x7fd5cc15f5e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5cc0d48c0 0x7fd5cc155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 --2- 172.21.15.66:0/2181576063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5cc0d30d0 0x7fd5cc155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 >> 172.21.15.66:0/2181576063 conn(0x7fd5cc0676e0 msgr2=0x7fd5cc0cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 shutdown_connections 2024-03-30T13:10:55.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.193+0000 7fd5d320d640 1 -- 172.21.15.66:0/2181576063 wait complete. 2024-03-30T13:10:55.207 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T13:10:55.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.315+0000 7f32063a8640 1 Processor -- start 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.315+0000 7f32063a8640 1 -- start start 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f32063a8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3200072f80 0x7f3200073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f32063a8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32000b61e0 0x7f3200072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f32063a8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000d0100 con 0x7f32000cdad0 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f32063a8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f32000738a0 con 0x7f32000b61e0 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f32063a8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f320015eb90 con 0x7f3200072f80 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f31ff7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32000b61e0 0x7f3200072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f3204945640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f32000cdad0 0x7f32000cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60686/0 (socket says 172.21.15.66:60686) 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f3204945640 1 -- 172.21.15.66:0/1387204971 learned_addr learned my addr 172.21.15.66:0/1387204971 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f31fffff640 1 --2- 172.21.15.66:0/1387204971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3200072f80 0x7f3200073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f31ff7fe640 1 -- 172.21.15.66:0/1387204971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3200072f80 msgr2=0x7f3200073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f31ff7fe640 1 --2- 172.21.15.66:0/1387204971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3200072f80 0x7f3200073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f31ff7fe640 1 -- 172.21.15.66:0/1387204971 >> v1:172.21.15.66:6789/0 conn(0x7f32000cdad0 legacy=0x7f32000cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.316+0000 7f31ff7fe640 1 -- 172.21.15.66:0/1387204971 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32000d03a0 con 0x7f32000b61e0 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.317+0000 7f31ff7fe640 1 --2- 172.21.15.66:0/1387204971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32000b61e0 0x7f3200072570 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31f00097a0 tx=0x7f31f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c6b5b2d06d934e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.317+0000 7f31feffd640 1 -- 172.21.15.66:0/1387204971 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00315a0 con 0x7f32000b61e0 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.317+0000 7f31feffd640 1 -- 172.21.15.66:0/1387204971 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f0037070 con 0x7f32000b61e0 2024-03-30T13:10:55.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.317+0000 7f31feffd640 1 -- 172.21.15.66:0/1387204971 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0031960 con 0x7f32000b61e0 2024-03-30T13:10:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 -- 172.21.15.66:0/1387204971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32000b61e0 msgr2=0x7f3200072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 --2- 172.21.15.66:0/1387204971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32000b61e0 0x7f3200072570 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31f00097a0 tx=0x7f31f002b840 comp rx=0 tx=0).stop 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 -- 172.21.15.66:0/1387204971 shutdown_connections 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 --2- 172.21.15.66:0/1387204971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32000b61e0 0x7f3200072570 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 --2- 172.21.15.66:0/1387204971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3200072f80 0x7f3200073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 -- 172.21.15.66:0/1387204971 >> 172.21.15.66:0/1387204971 conn(0x7f32000d10e0 msgr2=0x7f32000d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 -- 172.21.15.66:0/1387204971 shutdown_connections 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.318+0000 7f32063a8640 1 -- 172.21.15.66:0/1387204971 wait complete. 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.319+0000 7f32063a8640 1 Processor -- start 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.319+0000 7f32063a8640 1 -- start start 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f32063a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 0x7f32000afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f32063a8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32000b61e0 0x7f3200161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f32063a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32000cdad0 0x7f3200140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f32063a8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3200072ab0 con 0x7f3200072f80 2024-03-30T13:10:55.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f32063a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32000d0960 con 0x7f32000cdad0 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f32063a8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3200070cc0 con 0x7f32000b61e0 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 0x7f32000afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 0x7f32000afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50246/0 (socket says 172.21.15.66:50246) 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 -- 172.21.15.66:0/1661657282 learned_addr learned my addr 172.21.15.66:0/1661657282 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f3204945640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32000cdad0 0x7f3200140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31ff7fe640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32000b61e0 0x7f3200161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 -- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32000b61e0 msgr2=0x7f3200161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32000b61e0 0x7f3200161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 -- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32000cdad0 msgr2=0x7f3200140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32000cdad0 0x7f3200140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.320+0000 7f31fffff640 1 -- 172.21.15.66:0/1661657282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32000aec70 con 0x7f3200072f80 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f3204945640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32000cdad0 0x7f3200140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f31fffff640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 0x7f32000afd10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f31e800d800 tx=0x7f31e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e800b6c0 con 0x7f3200072f80 2024-03-30T13:10:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e8019070 con 0x7f3200072f80 2024-03-30T13:10:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e800ba80 con 0x7f3200072f80 2024-03-30T13:10:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32000aef50 con 0x7f3200072f80 2024-03-30T13:10:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.321+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32000af510 con 0x7f3200072f80 2024-03-30T13:10:55.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.322+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3200075900 con 0x7f3200072f80 2024-03-30T13:10:55.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.322+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f31e800bc20 con 0x7f3200072f80 2024-03-30T13:10:55.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.323+0000 7f31fcff9640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f31e006c530 0x7f31e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.323+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f31e808b9e0 con 0x7f3200072f80 2024-03-30T13:10:55.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.323+0000 7f31ff7fe640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f31e006c530 0x7f31e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.323+0000 7f31ff7fe640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f31e006c530 0x7f31e006e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f31f0036000 tx=0x7f31f003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.327+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31e805a130 con 0x7f3200072f80 2024-03-30T13:10:55.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.512+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f32000d1ff0 con 0x7f3200072f80 2024-03-30T13:10:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.650+0000 7f31fcff9640 1 -- 172.21.15.66:0/1661657282 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f31e8059a30 con 0x7f3200072f80 2024-03-30T13:10:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-03-30T13:10:55.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f31e006c530 msgr2=0x7f31e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f31e006c530 0x7f31e006e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f31f0036000 tx=0x7f31f003d040 comp rx=0 tx=0).stop 2024-03-30T13:10:55.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 msgr2=0x7f32000afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 0x7f32000afd10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f31e800d800 tx=0x7f31e800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 shutdown_connections 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f31e006c530 0x7f31e006e9f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32000cdad0 0x7f3200140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.652+0000 7f32063a8640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32000b61e0 0x7f3200161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.653+0000 7f32063a8640 1 --2- 172.21.15.66:0/1661657282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3200072f80 0x7f32000afd10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.653+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 >> 172.21.15.66:0/1661657282 conn(0x7f32000d10e0 msgr2=0x7f320006e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.653+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 shutdown_connections 2024-03-30T13:10:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.653+0000 7f32063a8640 1 -- 172.21.15.66:0/1661657282 wait complete. 2024-03-30T13:10:55.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T13:10:55.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.782+0000 7f821e8b3640 1 Processor -- start 2024-03-30T13:10:55.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.782+0000 7f821e8b3640 1 -- start start 2024-03-30T13:10:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821e8b3640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f821806c850 0x7f8218162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821e8b3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82180d06a0 con 0x7f8218161bc0 2024-03-30T13:10:55.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821e8b3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82180d0980 con 0x7f821806d790 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821e8b3640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f82180d0470 con 0x7f821806c850 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821ce50640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f821806c850 0x7f8218162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f8217fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8218161bc0 0x7f821806d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60688/0 (socket says 172.21.15.66:60688) 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f8217fff640 1 -- 172.21.15.66:0/652005574 learned_addr learned my addr 172.21.15.66:0/652005574 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821ce50640 1 -- 172.21.15.66:0/652005574 >> v1:172.21.15.117:6789/0 conn(0x7f821806d790 legacy=0x7f82180cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.783+0000 7f821ce50640 1 -- 172.21.15.66:0/652005574 >> v1:172.21.15.66:6789/0 conn(0x7f8218161bc0 legacy=0x7f821806d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.784+0000 7f8216ffd640 1 -- 172.21.15.66:0/652005574 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488249524 0 0) 0x7f82180d06a0 con 0x7f8218161bc0 2024-03-30T13:10:55.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.784+0000 7f821ce50640 1 -- 172.21.15.66:0/652005574 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82180d0c60 con 0x7f821806c850 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.784+0000 7f821ce50640 1 --2- 172.21.15.66:0/652005574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f8218162da0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8210009d90 tx=0x7f821002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae29c257b69130a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.784+0000 7f8216ffd640 1 -- 172.21.15.66:0/652005574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210039070 con 0x7f821806c850 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.784+0000 7f8216ffd640 1 -- 172.21.15.66:0/652005574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821002cbd0 con 0x7f821806c850 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.784+0000 7f8216ffd640 1 -- 172.21.15.66:0/652005574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210034440 con 0x7f821806c850 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 -- 172.21.15.66:0/652005574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 msgr2=0x7f8218162da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 --2- 172.21.15.66:0/652005574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f8218162da0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8210009d90 tx=0x7f821002be20 comp rx=0 tx=0).stop 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 -- 172.21.15.66:0/652005574 shutdown_connections 2024-03-30T13:10:55.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 --2- 172.21.15.66:0/652005574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f8218162da0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 -- 172.21.15.66:0/652005574 >> 172.21.15.66:0/652005574 conn(0x7f82180b3d00 msgr2=0x7f82180b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 -- 172.21.15.66:0/652005574 shutdown_connections 2024-03-30T13:10:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.785+0000 7f821e8b3640 1 -- 172.21.15.66:0/652005574 wait complete. 2024-03-30T13:10:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.786+0000 7f821e8b3640 1 Processor -- start 2024-03-30T13:10:55.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.786+0000 7f821e8b3640 1 -- start start 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821e8b3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f82180b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821e8b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821806d790 0x7f8218152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821e8b3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8218161bc0 0x7f82180b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821e8b3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8218161fa0 con 0x7f821806c850 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821e8b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82181632e0 con 0x7f821806d790 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821e8b3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8218162120 con 0x7f8218161bc0 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f82180b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f82180b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50256/0 (socket says 172.21.15.66:50256) 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 -- 172.21.15.66:0/1238831902 learned_addr learned my addr 172.21.15.66:0/1238831902 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f82177fe640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821806d790 0x7f8218152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f821ce50640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8218161bc0 0x7f82180b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 -- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8218161bc0 msgr2=0x7f82180b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8218161bc0 0x7f82180b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 -- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821806d790 msgr2=0x7f8218152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.787+0000 7f8217fff640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821806d790 0x7f8218152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f8217fff640 1 -- 172.21.15.66:0/1238831902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82180b5e20 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f82177fe640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821806d790 0x7f8218152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f8217fff640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f82180b5840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c430 tx=0x7f820c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c0074e0 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c0047e0 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82180b6100 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c004bd0 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.788+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82180c5ee0 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.789+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82180b6620 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.790+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f820c011070 con 0x7f821806c850 2024-03-30T13:10:55.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.790+0000 7f8214ff9640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f820406c530 0x7f820406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.790+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f820c08b3c0 con 0x7f821806c850 2024-03-30T13:10:55.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.790+0000 7f82177fe640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f820406c530 0x7f820406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.791+0000 7f82177fe640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f820406c530 0x7f820406e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8218160e80 tx=0x7f8208009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.795+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f82180b6620 con 0x7f821806c850 2024-03-30T13:10:55.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:55.984+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8218075890 con 0x7f821806c850 2024-03-30T13:10:56.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.660+0000 7f8214ff9640 1 -- 172.21.15.66:0/1238831902 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f820c00e280 con 0x7f821806c850 2024-03-30T13:10:56.664 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-03-30T13:10:56.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f820406c530 msgr2=0x7f820406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f820406c530 0x7f820406e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8218160e80 tx=0x7f8208009040 comp rx=0 tx=0).stop 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 msgr2=0x7f82180b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f82180b5840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c430 tx=0x7f820c00c8f0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 shutdown_connections 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f820406c530 0x7f820406e9f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8218161bc0 0x7f82180b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821806d790 0x7f8218152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 --2- 172.21.15.66:0/1238831902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f821806c850 0x7f82180b5840 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 >> 172.21.15.66:0/1238831902 conn(0x7f82180b3d00 msgr2=0x7f8218070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 shutdown_connections 2024-03-30T13:10:56.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.663+0000 7f821e8b3640 1 -- 172.21.15.66:0/1238831902 wait complete. 2024-03-30T13:10:56.692 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T13:10:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.801+0000 7f82b5b90640 1 Processor -- start 2024-03-30T13:10:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.801+0000 7f82b5b90640 1 -- start start 2024-03-30T13:10:56.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82b5b90640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 0x7f82b00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82b5b90640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f82b015f800 0x7f82b00ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82b5b90640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82b015fcb0 con 0x7f82b0144cb0 2024-03-30T13:10:56.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82b5b90640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f82b00b3e80 con 0x7f82b015f800 2024-03-30T13:10:56.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82b5b90640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f82b00b35e0 con 0x7f82b0072d50 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82af7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f82b0144cb0 0x7f82b0072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60698/0 (socket says 172.21.15.66:60698) 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82af7fe640 1 -- 172.21.15.66:0/84348757 learned_addr learned my addr 172.21.15.66:0/84348757 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82aeffd640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 0x7f82b00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317035798 0 0) 0x7f82a0001490 con 0x7f82b0144cb0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82affff640 1 --2- 172.21.15.66:0/84348757 >> v2:172.21.15.117:3300/0 conn(0x7f82b015f800 0x7f82b00ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.802+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8298003520 con 0x7f82b0144cb0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4098342510 0 0) 0x7f8298003520 con 0x7f82b0144cb0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 msgr2=0x7f82b00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 0x7f82b00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 >> v2:172.21.15.117:3300/0 conn(0x7f82b015f800 msgr2=0x7f82b00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 --2- 172.21.15.66:0/84348757 >> v2:172.21.15.117:3300/0 conn(0x7f82b015f800 0x7f82b00ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b00b38c0 con 0x7f82b0144cb0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f82a0001bc0 con 0x7f82b0144cb0 2024-03-30T13:10:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 >> v1:172.21.15.66:6789/0 conn(0x7f82b0144cb0 legacy=0x7f82b0072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82980039b0 0x7f8298003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8298004800 0x7f8298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 0x7f82980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00b3e80 con 0x7f8298004800 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00b35e0 con 0x7f82980039b0 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f829800b9c0 con 0x7f82b0072d50 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82aeffd640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8298004800 0x7f8298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82af7fe640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82980039b0 0x7f8298003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82affff640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 0x7f82980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82aeffd640 1 -- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 msgr2=0x7f82980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82aeffd640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82b0072d50 0x7f82980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82aeffd640 1 -- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82980039b0 msgr2=0x7f8298003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82aeffd640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82980039b0 0x7f8298003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.803+0000 7f82aeffd640 1 -- 172.21.15.66:0/84348757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8298003520 con 0x7f8298004800 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.804+0000 7f82aeffd640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8298004800 0x7f8298004c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009ac0 tx=0x7f8298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70ad7ba558a48e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.804+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4039070 con 0x7f8298004800 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.804+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a402c6b0 con 0x7f8298004800 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.804+0000 7f82ae7fc640 1 -- 172.21.15.66:0/84348757 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4009690 con 0x7f8298004800 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82b5b90640 1 -- 172.21.15.66:0/84348757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8298004800 msgr2=0x7f8298004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82b5b90640 1 --2- 172.21.15.66:0/84348757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8298004800 0x7f8298004c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009ac0 tx=0x7f8298005830 comp rx=0 tx=0).stop 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82affff640 1 -- 172.21.15.66:0/84348757 reap_dead start 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82b5b90640 1 -- 172.21.15.66:0/84348757 shutdown_connections 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82b5b90640 1 -- 172.21.15.66:0/84348757 >> 172.21.15.66:0/84348757 conn(0x7f82b00b5b00 msgr2=0x7f82b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82b5b90640 1 -- 172.21.15.66:0/84348757 shutdown_connections 2024-03-30T13:10:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.805+0000 7f82b5b90640 1 -- 172.21.15.66:0/84348757 wait complete. 2024-03-30T13:10:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.806+0000 7f82b5b90640 1 Processor -- start 2024-03-30T13:10:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.806+0000 7f82b5b90640 1 -- start start 2024-03-30T13:10:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82b5b90640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8298004800 0x7f82b0136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82b5b90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82b015aac0 0x7f82b01f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82b5b90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 0x7f82b015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82b5b90640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00b2f20 con 0x7f82b015d340 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82b5b90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00cdb30 con 0x7f82b015aac0 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82b5b90640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f82b00b30a0 con 0x7f8298004800 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 0x7f82b015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 0x7f82b015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50270/0 (socket says 172.21.15.66:50270) 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 -- 172.21.15.66:0/839896633 learned_addr learned my addr 172.21.15.66:0/839896633 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82af7fe640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8298004800 0x7f82b0136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82aeffd640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82b015aac0 0x7f82b01f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 -- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8298004800 msgr2=0x7f82b0136480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8298004800 0x7f82b0136480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 -- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82b015aac0 msgr2=0x7f82b01f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82b015aac0 0x7f82b01f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 -- 172.21.15.66:0/839896633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b0136b80 con 0x7f82b015d340 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.807+0000 7f82affff640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 0x7f82b015d7c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82b00b53d0 tx=0x7f82a8011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.808+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004030 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.808+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a801b070 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.808+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b0136e60 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.808+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a80043f0 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.808+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b0137420 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.809+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b00687f0 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.810+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f82a8012040 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.810+0000 7f8297fff640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f829c06c560 0x7f829c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.810+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f82a808b810 con 0x7f82b015d340 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.810+0000 7f82af7fe640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f829c06c560 0x7f829c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.811+0000 7f82af7fe640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f829c06c560 0x7f829c06ea20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f82a0002230 tx=0x7f82a0006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:56.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:56.815+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f82b00687f0 con 0x7f82b015d340 2024-03-30T13:10:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.040+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f82b0075890 con 0x7f82b015d340 2024-03-30T13:10:57.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.668+0000 7f8297fff640 1 -- 172.21.15.66:0/839896633 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f82a8059d00 con 0x7f82b015d340 2024-03-30T13:10:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 5 and data pool 6 2024-03-30T13:10:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.674+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f829c06c560 msgr2=0x7f829c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.674+0000 7f82b5b90640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f829c06c560 0x7f829c06ea20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f82a0002230 tx=0x7f82a0006040 comp rx=0 tx=0).stop 2024-03-30T13:10:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.674+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 msgr2=0x7f82b015d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.674+0000 7f82b5b90640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 0x7f82b015d7c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f82b00b53d0 tx=0x7f82a8011c30 comp rx=0 tx=0).stop 2024-03-30T13:10:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.675+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 shutdown_connections 2024-03-30T13:10:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.675+0000 7f82b5b90640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f829c06c560 0x7f829c06ea20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.675+0000 7f82b5b90640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82b015d340 0x7f82b015d7c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.675+0000 7f82b5b90640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82b015aac0 0x7f82b01f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.675+0000 7f82b5b90640 1 --2- 172.21.15.66:0/839896633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8298004800 0x7f82b0136480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.676+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 >> 172.21.15.66:0/839896633 conn(0x7f82b00b5b00 msgr2=0x7f82b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:57.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.684+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 shutdown_connections 2024-03-30T13:10:57.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.686+0000 7f82b5b90640 1 -- 172.21.15.66:0/839896633 wait complete. 2024-03-30T13:10:57.714 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:57.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.823+0000 7f29b599d640 1 Processor -- start 2024-03-30T13:10:57.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.823+0000 7f29b599d640 1 -- start start 2024-03-30T13:10:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b599d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29b00d0d30 0x7f29b00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b599d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29b0161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b599d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b00b5b60 con 0x7f29b00d1110 2024-03-30T13:10:57.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b599d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f29b00c7b90 con 0x7f29b00d0d30 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b599d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00d4fc0 con 0x7f29b00d4950 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b519c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f29b00d1110 0x7f29b00b3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60714/0 (socket says 172.21.15.66:60714) 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b519c640 1 -- 172.21.15.66:0/3373726217 learned_addr learned my addr 172.21.15.66:0/3373726217 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29affff640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29b0161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29b499b640 1 --2- 172.21.15.66:0/3373726217 >> v2:172.21.15.117:3300/0 conn(0x7f29b00d0d30 0x7f29b00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468950860 0 0) 0x7f29b00b5b60 con 0x7f29b00d1110 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29a0003520 con 0x7f29b00d1110 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 521162391 0 0) 0x7f29a0003520 con 0x7f29b00d1110 2024-03-30T13:10:57.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 msgr2=0x7f29b0161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29b0161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 >> v2:172.21.15.117:3300/0 conn(0x7f29b00d0d30 msgr2=0x7f29b00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 --2- 172.21.15.66:0/3373726217 >> v2:172.21.15.117:3300/0 conn(0x7f29b00d0d30 0x7f29b00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00b57f0 con 0x7f29b00d1110 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f29a80019f0 con 0x7f29b00d1110 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 >> v1:172.21.15.66:6789/0 conn(0x7f29b00d1110 legacy=0x7f29b00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.824+0000 7f29af7fe640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29a00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29af7fe640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a0003f30 0x7f29a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29af7fe640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00c7b90 con 0x7f29a0004d40 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00d4fc0 con 0x7f29a0003f30 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f29a000b9f0 con 0x7f29b00d4950 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b519c640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b499b640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29a00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29affff640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a0003f30 0x7f29a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b519c640 1 -- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 msgr2=0x7f29a00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b519c640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29a00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b519c640 1 -- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a0003f30 msgr2=0x7f29a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b519c640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29a0003f30 0x7f29a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b519c640 1 -- 172.21.15.66:0/3373726217 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a0003520 con 0x7f29a0004d40 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.825+0000 7f29b499b640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00d4950 0x7f29a00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29b519c640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f29a800a870 tx=0x7f29a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ea19b88cea362 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800a4c0 con 0x7f29a0004d40 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a8034630 con 0x7f29a0004d40 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29af7fe640 1 -- 172.21.15.66:0/3373726217 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8034a20 con 0x7f29a0004d40 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29b599d640 1 -- 172.21.15.66:0/3373726217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a0004d40 msgr2=0x7f29a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29b599d640 1 --2- 172.21.15.66:0/3373726217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a0004d40 0x7f29a00051c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f29a800a870 tx=0x7f29a0005830 comp rx=0 tx=0).stop 2024-03-30T13:10:57.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29b519c640 1 -- 172.21.15.66:0/3373726217 reap_dead start 2024-03-30T13:10:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29b599d640 1 -- 172.21.15.66:0/3373726217 shutdown_connections 2024-03-30T13:10:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.826+0000 7f29b599d640 1 -- 172.21.15.66:0/3373726217 >> 172.21.15.66:0/3373726217 conn(0x7f29b009d570 msgr2=0x7f29b00d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.827+0000 7f29b599d640 1 -- 172.21.15.66:0/3373726217 shutdown_connections 2024-03-30T13:10:57.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.827+0000 7f29b599d640 1 -- 172.21.15.66:0/3373726217 wait complete. 2024-03-30T13:10:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.828+0000 7f29b599d640 1 Processor -- start 2024-03-30T13:10:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.828+0000 7f29b599d640 1 -- start start 2024-03-30T13:10:57.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b00ca170 0x7f29b01532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00ca550 0x7f29b0153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 0x7f29b00b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00b3300 con 0x7f29b00b78f0 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00b3180 con 0x7f29b00ca170 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00d3150 con 0x7f29b00ca550 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 0x7f29b00b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 0x7f29b00b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50284/0 (socket says 172.21.15.66:50284) 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 -- 172.21.15.66:0/272412003 learned_addr learned my addr 172.21.15.66:0/272412003 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b499b640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b00ca170 0x7f29b01532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29affff640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00ca550 0x7f29b0153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 -- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00ca550 msgr2=0x7f29b0153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00ca550 0x7f29b0153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 -- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b00ca170 msgr2=0x7f29b01532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b00ca170 0x7f29b01532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 -- 172.21.15.66:0/272412003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00b8690 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b499b640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b00ca170 0x7f29b01532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b519c640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 0x7f29b00b7d70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f29a800a870 tx=0x7f29b00cef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8039040 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a8043070 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b00b8970 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4009690 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.829+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b00b8f30 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.831+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b0075890 con 0x7f29b00b78f0 2024-03-30T13:10:57.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.831+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f29a8034a00 con 0x7f29b00b78f0 2024-03-30T13:10:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.831+0000 7f29ad7fa640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f299406c530 0x7f299406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.831+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f29a80ad550 con 0x7f29b00b78f0 2024-03-30T13:10:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.831+0000 7f29b499b640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f299406c530 0x7f299406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.832+0000 7f29b499b640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f299406c530 0x7f299406e9f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009ce0 tx=0x7f29a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:57.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:57.836+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f29a807b9e0 con 0x7f29b00b78f0 2024-03-30T13:10:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.022+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29b00b1a10 con 0x7f29b00b78f0 2024-03-30T13:10:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.023+0000 7f29ad7fa640 1 -- 172.21.15.66:0/272412003 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16943 (secure 0 0 0) 0x7f29a8032e10 con 0x7f29b00b78f0 2024-03-30T13:10:58.027 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:58.027 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:57.041358+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:58.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f299406c530 msgr2=0x7f299406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f299406c530 0x7f299406e9f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009ce0 tx=0x7f29a4008040 comp rx=0 tx=0).stop 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 msgr2=0x7f29b00b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 0x7f29b00b7d70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f29a800a870 tx=0x7f29b00cef80 comp rx=0 tx=0).stop 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 shutdown_connections 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f299406c530 0x7f299406e9f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00b78f0 0x7f29b00b7d70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00ca550 0x7f29b0153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 --2- 172.21.15.66:0/272412003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b00ca170 0x7f29b01532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 >> 172.21.15.66:0/272412003 conn(0x7f29b009d570 msgr2=0x7f29b015f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 shutdown_connections 2024-03-30T13:10:58.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.025+0000 7f29b599d640 1 -- 172.21.15.66:0/272412003 wait complete. 2024-03-30T13:10:58.040 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.143+0000 7f55486c8640 1 Processor -- start 2024-03-30T13:10:58.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.143+0000 7f55486c8640 1 -- start start 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f55486c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f5540162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f55486c8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55400d0860 con 0x7f55400cf8e0 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f55486c8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5540162b80 con 0x7f55400b3cb0 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f55486c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f55400d21e0 con 0x7f55400d0480 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f5545c63640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f55400cf8e0 0x7f5540141da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60730/0 (socket says 172.21.15.66:60730) 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f5545c63640 1 -- 172.21.15.66:0/4233405079 learned_addr learned my addr 172.21.15.66:0/4233405079 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166965885 0 0) 0x7f55400d0860 con 0x7f55400cf8e0 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f5546464640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f5540162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.144+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f552c003520 con 0x7f55400cf8e0 2024-03-30T13:10:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 755085968 0 0) 0x7f552c003520 con 0x7f55400cf8e0 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 msgr2=0x7f5540162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f5540162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 >> v1:172.21.15.117:6789/0 conn(0x7f55400b3cb0 legacy=0x7f55400d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5540162e60 con 0x7f55400cf8e0 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f5530001670 con 0x7f55400cf8e0 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5546464640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f5540162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 >> v1:172.21.15.66:6789/0 conn(0x7f55400cf8e0 legacy=0x7f5540141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f552c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552c003f20 0x7f552c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55400d21e0 con 0x7f552c003f20 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f552c00ba20 con 0x7f552c004d40 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f552c00bc20 con 0x7f55400d0480 2024-03-30T13:10:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545c63640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552c003f20 0x7f552c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5546464640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f552c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5546c65640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545c63640 1 -- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 msgr2=0x7f552c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545c63640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55400d0480 0x7f552c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545c63640 1 -- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c004d40 msgr2=0x7f552c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545c63640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5545c63640 1 -- 172.21.15.66:0/4233405079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c003520 con 0x7f552c003f20 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.145+0000 7f5546c65640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.146+0000 7f5545c63640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552c003f20 0x7f552c004800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f553000c290 tx=0x7f552c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2719a59f6d2b2a03 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.146+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530007d50 con 0x7f552c003f20 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.146+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5530030070 con 0x7f552c003f20 2024-03-30T13:10:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.146+0000 7f5545462640 1 -- 172.21.15.66:0/4233405079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530005d70 con 0x7f552c003f20 2024-03-30T13:10:58.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f55486c8640 1 -- 172.21.15.66:0/4233405079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552c003f20 msgr2=0x7f552c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f55486c8640 1 --2- 172.21.15.66:0/4233405079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f552c003f20 0x7f552c004800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f553000c290 tx=0x7f552c005830 comp rx=0 tx=0).stop 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f5546c65640 1 -- 172.21.15.66:0/4233405079 reap_dead start 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f55486c8640 1 -- 172.21.15.66:0/4233405079 shutdown_connections 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f55486c8640 1 -- 172.21.15.66:0/4233405079 >> 172.21.15.66:0/4233405079 conn(0x7f55400672c0 msgr2=0x7f55400676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f55486c8640 1 -- 172.21.15.66:0/4233405079 shutdown_connections 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.147+0000 7f55486c8640 1 -- 172.21.15.66:0/4233405079 wait complete. 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.148+0000 7f55486c8640 1 Processor -- start 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.148+0000 7f55486c8640 1 -- start start 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c003f20 0x7f55401f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 0x7f55401464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55401469e0 0x7f55400c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55401424b0 con 0x7f55400cbbe0 2024-03-30T13:10:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55400d1e70 con 0x7f55401469e0 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f55400d2d80 con 0x7f552c003f20 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 0x7f55401464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 0x7f55401464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50310/0 (socket says 172.21.15.66:50310) 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 -- 172.21.15.66:0/3457878037 learned_addr learned my addr 172.21.15.66:0/3457878037 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5546464640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c003f20 0x7f55401f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5546c65640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55401469e0 0x7f55400c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 -- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c003f20 msgr2=0x7f55401f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c003f20 0x7f55401f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 -- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55401469e0 msgr2=0x7f55400c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55401469e0 0x7f55400c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 -- 172.21.15.66:0/3457878037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55401f3f10 con 0x7f55400cbbe0 2024-03-30T13:10:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5546c65640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55401469e0 0x7f55400c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5545c63640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 0x7f55401464a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f553000c2f0 tx=0x7f5530001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530007d50 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5530004c10 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.149+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554015e3e0 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.150+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530007070 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.150+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55400c9640 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.151+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55400c9190 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.151+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5530007210 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.152+0000 7f5536ffd640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f552406c560 0x7f552406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.152+0000 7f5546464640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f552406c560 0x7f552406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.152+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f55300ac960 con 0x7f55400cbbe0 2024-03-30T13:10:58.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.152+0000 7f5546464640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f552406c560 0x7f552406ea20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f553c009b10 tx=0x7f553c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:58.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.156+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f55401f3f10 con 0x7f55400cbbe0 2024-03-30T13:10:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.343+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5540075a10 con 0x7f55400cbbe0 2024-03-30T13:10:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.344+0000 7f5536ffd640 1 -- 172.21.15.66:0/3457878037 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16943 (secure 0 0 0) 0x7f553007ae00 con 0x7f55400cbbe0 2024-03-30T13:10:58.348 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:10:58.348 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:57.041358+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f552406c560 msgr2=0x7f552406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f552406c560 0x7f552406ea20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f553c009b10 tx=0x7f553c008040 comp rx=0 tx=0).stop 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 msgr2=0x7f55401464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 0x7f55401464a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f553000c2f0 tx=0x7f5530001a40 comp rx=0 tx=0).stop 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 shutdown_connections 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f552406c560 0x7f552406ea20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55401469e0 0x7f55400c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55400cbbe0 0x7f55401464a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 --2- 172.21.15.66:0/3457878037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c003f20 0x7f55401f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 >> 172.21.15.66:0/3457878037 conn(0x7f55400672c0 msgr2=0x7f5540144640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 shutdown_connections 2024-03-30T13:10:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.346+0000 7f55486c8640 1 -- 172.21.15.66:0/3457878037 wait complete. 2024-03-30T13:10:58.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T13:10:58.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.468+0000 7fc19c042640 1 Processor -- start 2024-03-30T13:10:58.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.469+0000 7fc19c042640 1 -- start start 2024-03-30T13:10:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.469+0000 7fc19c042640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1940b9e30 0x7fc1940d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.469+0000 7fc19c042640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1940d2d00 con 0x7fc1940d4b40 2024-03-30T13:10:58.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.469+0000 7fc19c042640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1940d2fe0 con 0x7fc19415e840 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.469+0000 7fc19c042640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc1940d3910 con 0x7fc1940b9e30 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc1995dd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc1940d4b40 0x7fc1940b7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60740/0 (socket says 172.21.15.66:60740) 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc1995dd640 1 -- 172.21.15.66:0/4285639464 learned_addr learned my addr 172.21.15.66:0/4285639464 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc19a5df640 1 --2- 172.21.15.66:0/4285639464 >> v2:172.21.15.117:3300/0 conn(0x7fc1940b9e30 0x7fc1940d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785512860 0 0) 0x7fc1940d2d00 con 0x7fc1940d4b40 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc180003520 con 0x7fc1940d4b40 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1586231538 0 0) 0x7fc180003520 con 0x7fc1940d4b40 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 >> v2:172.21.15.117:3300/0 conn(0x7fc1940b9e30 msgr2=0x7fc1940d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 --2- 172.21.15.66:0/4285639464 >> v2:172.21.15.117:3300/0 conn(0x7fc1940b9e30 0x7fc1940d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 >> v1:172.21.15.117:6789/0 conn(0x7fc19415e840 legacy=0x7fc1940d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1940d32c0 con 0x7fc1940d4b40 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7fc184001670 con 0x7fc1940d4b40 2024-03-30T13:10:58.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 >> v1:172.21.15.66:6789/0 conn(0x7fc1940d4b40 legacy=0x7fc1940b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1800039b0 0x7fc180003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc180004800 0x7fc180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1800051c0 0x7fc18000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940d3910 con 0x7fc1800039b0 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc180014970 con 0x7fc1800051c0 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc180014b70 con 0x7fc180004800 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc199dde640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1800039b0 0x7fc180003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc1995dd640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc180004800 0x7fc180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.470+0000 7fc19a5df640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1800051c0 0x7fc18000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc199dde640 1 -- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc180004800 msgr2=0x7fc180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc199dde640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc180004800 0x7fc180004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc199dde640 1 -- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1800051c0 msgr2=0x7fc18000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc199dde640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1800051c0 0x7fc18000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc19a5df640 1 -- 172.21.15.66:0/4285639464 reap_dead start 2024-03-30T13:10:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc199dde640 1 -- 172.21.15.66:0/4285639464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc180003520 con 0x7fc1800039b0 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.471+0000 7fc199dde640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1800039b0 0x7fc180003dd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc190001750 tx=0x7fc180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a67810318f27f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190001370 con 0x7fc1800039b0 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1900052d0 con 0x7fc1800039b0 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc198ddc640 1 -- 172.21.15.66:0/4285639464 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190003830 con 0x7fc1800039b0 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc19c042640 1 -- 172.21.15.66:0/4285639464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1800039b0 msgr2=0x7fc180003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc19c042640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1800039b0 0x7fc180003dd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc190001750 tx=0x7fc180005830 comp rx=0 tx=0).stop 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc19c042640 1 -- 172.21.15.66:0/4285639464 shutdown_connections 2024-03-30T13:10:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc19c042640 1 --2- 172.21.15.66:0/4285639464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1800039b0 0x7fc180003dd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.472+0000 7fc19c042640 1 -- 172.21.15.66:0/4285639464 >> 172.21.15.66:0/4285639464 conn(0x7fc1940676e0 msgr2=0x7fc194067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.473+0000 7fc19c042640 1 -- 172.21.15.66:0/4285639464 shutdown_connections 2024-03-30T13:10:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.473+0000 7fc19c042640 1 -- 172.21.15.66:0/4285639464 wait complete. 2024-03-30T13:10:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.475+0000 7fc19c042640 1 Processor -- start 2024-03-30T13:10:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.475+0000 7fc19c042640 1 -- start start 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19c042640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1800039b0 0x7fc1941561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19c042640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1941543f0 0x7fc1940b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19c042640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 0x7fc1940c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19c042640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940d3f20 con 0x7fc1940c9120 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19c042640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1940d3da0 con 0x7fc1941543f0 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19c042640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc1940d40a0 con 0x7fc1800039b0 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 0x7fc1940c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 0x7fc1940c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50334/0 (socket says 172.21.15.66:50334) 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 -- 172.21.15.66:0/3641726306 learned_addr learned my addr 172.21.15.66:0/3641726306 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc199dde640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1800039b0 0x7fc1941561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc1995dd640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1941543f0 0x7fc1940b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 -- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1800039b0 msgr2=0x7fc1941561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1800039b0 0x7fc1941561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 -- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1941543f0 msgr2=0x7fc1940b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1941543f0 0x7fc1940b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc19a5df640 1 -- 172.21.15.66:0/3641726306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc194156730 con 0x7fc1940c9120 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.476+0000 7fc199dde640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1800039b0 0x7fc1941561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.477+0000 7fc19a5df640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 0x7fc1940c95a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc1940d5090 tx=0x7fc18c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.477+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00fd80 con 0x7fc1940c9120 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.477+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c015430 con 0x7fc1940c9120 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.477+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc194154ef0 con 0x7fc1940c9120 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.477+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c015820 con 0x7fc1940c9120 2024-03-30T13:10:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.477+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941559f0 con 0x7fc1940c9120 2024-03-30T13:10:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.478+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc194075890 con 0x7fc1940c9120 2024-03-30T13:10:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.478+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc18c0159c0 con 0x7fc1940c9120 2024-03-30T13:10:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.479+0000 7fc18a7fc640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc17806c530 0x7fc17806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.479+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc18c08ca70 con 0x7fc1940c9120 2024-03-30T13:10:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.479+0000 7fc199dde640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc17806c530 0x7fc17806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.479+0000 7fc199dde640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc17806c530 0x7fc17806e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc190033000 tx=0x7fc190009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.483+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc18c019030 con 0x7fc1940c9120 2024-03-30T13:10:58.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:58.714+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc1940d1510 con 0x7fc1940c9120 2024-03-30T13:10:59.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.681+0000 7fc18a7fc640 1 -- 172.21.15.66:0/3641726306 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fc18c010040 con 0x7fc1940c9120 2024-03-30T13:10:59.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc17806c530 msgr2=0x7fc17806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc17806c530 0x7fc17806e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc190033000 tx=0x7fc190009030 comp rx=0 tx=0).stop 2024-03-30T13:10:59.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 msgr2=0x7fc1940c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 0x7fc1940c95a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc1940d5090 tx=0x7fc18c00f580 comp rx=0 tx=0).stop 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 shutdown_connections 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc17806c530 0x7fc17806e9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1940c9120 0x7fc1940c95a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1941543f0 0x7fc1940b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 --2- 172.21.15.66:0/3641726306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1800039b0 0x7fc1941561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 >> 172.21.15.66:0/3641726306 conn(0x7fc1940676e0 msgr2=0x7fc19415ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 shutdown_connections 2024-03-30T13:10:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.684+0000 7fc19c042640 1 -- 172.21.15.66:0/3641726306 wait complete. 2024-03-30T13:10:59.699 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T13:10:59.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T13:10:59.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.806+0000 7f40e3a35640 1 Processor -- start 2024-03-30T13:10:59.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.806+0000 7f40e3a35640 1 -- start start 2024-03-30T13:10:59.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e3a35640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 0x7f40dc0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e3a35640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40dc06c690 0x7f40dc162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e3a35640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40dc1476b0 con 0x7f40dc1472d0 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e3a35640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f40dc148140 con 0x7f40dc06c690 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e3a35640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc071040 con 0x7f40dc06db00 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e17d1640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f40dc1472d0 0x7f40dc06d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60742/0 (socket says 172.21.15.66:60742) 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.807+0000 7f40e17d1640 1 -- 172.21.15.66:0/698787607 learned_addr learned my addr 172.21.15.66:0/698787607 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 0x7f40dc0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40e1fd2640 1 --2- 172.21.15.66:0/698787607 >> v2:172.21.15.117:3300/0 conn(0x7f40dc06c690 0x7f40dc162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052301915 0 0) 0x7f40dc1476b0 con 0x7f40dc1472d0 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40c8003520 con 0x7f40dc1472d0 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3592055296 0 0) 0x7f40c8003520 con 0x7f40dc1472d0 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 msgr2=0x7f40dc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 0x7f40dc0cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 >> v2:172.21.15.117:3300/0 conn(0x7f40dc06c690 msgr2=0x7f40dc162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 --2- 172.21.15.66:0/698787607 >> v2:172.21.15.117:3300/0 conn(0x7f40dc06c690 0x7f40dc162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc06cb00 con 0x7f40dc1472d0 2024-03-30T13:10:59.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f40d8001670 con 0x7f40dc1472d0 2024-03-30T13:10:59.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 >> v1:172.21.15.66:6789/0 conn(0x7f40dc1472d0 legacy=0x7f40dc06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c80039b0 0x7f40c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40c8004800 0x7f40c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 0x7f40c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc071040 con 0x7f40c8004800 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40c800b9c0 con 0x7f40c80039b0 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f40c800bb40 con 0x7f40dc06db00 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40c8004800 0x7f40c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.808+0000 7f40e17d1640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c80039b0 0x7f40c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.809+0000 7f40e1fd2640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 0x7f40c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.809+0000 7f40e0fd0640 1 -- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 msgr2=0x7f40c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.809+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc06db00 0x7f40c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.809+0000 7f40e0fd0640 1 -- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c80039b0 msgr2=0x7f40c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.809+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c80039b0 0x7f40c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.809+0000 7f40e0fd0640 1 -- 172.21.15.66:0/698787607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c8003520 con 0x7f40c8004800 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40c8004800 0x7f40c8004c80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009af0 tx=0x7f40c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae30a7ef40062f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc039070 con 0x7f40c8004800 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40cc02c6b0 con 0x7f40c8004800 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40d3fff640 1 -- 172.21.15.66:0/698787607 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc009690 con 0x7f40c8004800 2024-03-30T13:10:59.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e3a35640 1 -- 172.21.15.66:0/698787607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40c8004800 msgr2=0x7f40c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e3a35640 1 --2- 172.21.15.66:0/698787607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40c8004800 0x7f40c8004c80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009af0 tx=0x7f40c8005830 comp rx=0 tx=0).stop 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e1fd2640 1 -- 172.21.15.66:0/698787607 reap_dead start 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e3a35640 1 -- 172.21.15.66:0/698787607 shutdown_connections 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e3a35640 1 -- 172.21.15.66:0/698787607 >> 172.21.15.66:0/698787607 conn(0x7f40dc0b3e70 msgr2=0x7f40dc0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e3a35640 1 -- 172.21.15.66:0/698787607 shutdown_connections 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.810+0000 7f40e3a35640 1 -- 172.21.15.66:0/698787607 wait complete. 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 Processor -- start 2024-03-30T13:10:59.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 -- start start 2024-03-30T13:10:59.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c8004800 0x7f40dc161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 0x7f40dc142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc15f580 0x7f40dc15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc071850 con 0x7f40dc13fe80 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc071570 con 0x7f40c8004800 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.812+0000 7f40e3a35640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc071b30 con 0x7f40dc15f580 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 0x7f40dc142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 0x7f40dc142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50358/0 (socket says 172.21.15.66:50358) 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 -- 172.21.15.66:0/3433936790 learned_addr learned my addr 172.21.15.66:0/3433936790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e17d1640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c8004800 0x7f40dc161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e1fd2640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc15f580 0x7f40dc15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 -- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc15f580 msgr2=0x7f40dc15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc15f580 0x7f40dc15fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 -- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c8004800 msgr2=0x7f40dc161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c8004800 0x7f40dc161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 -- 172.21.15.66:0/3433936790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc15ec90 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e0fd0640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 0x7f40dc142e80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f40cc037040 tx=0x7f40cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc039050 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40cc045070 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc003a50 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.813+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc145cf0 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.814+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc144900 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.815+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc075890 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.815+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f40cc003bf0 con 0x7f40dc13fe80 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.815+0000 7f40d1ffb640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f40c006c530 0x7f40c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.816+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f40cc0ad270 con 0x7f40dc13fe80 2024-03-30T13:10:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.816+0000 7f40e17d1640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f40c006c530 0x7f40c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.816+0000 7f40e17d1640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f40c006c530 0x7f40c006e9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40d8001550 tx=0x7f40d80078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:10:59.820+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f40cc04e020 con 0x7f40dc13fe80 2024-03-30T13:11:00.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.044+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f40dc0b3b20 con 0x7f40dc13fe80 2024-03-30T13:11:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.683+0000 7f40d1ffb640 1 -- 172.21.15.66:0/3433936790 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f40cc07b200 con 0x7f40dc13fe80 2024-03-30T13:11:00.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f40c006c530 msgr2=0x7f40c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f40c006c530 0x7f40c006e9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40d8001550 tx=0x7f40d80078f0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 msgr2=0x7f40dc142e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 0x7f40dc142e80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f40cc037040 tx=0x7f40cc004730 comp rx=0 tx=0).stop 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 shutdown_connections 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f40c006c530 0x7f40c006e9f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40dc15f580 0x7f40dc15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc13fe80 0x7f40dc142e80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 --2- 172.21.15.66:0/3433936790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40c8004800 0x7f40dc161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 >> 172.21.15.66:0/3433936790 conn(0x7f40dc0b3e70 msgr2=0x7f40dc0b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 shutdown_connections 2024-03-30T13:11:00.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.685+0000 7f40e3a35640 1 -- 172.21.15.66:0/3433936790 wait complete. 2024-03-30T13:11:00.700 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T13:11:00.700 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.803+0000 7f9a87731640 1 Processor -- start 2024-03-30T13:11:00.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.803+0000 7f9a87731640 1 -- start start 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a87731640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a87731640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a800d30d0 0x7f9a800b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a87731640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a800d36f0 con 0x7f9a800d48c0 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a87731640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9a800d3cb0 con 0x7f9a800d30d0 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a87731640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9a800d4d00 con 0x7f9a8015e950 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a800d30d0 0x7f9a800b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9a800d30d0 0x7f9a800b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50364/0 (socket says 172.21.15.66:50364) 2024-03-30T13:11:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 -- 172.21.15.66:0/1334641166 learned_addr learned my addr 172.21.15.66:0/1334641166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a854cd640 1 --2- 172.21.15.66:0/1334641166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 -- 172.21.15.66:0/1334641166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 msgr2=0x7f9a800d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 --2- 172.21.15.66:0/1334641166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 -- 172.21.15.66:0/1334641166 >> v1:172.21.15.117:6789/0 conn(0x7f9a800d48c0 legacy=0x7f9a800d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.804+0000 7f9a85cce640 1 -- 172.21.15.66:0/1334641166 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800d39d0 con 0x7f9a800d30d0 2024-03-30T13:11:00.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.805+0000 7f9a85cce640 1 --2- 172.21.15.66:0/1334641166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800b82c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009e50 tx=0x7f9a7c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85802f76af59b03c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:00.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.805+0000 7f9a77fff640 1 -- 172.21.15.66:0/1334641166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c02cb90 con 0x7f9a800d30d0 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.805+0000 7f9a77fff640 1 -- 172.21.15.66:0/1334641166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c02cd30 con 0x7f9a800d30d0 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.805+0000 7f9a77fff640 1 -- 172.21.15.66:0/1334641166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c034610 con 0x7f9a800d30d0 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 -- 172.21.15.66:0/1334641166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 msgr2=0x7f9a800b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 --2- 172.21.15.66:0/1334641166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800b82c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009e50 tx=0x7f9a7c02bff0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 -- 172.21.15.66:0/1334641166 shutdown_connections 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 --2- 172.21.15.66:0/1334641166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800b82c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 --2- 172.21.15.66:0/1334641166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 -- 172.21.15.66:0/1334641166 >> 172.21.15.66:0/1334641166 conn(0x7f9a800676e0 msgr2=0x7f9a80067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 -- 172.21.15.66:0/1334641166 shutdown_connections 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.806+0000 7f9a87731640 1 -- 172.21.15.66:0/1334641166 wait complete. 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.807+0000 7f9a87731640 1 Processor -- start 2024-03-30T13:11:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.807+0000 7f9a87731640 1 -- start start 2024-03-30T13:11:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a87731640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a87731640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a800d48c0 0x7f9a80161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a87731640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a87731640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8015ed30 con 0x7f9a800d30d0 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a87731640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a800b7da0 con 0x7f9a800d48c0 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a87731640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8015eeb0 con 0x7f9a8015e950 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50370/0 (socket says 172.21.15.66:50370) 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 -- 172.21.15.66:0/2111768557 learned_addr learned my addr 172.21.15.66:0/2111768557 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a85cce640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a84ccc640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a800d48c0 0x7f9a80161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 -- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 msgr2=0x7f9a800ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 -- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a800d48c0 msgr2=0x7f9a80161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a800d48c0 0x7f9a80161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 -- 172.21.15.66:0/2111768557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800c8590 con 0x7f9a800d30d0 2024-03-30T13:11:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a84ccc640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a800d48c0 0x7f9a80161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.808+0000 7f9a854cd640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800c7f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800d800 tx=0x7f9a7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.809+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800b6c0 con 0x7f9a800d30d0 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.809+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a78019070 con 0x7f9a800d30d0 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.809+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a800c8870 con 0x7f9a800d30d0 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.809+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800ba80 con 0x7f9a800d30d0 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.809+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a800c8e30 con 0x7f9a800d30d0 2024-03-30T13:11:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.810+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a80075a10 con 0x7f9a800d30d0 2024-03-30T13:11:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.810+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9a7800bc20 con 0x7f9a800d30d0 2024-03-30T13:11:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.811+0000 7f9a75ffb640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9a6406c530 0x7f9a6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.811+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9a7808bcd0 con 0x7f9a800d30d0 2024-03-30T13:11:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.811+0000 7f9a84ccc640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9a6406c530 0x7f9a6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.811+0000 7f9a84ccc640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9a6406c530 0x7f9a6406e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a800b5b00 tx=0x7f9a70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:00.815+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9a7805a1f0 con 0x7f9a800d30d0 2024-03-30T13:11:01.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.038+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a800b6cf0 con 0x7f9a800d30d0 2024-03-30T13:11:01.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.038+0000 7f9a75ffb640 1 -- 172.21.15.66:0/2111768557 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f9a7805ae20 con 0x7f9a800d30d0 2024-03-30T13:11:01.042 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:01.042 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:57.041663+0000","modified":"2024-03-30T13:11:00.045554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1537031567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1537031567},{"type":"v1","addr":"172.21.15.66:6837","nonce":1537031567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-03-30T13:11:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 16 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9a6406c530 msgr2=0x7f9a6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9a6406c530 0x7f9a6406e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a800b5b00 tx=0x7f9a70009040 comp rx=0 tx=0).stop 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 msgr2=0x7f9a800c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800c7f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800d800 tx=0x7f9a7800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 shutdown_connections 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9a6406c530 0x7f9a6406e9f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a8015e950 0x7f9a800ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a800d48c0 0x7f9a80161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 --2- 172.21.15.66:0/2111768557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a800d30d0 0x7f9a800c7f00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 >> 172.21.15.66:0/2111768557 conn(0x7f9a800676e0 msgr2=0x7f9a8015dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 shutdown_connections 2024-03-30T13:11:01.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.041+0000 7f9a87731640 1 -- 172.21.15.66:0/2111768557 wait complete. 2024-03-30T13:11:01.056 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:01.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.159+0000 7fe4db104640 1 Processor -- start 2024-03-30T13:11:01.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.160+0000 7fe4db104640 1 -- start start 2024-03-30T13:11:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.160+0000 7fe4db104640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe4d40b92d0 0x7fe4d40d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.160+0000 7fe4db104640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d40d36f0 con 0x7fe4d415e950 2024-03-30T13:11:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.160+0000 7fe4db104640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d40d39d0 con 0x7fe4d40d4a90 2024-03-30T13:11:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.160+0000 7fe4db104640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d40d3cb0 con 0x7fe4d40b92d0 2024-03-30T13:11:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d8ea0640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe4d415e950 0x7fe4d40d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60756/0 (socket says 172.21.15.66:60756) 2024-03-30T13:11:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d8ea0640 1 -- 172.21.15.66:0/719850263 learned_addr learned my addr 172.21.15.66:0/719850263 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d96a1640 1 --2- 172.21.15.66:0/719850263 >> v2:172.21.15.117:3300/0 conn(0x7fe4d40b92d0 0x7fe4d40d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151434816 0 0) 0x7fe4d40d36f0 con 0x7fe4d415e950 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4c0003520 con 0x7fe4d415e950 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1091138608 0 0) 0x7fe4c0003520 con 0x7fe4d415e950 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 >> v2:172.21.15.117:3300/0 conn(0x7fe4d40b92d0 msgr2=0x7fe4d40d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 --2- 172.21.15.66:0/719850263 >> v2:172.21.15.117:3300/0 conn(0x7fe4d40b92d0 0x7fe4d40d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 >> v1:172.21.15.117:6789/0 conn(0x7fe4d40d4a90 legacy=0x7fe4d40d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d40b7ac0 con 0x7fe4d415e950 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7fe4c4001670 con 0x7fe4d415e950 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.161+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 >> v1:172.21.15.66:6789/0 conn(0x7fe4d415e950 legacy=0x7fe4d40d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d37fe640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c00039b0 0x7fe4c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d37fe640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c00043b0 0x7fe4c0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d37fe640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40d3cb0 con 0x7fe4c0005080 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c000bf30 con 0x7fe4c00039b0 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c000c130 con 0x7fe4c00043b0 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d96a1640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d8ea0640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c00039b0 0x7fe4c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d3fff640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c00043b0 0x7fe4c0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d96a1640 1 -- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c00043b0 msgr2=0x7fe4c0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d96a1640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c00043b0 0x7fe4c0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d96a1640 1 -- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c00039b0 msgr2=0x7fe4c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d96a1640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c00039b0 0x7fe4c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d96a1640 1 -- 172.21.15.66:0/719850263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0003520 con 0x7fe4c0005080 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.162+0000 7fe4d8ea0640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c00039b0 0x7fe4c0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4d96a1640 1 -- 172.21.15.66:0/719850263 reap_dead start 2024-03-30T13:11:01.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4d96a1640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4c000b9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009f90 tx=0x7fe4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb439c42da84d55b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4007d30 con 0x7fe4c0005080 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc02cc20 con 0x7fe4c0005080 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4d37fe640 1 -- 172.21.15.66:0/719850263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc009c10 con 0x7fe4c0005080 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4db104640 1 -- 172.21.15.66:0/719850263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 msgr2=0x7fe4c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.163+0000 7fe4db104640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4c000b9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009f90 tx=0x7fe4c0005830 comp rx=0 tx=0).stop 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.164+0000 7fe4db104640 1 -- 172.21.15.66:0/719850263 shutdown_connections 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.164+0000 7fe4db104640 1 --2- 172.21.15.66:0/719850263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4c000b9c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.164+0000 7fe4db104640 1 -- 172.21.15.66:0/719850263 >> 172.21.15.66:0/719850263 conn(0x7fe4d40676e0 msgr2=0x7fe4d4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.164+0000 7fe4db104640 1 -- 172.21.15.66:0/719850263 shutdown_connections 2024-03-30T13:11:01.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.164+0000 7fe4db104640 1 -- 172.21.15.66:0/719850263 wait complete. 2024-03-30T13:11:01.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.165+0000 7fe4db104640 1 Processor -- start 2024-03-30T13:11:01.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.165+0000 7fe4db104640 1 -- start start 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4db104640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4d414dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4db104640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d414e1f0 0x7fe4d4161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4db104640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d4161580 0x7fe4d414bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4db104640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d415eeb0 con 0x7fe4c0005080 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4db104640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d415ed30 con 0x7fe4d414e1f0 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4db104640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40d4380 con 0x7fe4d4161580 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4d414dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4d414dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50388/0 (socket says 172.21.15.66:50388) 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 -- 172.21.15.66:0/2422936650 learned_addr learned my addr 172.21.15.66:0/2422936650 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:01.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d96a1640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d4161580 0x7fe4d414bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d3fff640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d414e1f0 0x7fe4d4161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 -- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d4161580 msgr2=0x7fe4d414bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d4161580 0x7fe4d414bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 -- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d414e1f0 msgr2=0x7fe4d4161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d414e1f0 0x7fe4d4161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 -- 172.21.15.66:0/2422936650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d415c090 con 0x7fe4c0005080 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d3fff640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d414e1f0 0x7fe4d4161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d8ea0640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4d414dcb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4007290 tx=0x7fe4c4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400f960 con 0x7fe4c0005080 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.166+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c4018070 con 0x7fe4c0005080 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.167+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d414aba0 con 0x7fe4c0005080 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.167+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400fd00 con 0x7fe4c0005080 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.167+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d414b160 con 0x7fe4c0005080 2024-03-30T13:11:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.168+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe4c401d460 con 0x7fe4c0005080 2024-03-30T13:11:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.168+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d40687f0 con 0x7fe4c0005080 2024-03-30T13:11:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.169+0000 7fe4d17fa640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe4b406c530 0x7fe4b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.169+0000 7fe4d3fff640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe4b406c530 0x7fe4b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.169+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fe4c408b980 con 0x7fe4c0005080 2024-03-30T13:11:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.169+0000 7fe4d3fff640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe4b406c530 0x7fe4b406e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4d415c440 tx=0x7fe4bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.173+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4d40687f0 con 0x7fe4c0005080 2024-03-30T13:11:01.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.359+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d4075a10 con 0x7fe4c0005080 2024-03-30T13:11:01.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.360+0000 7fe4d17fa640 1 -- 172.21.15.66:0/2422936650 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16930 (secure 0 0 0) 0x7fe4c4059850 con 0x7fe4c0005080 2024-03-30T13:11:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:01.364 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":40,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:10:58.668474+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:01.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe4b406c530 msgr2=0x7fe4b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe4b406c530 0x7fe4b406e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4d415c440 tx=0x7fe4bc009040 comp rx=0 tx=0).stop 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 msgr2=0x7fe4d414dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4d414dcb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4007290 tx=0x7fe4c4007750 comp rx=0 tx=0).stop 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 shutdown_connections 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe4b406c530 0x7fe4b406e9f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d4161580 0x7fe4d414bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d414e1f0 0x7fe4d4161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 --2- 172.21.15.66:0/2422936650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4c0005080 0x7fe4d414dcb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 >> 172.21.15.66:0/2422936650 conn(0x7fe4d40676e0 msgr2=0x7fe4d415daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 shutdown_connections 2024-03-30T13:11:01.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.362+0000 7fe4db104640 1 -- 172.21.15.66:0/2422936650 wait complete. 2024-03-30T13:11:01.377 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T13:11:01.377 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.481+0000 7f41ec172640 1 Processor -- start 2024-03-30T13:11:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.481+0000 7f41ec172640 1 -- start start 2024-03-30T13:11:01.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41ec172640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41e4145090 0x7f41e40b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41ec172640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e40b5aa0 0x7f41e415fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41ec172640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e40b5f80 con 0x7f41e4160370 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41ec172640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f41e41607e0 con 0x7f41e4145090 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41ec172640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f41e40b6260 con 0x7f41e40b5aa0 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e9f0e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41e4145090 0x7f41e40b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e9f0e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f41e4145090 0x7f41e40b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50392/0 (socket says 172.21.15.66:50392) 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e9f0e640 1 -- 172.21.15.66:0/142624230 learned_addr learned my addr 172.21.15.66:0/142624230 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e970d640 1 --2- 172.21.15.66:0/142624230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e40b5aa0 0x7f41e415fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e9f0e640 1 -- 172.21.15.66:0/142624230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e40b5aa0 msgr2=0x7f41e415fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e9f0e640 1 --2- 172.21.15.66:0/142624230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e40b5aa0 0x7f41e415fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.482+0000 7f41e9f0e640 1 -- 172.21.15.66:0/142624230 >> v1:172.21.15.117:6789/0 conn(0x7f41e4160370 legacy=0x7f41e4070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.483+0000 7f41e9f0e640 1 -- 172.21.15.66:0/142624230 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40b6540 con 0x7f41e4145090 2024-03-30T13:11:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.483+0000 7f41e9f0e640 1 --2- 172.21.15.66:0/142624230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e4145090 0x7f41e40b1720 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009870 tx=0x7f41d402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e04f914d8e7782b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.483+0000 7f41e8f0c640 1 -- 172.21.15.66:0/142624230 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40317c0 con 0x7f41e4145090 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.483+0000 7f41e8f0c640 1 -- 172.21.15.66:0/142624230 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d4037070 con 0x7f41e4145090 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.483+0000 7f41e8f0c640 1 -- 172.21.15.66:0/142624230 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4031b80 con 0x7f41e4145090 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 -- 172.21.15.66:0/142624230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e4145090 msgr2=0x7f41e40b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 --2- 172.21.15.66:0/142624230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e4145090 0x7f41e40b1720 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009870 tx=0x7f41d402b860 comp rx=0 tx=0).stop 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 -- 172.21.15.66:0/142624230 shutdown_connections 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 --2- 172.21.15.66:0/142624230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e40b5aa0 0x7f41e415fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 --2- 172.21.15.66:0/142624230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e4145090 0x7f41e40b1720 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 -- 172.21.15.66:0/142624230 >> 172.21.15.66:0/142624230 conn(0x7f41e406f8b0 msgr2=0x7f41e406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 -- 172.21.15.66:0/142624230 shutdown_connections 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.484+0000 7f41ec172640 1 -- 172.21.15.66:0/142624230 wait complete. 2024-03-30T13:11:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.485+0000 7f41ec172640 1 Processor -- start 2024-03-30T13:11:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.485+0000 7f41ec172640 1 -- start start 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ec172640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 0x7f41e41564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ec172640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e4145090 0x7f41e415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ec172640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41e4160370 0x7f41e41f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ec172640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41e4146000 con 0x7f41e40b5aa0 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ec172640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d0c30 con 0x7f41e4160370 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ec172640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f41e4145470 con 0x7f41e4145090 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 0x7f41e41564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 0x7f41e41564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50396/0 (socket says 172.21.15.66:50396) 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 -- 172.21.15.66:0/3615275875 learned_addr learned my addr 172.21.15.66:0/3615275875 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41ea70f640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41e4160370 0x7f41e41f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e970d640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e4145090 0x7f41e415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 -- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e4145090 msgr2=0x7f41e415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e4145090 0x7f41e415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 -- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41e4160370 msgr2=0x7f41e41f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41e4160370 0x7f41e41f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e9f0e640 1 -- 172.21.15.66:0/3615275875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e415e370 con 0x7f41e40b5aa0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.486+0000 7f41e970d640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e4145090 0x7f41e415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.487+0000 7f41e9f0e640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 0x7f41e41564f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009870 tx=0x7f41d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.487+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4032650 con 0x7f41e40b5aa0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.487+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d40327f0 con 0x7f41e40b5aa0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.487+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4032be0 con 0x7f41e40b5aa0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.487+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e415e650 con 0x7f41e40b5aa0 2024-03-30T13:11:01.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.487+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e415ec10 con 0x7f41e40b5aa0 2024-03-30T13:11:01.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.488+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e4075a10 con 0x7f41e40b5aa0 2024-03-30T13:11:01.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.488+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41d404c060 con 0x7f41e40b5aa0 2024-03-30T13:11:01.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.489+0000 7f41da7fc640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41c806c530 0x7f41c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.489+0000 7f41e970d640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41c806c530 0x7f41c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.489+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f41d40acfa0 con 0x7f41e40b5aa0 2024-03-30T13:11:01.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.489+0000 7f41e970d640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41c806c530 0x7f41c806e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.493+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41d4040070 con 0x7f41e40b5aa0 2024-03-30T13:11:01.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.676+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41e40701d0 con 0x7f41c806c530 2024-03-30T13:11:01.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.682+0000 7f41da7fc640 1 -- 172.21.15.66:0/3615275875 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251050 (secure 0 0 0) 0x7f41e40701d0 con 0x7f41c806c530 2024-03-30T13:11:01.689 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:01.690 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:11:01.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41c806c530 msgr2=0x7f41c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41c806c530 0x7f41c806e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 msgr2=0x7f41e41564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 0x7f41e41564f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009870 tx=0x7f41d40046c0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 shutdown_connections 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f41c806c530 0x7f41c806e9f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41e4160370 0x7f41e41f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41e4145090 0x7f41e415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 --2- 172.21.15.66:0/3615275875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e40b5aa0 0x7f41e41564f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 >> 172.21.15.66:0/3615275875 conn(0x7f41e406f8b0 msgr2=0x7f41e415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 shutdown_connections 2024-03-30T13:11:01.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.685+0000 7f41ec172640 1 -- 172.21.15.66:0/3615275875 wait complete. 2024-03-30T13:11:01.700 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-03-30T13:11:00.810218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220852,"kb_used_data":4684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753868,"statfs":{"total":773094113280,"available":772867960832,"internally_reserved":0,"allocated":4796416,"data_stored":2731862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.155701"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672404+0000","last_change":"2024-03-30T13:10:57.684501+0000","last_active":"2024-03-30T13:10:58.672404+0000","last_peered":"2024-03-30T13:10:58.672404+0000","last_clean":"2024-03-30T13:10:58.672404+0000","last_became_active":"2024-03-30T13:10:57.684187+0000","last_became_peered":"2024-03-30T13:10:57.684187+0000","last_unstale":"2024-03-30T13:10:58.672404+0000","last_undegraded":"2024-03-30T13:10:58.672404+0000","last_fullsized":"2024-03-30T13:10:58.672404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:07.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672159+0000","last_change":"2024-03-30T13:10:57.681207+0000","last_active":"2024-03-30T13:10:58.672159+0000","last_peered":"2024-03-30T13:10:58.672159+0000","last_clean":"2024-03-30T13:10:58.672159+0000","last_became_active":"2024-03-30T13:10:57.680890+0000","last_became_peered":"2024-03-30T13:10:57.680890+0000","last_unstale":"2024-03-30T13:10:58.672159+0000","last_undegraded":"2024-03-30T13:10:58.672159+0000","last_fullsized":"2024-03-30T13:10:58.672159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:30:42.096995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745282+0000","last_change":"2024-03-30T13:10:57.695761+0000","last_active":"2024-03-30T13:10:59.745282+0000","last_peered":"2024-03-30T13:10:59.745282+0000","last_clean":"2024-03-30T13:10:59.745282+0000","last_became_active":"2024-03-30T13:10:57.695488+0000","last_became_peered":"2024-03-30T13:10:57.695488+0000","last_unstale":"2024-03-30T13:10:59.745282+0000","last_undegraded":"2024-03-30T13:10:59.745282+0000","last_fullsized":"2024-03-30T13:10:59.745282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678726+0000","last_change":"2024-03-30T13:10:57.682241+0000","last_active":"2024-03-30T13:10:58.678726+0000","last_peered":"2024-03-30T13:10:58.678726+0000","last_clean":"2024-03-30T13:10:58.678726+0000","last_became_active":"2024-03-30T13:10:57.681383+0000","last_became_peered":"2024-03-30T13:10:57.681383+0000","last_unstale":"2024-03-30T13:10:58.678726+0000","last_undegraded":"2024-03-30T13:10:58.678726+0000","last_fullsized":"2024-03-30T13:10:58.678726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:43:40.092786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678584+0000","last_change":"2024-03-30T13:10:57.680923+0000","last_active":"2024-03-30T13:10:58.678584+0000","last_peered":"2024-03-30T13:10:58.678584+0000","last_clean":"2024-03-30T13:10:58.678584+0000","last_became_active":"2024-03-30T13:10:57.680625+0000","last_became_peered":"2024-03-30T13:10:57.680625+0000","last_unstale":"2024-03-30T13:10:58.678584+0000","last_undegraded":"2024-03-30T13:10:58.678584+0000","last_fullsized":"2024-03-30T13:10:58.678584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:27:38.309021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676495+0000","last_change":"2024-03-30T13:10:57.687677+0000","last_active":"2024-03-30T13:10:58.676495+0000","last_peered":"2024-03-30T13:10:58.676495+0000","last_clean":"2024-03-30T13:10:58.676495+0000","last_became_active":"2024-03-30T13:10:57.687348+0000","last_became_peered":"2024-03-30T13:10:57.687348+0000","last_unstale":"2024-03-30T13:10:58.676495+0000","last_undegraded":"2024-03-30T13:10:58.676495+0000","last_fullsized":"2024-03-30T13:10:58.676495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:33:20.876411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.677018+0000","last_change":"2024-03-30T13:10:56.677018+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.677018+0000","last_undegraded":"2024-03-30T13:10:56.677018+0000","last_fullsized":"2024-03-30T13:10:56.677018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671840+0000","last_change":"2024-03-30T13:10:57.679497+0000","last_active":"2024-03-30T13:10:58.671840+0000","last_peered":"2024-03-30T13:10:58.671840+0000","last_clean":"2024-03-30T13:10:58.671840+0000","last_became_active":"2024-03-30T13:10:57.679201+0000","last_became_peered":"2024-03-30T13:10:57.679201+0000","last_unstale":"2024-03-30T13:10:58.671840+0000","last_undegraded":"2024-03-30T13:10:58.671840+0000","last_fullsized":"2024-03-30T13:10:58.671840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:04:54.328931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745602+0000","last_change":"2024-03-30T13:10:57.690985+0000","last_active":"2024-03-30T13:10:59.745602+0000","last_peered":"2024-03-30T13:10:59.745602+0000","last_clean":"2024-03-30T13:10:59.745602+0000","last_became_active":"2024-03-30T13:10:57.690657+0000","last_became_peered":"2024-03-30T13:10:57.690657+0000","last_unstale":"2024-03-30T13:10:59.745602+0000","last_undegraded":"2024-03-30T13:10:59.745602+0000","last_fullsized":"2024-03-30T13:10:59.745602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676334+0000","last_change":"2024-03-30T13:10:56.676334+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676334+0000","last_undegraded":"2024-03-30T13:10:56.676334+0000","last_fullsized":"2024-03-30T13:10:56.676334+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.673354+0000","last_change":"2024-03-30T13:10:56.673354+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.673354+0000","last_undegraded":"2024-03-30T13:10:56.673354+0000","last_fullsized":"2024-03-30T13:10:56.673354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745292+0000","last_change":"2024-03-30T13:10:57.691635+0000","last_active":"2024-03-30T13:10:59.745292+0000","last_peered":"2024-03-30T13:10:59.745292+0000","last_clean":"2024-03-30T13:10:59.745292+0000","last_became_active":"2024-03-30T13:10:57.691340+0000","last_became_peered":"2024-03-30T13:10:57.691340+0000","last_unstale":"2024-03-30T13:10:59.745292+0000","last_undegraded":"2024-03-30T13:10:59.745292+0000","last_fullsized":"2024-03-30T13:10:59.745292+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676222+0000","last_change":"2024-03-30T13:10:56.676222+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676222+0000","last_undegraded":"2024-03-30T13:10:56.676222+0000","last_fullsized":"2024-03-30T13:10:56.676222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672408+0000","last_change":"2024-03-30T13:10:57.686031+0000","last_active":"2024-03-30T13:10:58.672408+0000","last_peered":"2024-03-30T13:10:58.672408+0000","last_clean":"2024-03-30T13:10:58.672408+0000","last_became_active":"2024-03-30T13:10:57.685814+0000","last_became_peered":"2024-03-30T13:10:57.685814+0000","last_unstale":"2024-03-30T13:10:58.672408+0000","last_undegraded":"2024-03-30T13:10:58.672408+0000","last_fullsized":"2024-03-30T13:10:58.672408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676235+0000","last_change":"2024-03-30T13:10:56.676235+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676235+0000","last_undegraded":"2024-03-30T13:10:56.676235+0000","last_fullsized":"2024-03-30T13:10:56.676235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032816+0000","last_change":"2024-03-30T13:10:57.681811+0000","last_active":"2024-03-30T13:10:59.032816+0000","last_peered":"2024-03-30T13:10:59.032816+0000","last_clean":"2024-03-30T13:10:59.032816+0000","last_became_active":"2024-03-30T13:10:57.681471+0000","last_became_peered":"2024-03-30T13:10:57.681471+0000","last_unstale":"2024-03-30T13:10:59.032816+0000","last_undegraded":"2024-03-30T13:10:59.032816+0000","last_fullsized":"2024-03-30T13:10:59.032816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033359+0000","last_change":"2024-03-30T13:10:57.687549+0000","last_active":"2024-03-30T13:10:59.033359+0000","last_peered":"2024-03-30T13:10:59.033359+0000","last_clean":"2024-03-30T13:10:59.033359+0000","last_became_active":"2024-03-30T13:10:57.687268+0000","last_became_peered":"2024-03-30T13:10:57.687268+0000","last_unstale":"2024-03-30T13:10:59.033359+0000","last_undegraded":"2024-03-30T13:10:59.033359+0000","last_fullsized":"2024-03-30T13:10:59.033359+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:35.714074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746089+0000","last_change":"2024-03-30T13:10:57.683841+0000","last_active":"2024-03-30T13:10:59.746089+0000","last_peered":"2024-03-30T13:10:59.746089+0000","last_clean":"2024-03-30T13:10:59.746089+0000","last_became_active":"2024-03-30T13:10:57.683568+0000","last_became_peered":"2024-03-30T13:10:57.683568+0000","last_unstale":"2024-03-30T13:10:59.746089+0000","last_undegraded":"2024-03-30T13:10:59.746089+0000","last_fullsized":"2024-03-30T13:10:59.746089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:51:46.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678919+0000","last_change":"2024-03-30T13:10:57.688521+0000","last_active":"2024-03-30T13:10:58.678919+0000","last_peered":"2024-03-30T13:10:58.678919+0000","last_clean":"2024-03-30T13:10:58.678919+0000","last_became_active":"2024-03-30T13:10:57.688188+0000","last_became_peered":"2024-03-30T13:10:57.688188+0000","last_unstale":"2024-03-30T13:10:58.678919+0000","last_undegraded":"2024-03-30T13:10:58.678919+0000","last_fullsized":"2024-03-30T13:10:58.678919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:47.722209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745911+0000","last_change":"2024-03-30T13:10:57.682779+0000","last_active":"2024-03-30T13:10:59.745911+0000","last_peered":"2024-03-30T13:10:59.745911+0000","last_clean":"2024-03-30T13:10:59.745911+0000","last_became_active":"2024-03-30T13:10:57.682444+0000","last_became_peered":"2024-03-30T13:10:57.682444+0000","last_unstale":"2024-03-30T13:10:59.745911+0000","last_undegraded":"2024-03-30T13:10:59.745911+0000","last_fullsized":"2024-03-30T13:10:59.745911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:02:29.613267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672740+0000","last_change":"2024-03-30T13:10:56.672740+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672740+0000","last_undegraded":"2024-03-30T13:10:56.672740+0000","last_fullsized":"2024-03-30T13:10:56.672740+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672797+0000","last_change":"2024-03-30T13:10:57.692104+0000","last_active":"2024-03-30T13:10:58.672797+0000","last_peered":"2024-03-30T13:10:58.672797+0000","last_clean":"2024-03-30T13:10:58.672797+0000","last_became_active":"2024-03-30T13:10:57.691816+0000","last_became_peered":"2024-03-30T13:10:57.691816+0000","last_unstale":"2024-03-30T13:10:58.672797+0000","last_undegraded":"2024-03-30T13:10:58.672797+0000","last_fullsized":"2024-03-30T13:10:58.672797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032948+0000","last_change":"2024-03-30T13:10:57.691507+0000","last_active":"2024-03-30T13:10:59.032948+0000","last_peered":"2024-03-30T13:10:59.032948+0000","last_clean":"2024-03-30T13:10:59.032948+0000","last_became_active":"2024-03-30T13:10:57.691248+0000","last_became_peered":"2024-03-30T13:10:57.691248+0000","last_unstale":"2024-03-30T13:10:59.032948+0000","last_undegraded":"2024-03-30T13:10:59.032948+0000","last_fullsized":"2024-03-30T13:10:59.032948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678498+0000","last_change":"2024-03-30T13:10:57.685316+0000","last_active":"2024-03-30T13:10:58.678498+0000","last_peered":"2024-03-30T13:10:58.678498+0000","last_clean":"2024-03-30T13:10:58.678498+0000","last_became_active":"2024-03-30T13:10:57.685053+0000","last_became_peered":"2024-03-30T13:10:57.685053+0000","last_unstale":"2024-03-30T13:10:58.678498+0000","last_undegraded":"2024-03-30T13:10:58.678498+0000","last_fullsized":"2024-03-30T13:10:58.678498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:43:19.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746090+0000","last_change":"2024-03-30T13:10:57.684225+0000","last_active":"2024-03-30T13:10:59.746090+0000","last_peered":"2024-03-30T13:10:59.746090+0000","last_clean":"2024-03-30T13:10:59.746090+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:59.746090+0000","last_undegraded":"2024-03-30T13:10:59.746090+0000","last_fullsized":"2024-03-30T13:10:59.746090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:44:06.219419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676846+0000","last_change":"2024-03-30T13:10:57.687484+0000","last_active":"2024-03-30T13:10:58.676846+0000","last_peered":"2024-03-30T13:10:58.676846+0000","last_clean":"2024-03-30T13:10:58.676846+0000","last_became_active":"2024-03-30T13:10:57.687191+0000","last_became_peered":"2024-03-30T13:10:57.687191+0000","last_unstale":"2024-03-30T13:10:58.676846+0000","last_undegraded":"2024-03-30T13:10:58.676846+0000","last_fullsized":"2024-03-30T13:10:58.676846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:49:47.246424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746018+0000","last_change":"2024-03-30T13:10:57.687817+0000","last_active":"2024-03-30T13:10:59.746018+0000","last_peered":"2024-03-30T13:10:59.746018+0000","last_clean":"2024-03-30T13:10:59.746018+0000","last_became_active":"2024-03-30T13:10:57.687038+0000","last_became_peered":"2024-03-30T13:10:57.687038+0000","last_unstale":"2024-03-30T13:10:59.746018+0000","last_undegraded":"2024-03-30T13:10:59.746018+0000","last_fullsized":"2024-03-30T13:10:59.746018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:03:06.653896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.675608+0000","last_change":"2024-03-30T13:10:56.675608+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.675608+0000","last_undegraded":"2024-03-30T13:10:56.675608+0000","last_fullsized":"2024-03-30T13:10:56.675608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745905+0000","last_change":"2024-03-30T13:10:57.690320+0000","last_active":"2024-03-30T13:10:59.745905+0000","last_peered":"2024-03-30T13:10:59.745905+0000","last_clean":"2024-03-30T13:10:59.745905+0000","last_became_active":"2024-03-30T13:10:57.689740+0000","last_became_peered":"2024-03-30T13:10:57.689740+0000","last_unstale":"2024-03-30T13:10:59.745905+0000","last_undegraded":"2024-03-30T13:10:59.745905+0000","last_fullsized":"2024-03-30T13:10:59.745905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:55.823728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745918+0000","last_change":"2024-03-30T13:10:57.683564+0000","last_active":"2024-03-30T13:10:59.745918+0000","last_peered":"2024-03-30T13:10:59.745918+0000","last_clean":"2024-03-30T13:10:59.745918+0000","last_became_active":"2024-03-30T13:10:57.683225+0000","last_became_peered":"2024-03-30T13:10:57.683225+0000","last_unstale":"2024-03-30T13:10:59.745918+0000","last_undegraded":"2024-03-30T13:10:59.745918+0000","last_fullsized":"2024-03-30T13:10:59.745918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:36.633906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032986+0000","last_change":"2024-03-30T13:10:57.692664+0000","last_active":"2024-03-30T13:10:59.032986+0000","last_peered":"2024-03-30T13:10:59.032986+0000","last_clean":"2024-03-30T13:10:59.032986+0000","last_became_active":"2024-03-30T13:10:57.692376+0000","last_became_peered":"2024-03-30T13:10:57.692376+0000","last_unstale":"2024-03-30T13:10:59.032986+0000","last_undegraded":"2024-03-30T13:10:59.032986+0000","last_fullsized":"2024-03-30T13:10:59.032986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676356+0000","last_change":"2024-03-30T13:10:57.688812+0000","last_active":"2024-03-30T13:10:58.676356+0000","last_peered":"2024-03-30T13:10:58.676356+0000","last_clean":"2024-03-30T13:10:58.676356+0000","last_became_active":"2024-03-30T13:10:57.688187+0000","last_became_peered":"2024-03-30T13:10:57.688187+0000","last_unstale":"2024-03-30T13:10:58.676356+0000","last_undegraded":"2024-03-30T13:10:58.676356+0000","last_fullsized":"2024-03-30T13:10:58.676356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:13:34.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033219+0000","last_change":"2024-03-30T13:10:57.682984+0000","last_active":"2024-03-30T13:10:59.033219+0000","last_peered":"2024-03-30T13:10:59.033219+0000","last_clean":"2024-03-30T13:10:59.033219+0000","last_became_active":"2024-03-30T13:10:57.682685+0000","last_became_peered":"2024-03-30T13:10:57.682685+0000","last_unstale":"2024-03-30T13:10:59.033219+0000","last_undegraded":"2024-03-30T13:10:59.033219+0000","last_fullsized":"2024-03-30T13:10:59.033219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:28:51.189027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746047+0000","last_change":"2024-03-30T13:10:56.673822+0000","last_active":"2024-03-30T13:10:59.746047+0000","last_peered":"2024-03-30T13:10:59.746047+0000","last_clean":"2024-03-30T13:10:59.746047+0000","last_became_active":"2024-03-30T13:10:56.673533+0000","last_became_peered":"2024-03-30T13:10:56.673533+0000","last_unstale":"2024-03-30T13:10:59.746047+0000","last_undegraded":"2024-03-30T13:10:59.746047+0000","last_fullsized":"2024-03-30T13:10:59.746047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:51:29.821157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676839+0000","last_change":"2024-03-30T13:10:57.688749+0000","last_active":"2024-03-30T13:10:58.676839+0000","last_peered":"2024-03-30T13:10:58.676839+0000","last_clean":"2024-03-30T13:10:58.676839+0000","last_became_active":"2024-03-30T13:10:57.687457+0000","last_became_peered":"2024-03-30T13:10:57.687457+0000","last_unstale":"2024-03-30T13:10:58.676839+0000","last_undegraded":"2024-03-30T13:10:58.676839+0000","last_fullsized":"2024-03-30T13:10:58.676839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:42.131116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745498+0000","last_change":"2024-03-30T13:10:56.673044+0000","last_active":"2024-03-30T13:10:59.745498+0000","last_peered":"2024-03-30T13:10:59.745498+0000","last_clean":"2024-03-30T13:10:59.745498+0000","last_became_active":"2024-03-30T13:10:56.672750+0000","last_became_peered":"2024-03-30T13:10:56.672750+0000","last_unstale":"2024-03-30T13:10:59.745498+0000","last_undegraded":"2024-03-30T13:10:59.745498+0000","last_fullsized":"2024-03-30T13:10:59.745498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672861+0000","last_change":"2024-03-30T13:10:57.685414+0000","last_active":"2024-03-30T13:10:58.672861+0000","last_peered":"2024-03-30T13:10:58.672861+0000","last_clean":"2024-03-30T13:10:58.672861+0000","last_became_active":"2024-03-30T13:10:57.685131+0000","last_became_peered":"2024-03-30T13:10:57.685131+0000","last_unstale":"2024-03-30T13:10:58.672861+0000","last_undegraded":"2024-03-30T13:10:58.672861+0000","last_fullsized":"2024-03-30T13:10:58.672861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676898+0000","last_change":"2024-03-30T13:10:56.679798+0000","last_active":"2024-03-30T13:10:58.676898+0000","last_peered":"2024-03-30T13:10:58.676898+0000","last_clean":"2024-03-30T13:10:58.676898+0000","last_became_active":"2024-03-30T13:10:56.679500+0000","last_became_peered":"2024-03-30T13:10:56.679500+0000","last_unstale":"2024-03-30T13:10:58.676898+0000","last_undegraded":"2024-03-30T13:10:58.676898+0000","last_fullsized":"2024-03-30T13:10:58.676898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672316+0000","last_change":"2024-03-30T13:10:57.688829+0000","last_active":"2024-03-30T13:10:58.672316+0000","last_peered":"2024-03-30T13:10:58.672316+0000","last_clean":"2024-03-30T13:10:58.672316+0000","last_became_active":"2024-03-30T13:10:57.686054+0000","last_became_peered":"2024-03-30T13:10:57.686054+0000","last_unstale":"2024-03-30T13:10:58.672316+0000","last_undegraded":"2024-03-30T13:10:58.672316+0000","last_fullsized":"2024-03-30T13:10:58.672316+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:03:47.898459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672809+0000","last_change":"2024-03-30T13:10:56.674537+0000","last_active":"2024-03-30T13:10:58.672809+0000","last_peered":"2024-03-30T13:10:58.672809+0000","last_clean":"2024-03-30T13:10:58.672809+0000","last_became_active":"2024-03-30T13:10:56.674248+0000","last_became_peered":"2024-03-30T13:10:56.674248+0000","last_unstale":"2024-03-30T13:10:58.672809+0000","last_undegraded":"2024-03-30T13:10:58.672809+0000","last_fullsized":"2024-03-30T13:10:58.672809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745165+0000","last_change":"2024-03-30T13:10:57.693802+0000","last_active":"2024-03-30T13:10:59.745165+0000","last_peered":"2024-03-30T13:10:59.745165+0000","last_clean":"2024-03-30T13:10:59.745165+0000","last_became_active":"2024-03-30T13:10:57.693436+0000","last_became_peered":"2024-03-30T13:10:57.693436+0000","last_unstale":"2024-03-30T13:10:59.745165+0000","last_undegraded":"2024-03-30T13:10:59.745165+0000","last_fullsized":"2024-03-30T13:10:59.745165+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683595+0000","last_change":"2024-03-30T13:10:56.681576+0000","last_active":"2024-03-30T13:10:56.683595+0000","last_peered":"2024-03-30T13:10:56.683595+0000","last_clean":"2024-03-30T13:10:56.683595+0000","last_became_active":"2024-03-30T13:10:56.681081+0000","last_became_peered":"2024-03-30T13:10:56.681081+0000","last_unstale":"2024-03-30T13:10:56.683595+0000","last_undegraded":"2024-03-30T13:10:56.683595+0000","last_fullsized":"2024-03-30T13:10:56.683595+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:51:58.065326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.675811+0000","last_change":"2024-03-30T13:10:56.675811+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.675811+0000","last_undegraded":"2024-03-30T13:10:56.675811+0000","last_fullsized":"2024-03-30T13:10:56.675811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032969+0000","last_change":"2024-03-30T13:10:56.677670+0000","last_active":"2024-03-30T13:10:59.032969+0000","last_peered":"2024-03-30T13:10:59.032969+0000","last_clean":"2024-03-30T13:10:59.032969+0000","last_became_active":"2024-03-30T13:10:56.677367+0000","last_became_peered":"2024-03-30T13:10:56.677367+0000","last_unstale":"2024-03-30T13:10:59.032969+0000","last_undegraded":"2024-03-30T13:10:59.032969+0000","last_fullsized":"2024-03-30T13:10:59.032969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:14:10.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672971+0000","last_change":"2024-03-30T13:10:57.681840+0000","last_active":"2024-03-30T13:10:58.672971+0000","last_peered":"2024-03-30T13:10:58.672971+0000","last_clean":"2024-03-30T13:10:58.672971+0000","last_became_active":"2024-03-30T13:10:57.681548+0000","last_became_peered":"2024-03-30T13:10:57.681548+0000","last_unstale":"2024-03-30T13:10:58.672971+0000","last_undegraded":"2024-03-30T13:10:58.672971+0000","last_fullsized":"2024-03-30T13:10:58.672971+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676449+0000","last_change":"2024-03-30T13:10:56.682810+0000","last_active":"2024-03-30T13:10:58.676449+0000","last_peered":"2024-03-30T13:10:58.676449+0000","last_clean":"2024-03-30T13:10:58.676449+0000","last_became_active":"2024-03-30T13:10:56.682437+0000","last_became_peered":"2024-03-30T13:10:56.682437+0000","last_unstale":"2024-03-30T13:10:58.676449+0000","last_undegraded":"2024-03-30T13:10:58.676449+0000","last_fullsized":"2024-03-30T13:10:58.676449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672038+0000","last_change":"2024-03-30T13:10:57.678420+0000","last_active":"2024-03-30T13:10:58.672038+0000","last_peered":"2024-03-30T13:10:58.672038+0000","last_clean":"2024-03-30T13:10:58.672038+0000","last_became_active":"2024-03-30T13:10:57.678115+0000","last_became_peered":"2024-03-30T13:10:57.678115+0000","last_unstale":"2024-03-30T13:10:58.672038+0000","last_undegraded":"2024-03-30T13:10:58.672038+0000","last_fullsized":"2024-03-30T13:10:58.672038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:42:22.714841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672311+0000","last_change":"2024-03-30T13:10:56.678877+0000","last_active":"2024-03-30T13:10:58.672311+0000","last_peered":"2024-03-30T13:10:58.672311+0000","last_clean":"2024-03-30T13:10:58.672311+0000","last_became_active":"2024-03-30T13:10:56.678573+0000","last_became_peered":"2024-03-30T13:10:56.678573+0000","last_unstale":"2024-03-30T13:10:58.672311+0000","last_undegraded":"2024-03-30T13:10:58.672311+0000","last_fullsized":"2024-03-30T13:10:58.672311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678784+0000","last_change":"2024-03-30T13:10:57.685526+0000","last_active":"2024-03-30T13:10:58.678784+0000","last_peered":"2024-03-30T13:10:58.678784+0000","last_clean":"2024-03-30T13:10:58.678784+0000","last_became_active":"2024-03-30T13:10:57.681079+0000","last_became_peered":"2024-03-30T13:10:57.681079+0000","last_unstale":"2024-03-30T13:10:58.678784+0000","last_undegraded":"2024-03-30T13:10:58.678784+0000","last_fullsized":"2024-03-30T13:10:58.678784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:36:15.323960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672394+0000","last_change":"2024-03-30T13:10:56.680735+0000","last_active":"2024-03-30T13:10:58.672394+0000","last_peered":"2024-03-30T13:10:58.672394+0000","last_clean":"2024-03-30T13:10:58.672394+0000","last_became_active":"2024-03-30T13:10:56.680459+0000","last_became_peered":"2024-03-30T13:10:56.680459+0000","last_unstale":"2024-03-30T13:10:58.672394+0000","last_undegraded":"2024-03-30T13:10:58.672394+0000","last_fullsized":"2024-03-30T13:10:58.672394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:26.601466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033176+0000","last_change":"2024-03-30T13:10:57.687091+0000","last_active":"2024-03-30T13:10:59.033176+0000","last_peered":"2024-03-30T13:10:59.033176+0000","last_clean":"2024-03-30T13:10:59.033176+0000","last_became_active":"2024-03-30T13:10:57.686821+0000","last_became_peered":"2024-03-30T13:10:57.686821+0000","last_unstale":"2024-03-30T13:10:59.033176+0000","last_undegraded":"2024-03-30T13:10:59.033176+0000","last_fullsized":"2024-03-30T13:10:59.033176+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:16:42.538868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683691+0000","last_change":"2024-03-30T13:10:56.682722+0000","last_active":"2024-03-30T13:10:56.683691+0000","last_peered":"2024-03-30T13:10:56.683691+0000","last_clean":"2024-03-30T13:10:56.683691+0000","last_became_active":"2024-03-30T13:10:56.682272+0000","last_became_peered":"2024-03-30T13:10:56.682272+0000","last_unstale":"2024-03-30T13:10:56.683691+0000","last_undegraded":"2024-03-30T13:10:56.683691+0000","last_fullsized":"2024-03-30T13:10:56.683691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:51:40.244956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745459+0000","last_change":"2024-03-30T13:10:57.691155+0000","last_active":"2024-03-30T13:10:59.745459+0000","last_peered":"2024-03-30T13:10:59.745459+0000","last_clean":"2024-03-30T13:10:59.745459+0000","last_became_active":"2024-03-30T13:10:57.690702+0000","last_became_peered":"2024-03-30T13:10:57.690702+0000","last_unstale":"2024-03-30T13:10:59.745459+0000","last_undegraded":"2024-03-30T13:10:59.745459+0000","last_fullsized":"2024-03-30T13:10:59.745459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683944+0000","last_change":"2024-03-30T13:10:56.683050+0000","last_active":"2024-03-30T13:10:56.683944+0000","last_peered":"2024-03-30T13:10:56.683944+0000","last_clean":"2024-03-30T13:10:56.683944+0000","last_became_active":"2024-03-30T13:10:56.682514+0000","last_became_peered":"2024-03-30T13:10:56.682514+0000","last_unstale":"2024-03-30T13:10:56.683944+0000","last_undegraded":"2024-03-30T13:10:56.683944+0000","last_fullsized":"2024-03-30T13:10:56.683944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:40:52.009831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672597+0000","last_change":"2024-03-30T13:10:57.692928+0000","last_active":"2024-03-30T13:10:58.672597+0000","last_peered":"2024-03-30T13:10:58.672597+0000","last_clean":"2024-03-30T13:10:58.672597+0000","last_became_active":"2024-03-30T13:10:57.692672+0000","last_became_peered":"2024-03-30T13:10:57.692672+0000","last_unstale":"2024-03-30T13:10:58.672597+0000","last_undegraded":"2024-03-30T13:10:58.672597+0000","last_fullsized":"2024-03-30T13:10:58.672597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672248+0000","last_change":"2024-03-30T13:10:56.676836+0000","last_active":"2024-03-30T13:10:58.672248+0000","last_peered":"2024-03-30T13:10:58.672248+0000","last_clean":"2024-03-30T13:10:58.672248+0000","last_became_active":"2024-03-30T13:10:56.676586+0000","last_became_peered":"2024-03-30T13:10:56.676586+0000","last_unstale":"2024-03-30T13:10:58.672248+0000","last_undegraded":"2024-03-30T13:10:58.672248+0000","last_fullsized":"2024-03-30T13:10:58.672248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:15.014930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745111+0000","last_change":"2024-03-30T13:10:57.690831+0000","last_active":"2024-03-30T13:10:59.745111+0000","last_peered":"2024-03-30T13:10:59.745111+0000","last_clean":"2024-03-30T13:10:59.745111+0000","last_became_active":"2024-03-30T13:10:57.690522+0000","last_became_peered":"2024-03-30T13:10:57.690522+0000","last_unstale":"2024-03-30T13:10:59.745111+0000","last_undegraded":"2024-03-30T13:10:59.745111+0000","last_fullsized":"2024-03-30T13:10:59.745111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:16.019984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672474+0000","last_change":"2024-03-30T13:10:56.676370+0000","last_active":"2024-03-30T13:10:58.672474+0000","last_peered":"2024-03-30T13:10:58.672474+0000","last_clean":"2024-03-30T13:10:58.672474+0000","last_became_active":"2024-03-30T13:10:56.675812+0000","last_became_peered":"2024-03-30T13:10:56.675812+0000","last_unstale":"2024-03-30T13:10:58.672474+0000","last_undegraded":"2024-03-30T13:10:58.672474+0000","last_fullsized":"2024-03-30T13:10:58.672474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:24.867217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671859+0000","last_change":"2024-03-30T13:10:57.684924+0000","last_active":"2024-03-30T13:10:58.671859+0000","last_peered":"2024-03-30T13:10:58.671859+0000","last_clean":"2024-03-30T13:10:58.671859+0000","last_became_active":"2024-03-30T13:10:57.684588+0000","last_became_peered":"2024-03-30T13:10:57.684588+0000","last_unstale":"2024-03-30T13:10:58.671859+0000","last_undegraded":"2024-03-30T13:10:58.671859+0000","last_fullsized":"2024-03-30T13:10:58.671859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:20:39.043512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676374+0000","last_change":"2024-03-30T13:10:56.683571+0000","last_active":"2024-03-30T13:10:58.676374+0000","last_peered":"2024-03-30T13:10:58.676374+0000","last_clean":"2024-03-30T13:10:58.676374+0000","last_became_active":"2024-03-30T13:10:56.683215+0000","last_became_peered":"2024-03-30T13:10:56.683215+0000","last_unstale":"2024-03-30T13:10:58.676374+0000","last_undegraded":"2024-03-30T13:10:58.676374+0000","last_fullsized":"2024-03-30T13:10:58.676374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033087+0000","last_change":"2024-03-30T13:10:57.690221+0000","last_active":"2024-03-30T13:10:59.033087+0000","last_peered":"2024-03-30T13:10:59.033087+0000","last_clean":"2024-03-30T13:10:59.033087+0000","last_became_active":"2024-03-30T13:10:57.689942+0000","last_became_peered":"2024-03-30T13:10:57.689942+0000","last_unstale":"2024-03-30T13:10:59.033087+0000","last_undegraded":"2024-03-30T13:10:59.033087+0000","last_fullsized":"2024-03-30T13:10:59.033087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745733+0000","last_change":"2024-03-30T13:10:56.674381+0000","last_active":"2024-03-30T13:10:59.745733+0000","last_peered":"2024-03-30T13:10:59.745733+0000","last_clean":"2024-03-30T13:10:59.745733+0000","last_became_active":"2024-03-30T13:10:56.674086+0000","last_became_peered":"2024-03-30T13:10:56.674086+0000","last_unstale":"2024-03-30T13:10:59.745733+0000","last_undegraded":"2024-03-30T13:10:59.745733+0000","last_fullsized":"2024-03-30T13:10:59.745733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:02.130520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672207+0000","last_change":"2024-03-30T13:10:57.689668+0000","last_active":"2024-03-30T13:10:58.672207+0000","last_peered":"2024-03-30T13:10:58.672207+0000","last_clean":"2024-03-30T13:10:58.672207+0000","last_became_active":"2024-03-30T13:10:57.689363+0000","last_became_peered":"2024-03-30T13:10:57.689363+0000","last_unstale":"2024-03-30T13:10:58.672207+0000","last_undegraded":"2024-03-30T13:10:58.672207+0000","last_fullsized":"2024-03-30T13:10:58.672207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:25.435052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032876+0000","last_change":"2024-03-30T13:10:56.676925+0000","last_active":"2024-03-30T13:10:59.032876+0000","last_peered":"2024-03-30T13:10:59.032876+0000","last_clean":"2024-03-30T13:10:59.032876+0000","last_became_active":"2024-03-30T13:10:56.676627+0000","last_became_peered":"2024-03-30T13:10:56.676627+0000","last_unstale":"2024-03-30T13:10:59.032876+0000","last_undegraded":"2024-03-30T13:10:59.032876+0000","last_fullsized":"2024-03-30T13:10:59.032876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672078+0000","last_change":"2024-03-30T13:10:57.686177+0000","last_active":"2024-03-30T13:10:58.672078+0000","last_peered":"2024-03-30T13:10:58.672078+0000","last_clean":"2024-03-30T13:10:58.672078+0000","last_became_active":"2024-03-30T13:10:57.685857+0000","last_became_peered":"2024-03-30T13:10:57.685857+0000","last_unstale":"2024-03-30T13:10:58.672078+0000","last_undegraded":"2024-03-30T13:10:58.672078+0000","last_fullsized":"2024-03-30T13:10:58.672078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:28.503626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683617+0000","last_change":"2024-03-30T13:10:56.681832+0000","last_active":"2024-03-30T13:10:56.683617+0000","last_peered":"2024-03-30T13:10:56.683617+0000","last_clean":"2024-03-30T13:10:56.683617+0000","last_became_active":"2024-03-30T13:10:56.681522+0000","last_became_peered":"2024-03-30T13:10:56.681522+0000","last_unstale":"2024-03-30T13:10:56.683617+0000","last_undegraded":"2024-03-30T13:10:56.683617+0000","last_fullsized":"2024-03-30T13:10:56.683617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:30:19.213692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672924+0000","last_change":"2024-03-30T13:10:56.672924+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672924+0000","last_undegraded":"2024-03-30T13:10:56.672924+0000","last_fullsized":"2024-03-30T13:10:56.672924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672976+0000","last_change":"2024-03-30T13:10:56.681831+0000","last_active":"2024-03-30T13:10:58.672976+0000","last_peered":"2024-03-30T13:10:58.672976+0000","last_clean":"2024-03-30T13:10:58.672976+0000","last_became_active":"2024-03-30T13:10:56.681581+0000","last_became_peered":"2024-03-30T13:10:56.681581+0000","last_unstale":"2024-03-30T13:10:58.672976+0000","last_undegraded":"2024-03-30T13:10:58.672976+0000","last_fullsized":"2024-03-30T13:10:58.672976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:54.818674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.673095+0000","last_change":"2024-03-30T13:10:57.689473+0000","last_active":"2024-03-30T13:10:58.673095+0000","last_peered":"2024-03-30T13:10:58.673095+0000","last_clean":"2024-03-30T13:10:58.673095+0000","last_became_active":"2024-03-30T13:10:57.689184+0000","last_became_peered":"2024-03-30T13:10:57.689184+0000","last_unstale":"2024-03-30T13:10:58.673095+0000","last_undegraded":"2024-03-30T13:10:58.673095+0000","last_fullsized":"2024-03-30T13:10:58.673095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678850+0000","last_change":"2024-03-30T13:10:56.671520+0000","last_active":"2024-03-30T13:10:58.678850+0000","last_peered":"2024-03-30T13:10:58.678850+0000","last_clean":"2024-03-30T13:10:58.678850+0000","last_became_active":"2024-03-30T13:10:56.671198+0000","last_became_peered":"2024-03-30T13:10:56.671198+0000","last_unstale":"2024-03-30T13:10:58.678850+0000","last_undegraded":"2024-03-30T13:10:58.678850+0000","last_fullsized":"2024-03-30T13:10:58.678850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:48:29.323071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.679016+0000","last_change":"2024-03-30T13:10:57.682760+0000","last_active":"2024-03-30T13:10:58.679016+0000","last_peered":"2024-03-30T13:10:58.679016+0000","last_clean":"2024-03-30T13:10:58.679016+0000","last_became_active":"2024-03-30T13:10:57.682495+0000","last_became_peered":"2024-03-30T13:10:57.682495+0000","last_unstale":"2024-03-30T13:10:58.679016+0000","last_undegraded":"2024-03-30T13:10:58.679016+0000","last_fullsized":"2024-03-30T13:10:58.679016+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:03:14.415400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676601+0000","last_change":"2024-03-30T13:10:56.679764+0000","last_active":"2024-03-30T13:10:58.676601+0000","last_peered":"2024-03-30T13:10:58.676601+0000","last_clean":"2024-03-30T13:10:58.676601+0000","last_became_active":"2024-03-30T13:10:56.679483+0000","last_became_peered":"2024-03-30T13:10:56.679483+0000","last_unstale":"2024-03-30T13:10:58.676601+0000","last_undegraded":"2024-03-30T13:10:58.676601+0000","last_fullsized":"2024-03-30T13:10:58.676601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033042+0000","last_change":"2024-03-30T13:10:57.693368+0000","last_active":"2024-03-30T13:10:59.033042+0000","last_peered":"2024-03-30T13:10:59.033042+0000","last_clean":"2024-03-30T13:10:59.033042+0000","last_became_active":"2024-03-30T13:10:57.693076+0000","last_became_peered":"2024-03-30T13:10:57.693076+0000","last_unstale":"2024-03-30T13:10:59.033042+0000","last_undegraded":"2024-03-30T13:10:59.033042+0000","last_fullsized":"2024-03-30T13:10:59.033042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678626+0000","last_change":"2024-03-30T13:10:56.677591+0000","last_active":"2024-03-30T13:10:58.678626+0000","last_peered":"2024-03-30T13:10:58.678626+0000","last_clean":"2024-03-30T13:10:58.678626+0000","last_became_active":"2024-03-30T13:10:56.677289+0000","last_became_peered":"2024-03-30T13:10:56.677289+0000","last_unstale":"2024-03-30T13:10:58.678626+0000","last_undegraded":"2024-03-30T13:10:58.678626+0000","last_fullsized":"2024-03-30T13:10:58.678626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:55:48.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745181+0000","last_change":"2024-03-30T13:10:57.686202+0000","last_active":"2024-03-30T13:10:59.745181+0000","last_peered":"2024-03-30T13:10:59.745181+0000","last_clean":"2024-03-30T13:10:59.745181+0000","last_became_active":"2024-03-30T13:10:57.685942+0000","last_became_peered":"2024-03-30T13:10:57.685942+0000","last_unstale":"2024-03-30T13:10:59.745181+0000","last_undegraded":"2024-03-30T13:10:59.745181+0000","last_fullsized":"2024-03-30T13:10:59.745181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678579+0000","last_change":"2024-03-30T13:10:56.677268+0000","last_active":"2024-03-30T13:10:58.678579+0000","last_peered":"2024-03-30T13:10:58.678579+0000","last_clean":"2024-03-30T13:10:58.678579+0000","last_became_active":"2024-03-30T13:10:56.676985+0000","last_became_peered":"2024-03-30T13:10:56.676985+0000","last_unstale":"2024-03-30T13:10:58.678579+0000","last_undegraded":"2024-03-30T13:10:58.678579+0000","last_fullsized":"2024-03-30T13:10:58.678579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:31:09.416890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672390+0000","last_change":"2024-03-30T13:10:56.672390+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672390+0000","last_undegraded":"2024-03-30T13:10:56.672390+0000","last_fullsized":"2024-03-30T13:10:56.672390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033098+0000","last_change":"2024-03-30T13:10:56.680608+0000","last_active":"2024-03-30T13:10:59.033098+0000","last_peered":"2024-03-30T13:10:59.033098+0000","last_clean":"2024-03-30T13:10:59.033098+0000","last_became_active":"2024-03-30T13:10:56.680346+0000","last_became_peered":"2024-03-30T13:10:56.680346+0000","last_unstale":"2024-03-30T13:10:59.033098+0000","last_undegraded":"2024-03-30T13:10:59.033098+0000","last_fullsized":"2024-03-30T13:10:59.033098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672201+0000","last_change":"2024-03-30T13:10:57.678124+0000","last_active":"2024-03-30T13:10:58.672201+0000","last_peered":"2024-03-30T13:10:58.672201+0000","last_clean":"2024-03-30T13:10:58.672201+0000","last_became_active":"2024-03-30T13:10:57.677826+0000","last_became_peered":"2024-03-30T13:10:57.677826+0000","last_unstale":"2024-03-30T13:10:58.672201+0000","last_undegraded":"2024-03-30T13:10:58.672201+0000","last_fullsized":"2024-03-30T13:10:58.672201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:37.076824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678461+0000","last_change":"2024-03-30T13:10:56.678610+0000","last_active":"2024-03-30T13:10:58.678461+0000","last_peered":"2024-03-30T13:10:58.678461+0000","last_clean":"2024-03-30T13:10:58.678461+0000","last_became_active":"2024-03-30T13:10:56.678357+0000","last_became_peered":"2024-03-30T13:10:56.678357+0000","last_unstale":"2024-03-30T13:10:58.678461+0000","last_undegraded":"2024-03-30T13:10:58.678461+0000","last_fullsized":"2024-03-30T13:10:58.678461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:40.674849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676600+0000","last_change":"2024-03-30T13:10:57.687196+0000","last_active":"2024-03-30T13:10:58.676600+0000","last_peered":"2024-03-30T13:10:58.676600+0000","last_clean":"2024-03-30T13:10:58.676600+0000","last_became_active":"2024-03-30T13:10:57.686683+0000","last_became_peered":"2024-03-30T13:10:57.686683+0000","last_unstale":"2024-03-30T13:10:58.676600+0000","last_undegraded":"2024-03-30T13:10:58.676600+0000","last_fullsized":"2024-03-30T13:10:58.676600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:44:15.354797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683925+0000","last_change":"2024-03-30T13:10:56.682787+0000","last_active":"2024-03-30T13:10:56.683925+0000","last_peered":"2024-03-30T13:10:56.683925+0000","last_clean":"2024-03-30T13:10:56.683925+0000","last_became_active":"2024-03-30T13:10:56.682536+0000","last_became_peered":"2024-03-30T13:10:56.682536+0000","last_unstale":"2024-03-30T13:10:56.683925+0000","last_undegraded":"2024-03-30T13:10:56.683925+0000","last_fullsized":"2024-03-30T13:10:56.683925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:30:59.057923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033243+0000","last_change":"2024-03-30T13:10:57.686653+0000","last_active":"2024-03-30T13:10:59.033243+0000","last_peered":"2024-03-30T13:10:59.033243+0000","last_clean":"2024-03-30T13:10:59.033243+0000","last_became_active":"2024-03-30T13:10:57.686380+0000","last_became_peered":"2024-03-30T13:10:57.686380+0000","last_unstale":"2024-03-30T13:10:59.033243+0000","last_undegraded":"2024-03-30T13:10:59.033243+0000","last_fullsized":"2024-03-30T13:10:59.033243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:13:33.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680418+0000","last_change":"2024-03-30T13:10:56.682042+0000","last_active":"2024-03-30T13:10:58.680418+0000","last_peered":"2024-03-30T13:10:58.680418+0000","last_clean":"2024-03-30T13:10:58.680418+0000","last_became_active":"2024-03-30T13:10:56.681748+0000","last_became_peered":"2024-03-30T13:10:56.681748+0000","last_unstale":"2024-03-30T13:10:58.680418+0000","last_undegraded":"2024-03-30T13:10:58.680418+0000","last_fullsized":"2024-03-30T13:10:58.680418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:55:55.240936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672322+0000","last_change":"2024-03-30T13:10:57.676679+0000","last_active":"2024-03-30T13:10:58.672322+0000","last_peered":"2024-03-30T13:10:58.672322+0000","last_clean":"2024-03-30T13:10:58.672322+0000","last_became_active":"2024-03-30T13:10:57.675136+0000","last_became_peered":"2024-03-30T13:10:57.675136+0000","last_unstale":"2024-03-30T13:10:58.672322+0000","last_undegraded":"2024-03-30T13:10:58.672322+0000","last_fullsized":"2024-03-30T13:10:58.672322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:01:31.125961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745366+0000","last_change":"2024-03-30T13:10:56.682275+0000","last_active":"2024-03-30T13:10:59.745366+0000","last_peered":"2024-03-30T13:10:59.745366+0000","last_clean":"2024-03-30T13:10:59.745366+0000","last_became_active":"2024-03-30T13:10:56.681990+0000","last_became_peered":"2024-03-30T13:10:56.681990+0000","last_unstale":"2024-03-30T13:10:59.745366+0000","last_undegraded":"2024-03-30T13:10:59.745366+0000","last_fullsized":"2024-03-30T13:10:59.745366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745290+0000","last_change":"2024-03-30T13:10:57.692363+0000","last_active":"2024-03-30T13:10:59.745290+0000","last_peered":"2024-03-30T13:10:59.745290+0000","last_clean":"2024-03-30T13:10:59.745290+0000","last_became_active":"2024-03-30T13:10:57.692063+0000","last_became_peered":"2024-03-30T13:10:57.692063+0000","last_unstale":"2024-03-30T13:10:59.745290+0000","last_undegraded":"2024-03-30T13:10:59.745290+0000","last_fullsized":"2024-03-30T13:10:59.745290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745367+0000","last_change":"2024-03-30T13:10:56.680716+0000","last_active":"2024-03-30T13:10:59.745367+0000","last_peered":"2024-03-30T13:10:59.745367+0000","last_clean":"2024-03-30T13:10:59.745367+0000","last_became_active":"2024-03-30T13:10:56.680446+0000","last_became_peered":"2024-03-30T13:10:56.680446+0000","last_unstale":"2024-03-30T13:10:59.745367+0000","last_undegraded":"2024-03-30T13:10:59.745367+0000","last_fullsized":"2024-03-30T13:10:59.745367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676703+0000","last_change":"2024-03-30T13:10:57.684334+0000","last_active":"2024-03-30T13:10:58.676703+0000","last_peered":"2024-03-30T13:10:58.676703+0000","last_clean":"2024-03-30T13:10:58.676703+0000","last_became_active":"2024-03-30T13:10:57.684037+0000","last_became_peered":"2024-03-30T13:10:57.684037+0000","last_unstale":"2024-03-30T13:10:58.676703+0000","last_undegraded":"2024-03-30T13:10:58.676703+0000","last_fullsized":"2024-03-30T13:10:58.676703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:07:21.631648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745812+0000","last_change":"2024-03-30T13:10:56.676202+0000","last_active":"2024-03-30T13:10:59.745812+0000","last_peered":"2024-03-30T13:10:59.745812+0000","last_clean":"2024-03-30T13:10:59.745812+0000","last_became_active":"2024-03-30T13:10:56.675907+0000","last_became_peered":"2024-03-30T13:10:56.675907+0000","last_unstale":"2024-03-30T13:10:59.745812+0000","last_undegraded":"2024-03-30T13:10:59.745812+0000","last_fullsized":"2024-03-30T13:10:59.745812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:58.975900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672618+0000","last_change":"2024-03-30T13:10:57.673049+0000","last_active":"2024-03-30T13:10:58.672618+0000","last_peered":"2024-03-30T13:10:58.672618+0000","last_clean":"2024-03-30T13:10:58.672618+0000","last_became_active":"2024-03-30T13:10:57.672738+0000","last_became_peered":"2024-03-30T13:10:57.672738+0000","last_unstale":"2024-03-30T13:10:58.672618+0000","last_undegraded":"2024-03-30T13:10:58.672618+0000","last_fullsized":"2024-03-30T13:10:58.672618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672426+0000","last_change":"2024-03-30T13:10:56.675234+0000","last_active":"2024-03-30T13:10:58.672426+0000","last_peered":"2024-03-30T13:10:58.672426+0000","last_clean":"2024-03-30T13:10:58.672426+0000","last_became_active":"2024-03-30T13:10:56.674965+0000","last_became_peered":"2024-03-30T13:10:56.674965+0000","last_unstale":"2024-03-30T13:10:58.672426+0000","last_undegraded":"2024-03-30T13:10:58.672426+0000","last_fullsized":"2024-03-30T13:10:58.672426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:35.628852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033158+0000","last_change":"2024-03-30T13:10:57.683624+0000","last_active":"2024-03-30T13:10:59.033158+0000","last_peered":"2024-03-30T13:10:59.033158+0000","last_clean":"2024-03-30T13:10:59.033158+0000","last_became_active":"2024-03-30T13:10:57.683346+0000","last_became_peered":"2024-03-30T13:10:57.683346+0000","last_unstale":"2024-03-30T13:10:59.033158+0000","last_undegraded":"2024-03-30T13:10:59.033158+0000","last_fullsized":"2024-03-30T13:10:59.033158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671837+0000","last_change":"2024-03-30T13:10:56.682711+0000","last_active":"2024-03-30T13:10:58.671837+0000","last_peered":"2024-03-30T13:10:58.671837+0000","last_clean":"2024-03-30T13:10:58.671837+0000","last_became_active":"2024-03-30T13:10:56.682415+0000","last_became_peered":"2024-03-30T13:10:56.682415+0000","last_unstale":"2024-03-30T13:10:58.671837+0000","last_undegraded":"2024-03-30T13:10:58.671837+0000","last_fullsized":"2024-03-30T13:10:58.671837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:10:34.096760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.674446+0000","last_change":"2024-03-30T13:10:56.674446+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.674446+0000","last_undegraded":"2024-03-30T13:10:56.674446+0000","last_fullsized":"2024-03-30T13:10:56.674446+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033155+0000","last_change":"2024-03-30T13:10:56.681620+0000","last_active":"2024-03-30T13:10:59.033155+0000","last_peered":"2024-03-30T13:10:59.033155+0000","last_clean":"2024-03-30T13:10:59.033155+0000","last_became_active":"2024-03-30T13:10:56.681341+0000","last_became_peered":"2024-03-30T13:10:56.681341+0000","last_unstale":"2024-03-30T13:10:59.033155+0000","last_undegraded":"2024-03-30T13:10:59.033155+0000","last_fullsized":"2024-03-30T13:10:59.033155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:46:52.577909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032961+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:10:59.032961+0000","last_peered":"2024-03-30T13:10:59.032961+0000","last_clean":"2024-03-30T13:10:59.032961+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:10:59.032961+0000","last_undegraded":"2024-03-30T13:10:59.032961+0000","last_fullsized":"2024-03-30T13:10:59.032961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":199796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":199818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":782924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":178688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:01.705 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:01.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.844+0000 7f7d64022640 1 Processor -- start 2024-03-30T13:11:01.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.844+0000 7f7d64022640 1 -- start start 2024-03-30T13:11:01.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d64022640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7d5c15e950 0x7f7d5c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d64022640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d5c0d48c0 0x7f7d5c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d64022640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0d2f00 0x7f7d5c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d64022640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7d5c0d3cb0 con 0x7f7d5c15e950 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d64022640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7d5c0d4d00 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d64022640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d36f0 con 0x7f7d5c0d2f00 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d61dbe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7d5c15e950 0x7f7d5c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d61dbe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7d5c15e950 0x7f7d5c0d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39520/0 (socket says 172.21.15.66:39520) 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d61dbe640 1 -- 172.21.15.66:0/3717301578 learned_addr learned my addr 172.21.15.66:0/3717301578 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d615bd640 1 --2- 172.21.15.66:0/3717301578 >> v2:172.21.15.117:3300/0 conn(0x7f7d5c0d48c0 0x7f7d5c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.845+0000 7f7d625bf640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0d2f00 0x7f7d5c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d61dbe640 1 -- 172.21.15.66:0/3717301578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0d2f00 msgr2=0x7f7d5c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d61dbe640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0d2f00 0x7f7d5c0cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d61dbe640 1 -- 172.21.15.66:0/3717301578 >> v2:172.21.15.117:3300/0 conn(0x7f7d5c0d48c0 msgr2=0x7f7d5c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d61dbe640 1 --2- 172.21.15.66:0/3717301578 >> v2:172.21.15.117:3300/0 conn(0x7f7d5c0d48c0 0x7f7d5c0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d61dbe640 1 -- 172.21.15.66:0/3717301578 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c0d39d0 con 0x7f7d5c15e950 2024-03-30T13:11:01.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d615bd640 1 --2- 172.21.15.66:0/3717301578 >> v2:172.21.15.117:3300/0 conn(0x7f7d5c0d48c0 0x7f7d5c0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:01.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d625bf640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0d2f00 0x7f7d5c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:01.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.846+0000 7f7d61dbe640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c15e950 0x7f7d5c0d4380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c073790 tx=0x7f7d5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de9008be0a439fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d60dbc640 1 -- 172.21.15.66:0/3717301578 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58034070 con 0x7f7d5c15e950 2024-03-30T13:11:01.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d60dbc640 1 -- 172.21.15.66:0/3717301578 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d58030860 con 0x7f7d5c15e950 2024-03-30T13:11:01.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d60dbc640 1 -- 172.21.15.66:0/3717301578 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58030c30 con 0x7f7d5c15e950 2024-03-30T13:11:01.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 -- 172.21.15.66:0/3717301578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c15e950 msgr2=0x7f7d5c0d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c15e950 0x7f7d5c0d4380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c073790 tx=0x7f7d5802b9a0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 -- 172.21.15.66:0/3717301578 shutdown_connections 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0d2f00 0x7f7d5c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 --2- 172.21.15.66:0/3717301578 >> v2:172.21.15.117:3300/0 conn(0x7f7d5c0d48c0 0x7f7d5c0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 --2- 172.21.15.66:0/3717301578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c15e950 0x7f7d5c0d4380 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 -- 172.21.15.66:0/3717301578 >> 172.21.15.66:0/3717301578 conn(0x7f7d5c0676e0 msgr2=0x7f7d5c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 -- 172.21.15.66:0/3717301578 shutdown_connections 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.847+0000 7f7d64022640 1 -- 172.21.15.66:0/3717301578 wait complete. 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.848+0000 7f7d64022640 1 Processor -- start 2024-03-30T13:11:01.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.848+0000 7f7d64022640 1 -- start start 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d64022640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c0d2f00 0x7f7d5c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d64022640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 0x7f7d5c1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d64022640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c15e950 0x7f7d5c1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d64022640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d3390 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d64022640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0b7da0 con 0x7f7d5c0d2f00 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d64022640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c15ed30 con 0x7f7d5c15e950 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d615bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 0x7f7d5c1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d615bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 0x7f7d5c1f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39532/0 (socket says 172.21.15.66:39532) 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d615bd640 1 -- 172.21.15.66:0/1795333201 learned_addr learned my addr 172.21.15.66:0/1795333201 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d61dbe640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c0d2f00 0x7f7d5c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.849+0000 7f7d625bf640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c15e950 0x7f7d5c1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d615bd640 1 -- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c15e950 msgr2=0x7f7d5c1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d615bd640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c15e950 0x7f7d5c1f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d615bd640 1 -- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c0d2f00 msgr2=0x7f7d5c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d615bd640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c0d2f00 0x7f7d5c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d615bd640 1 -- 172.21.15.66:0/1795333201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c0ca2d0 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d615bd640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 0x7f7d5c1f41c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e730 tx=0x7f7d5000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000f040 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d50004230 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000ee40 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c0ca5b0 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.850+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c0c9ea0 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.851+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c075a10 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.852+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7d5000c6d0 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.852+0000 7f7d4e7fc640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d4006c530 0x7f7d4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.852+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f7d5008b450 con 0x7f7d5c0d48c0 2024-03-30T13:11:01.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.852+0000 7f7d61dbe640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d4006c530 0x7f7d4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.853+0000 7f7d61dbe640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d4006c530 0x7f7d4006e9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7d58036000 tx=0x7f7d5803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:01.857+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7d50017020 con 0x7f7d5c0d48c0 2024-03-30T13:11:02.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.040+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d5c15dcb0 con 0x7f7d4006c530 2024-03-30T13:11:02.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.047+0000 7f7d4e7fc640 1 -- 172.21.15.66:0/1795333201 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251050 (secure 0 0 0) 0x7f7d5c15dcb0 con 0x7f7d4006c530 2024-03-30T13:11:02.053 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:02.054 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d4006c530 msgr2=0x7f7d4006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d4006c530 0x7f7d4006e9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7d58036000 tx=0x7f7d5803b040 comp rx=0 tx=0).stop 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 msgr2=0x7f7d5c1f41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 0x7f7d5c1f41c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e730 tx=0x7f7d5000ebf0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 shutdown_connections 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d4006c530 0x7f7d4006e9f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c15e950 0x7f7d5c1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c0d48c0 0x7f7d5c1f41c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 --2- 172.21.15.66:0/1795333201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c0d2f00 0x7f7d5c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 >> 172.21.15.66:0/1795333201 conn(0x7f7d5c0676e0 msgr2=0x7f7d5c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 shutdown_connections 2024-03-30T13:11:02.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.050+0000 7f7d64022640 1 -- 172.21.15.66:0/1795333201 wait complete. 2024-03-30T13:11:02.064 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-03-30T13:11:00.810218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220852,"kb_used_data":4684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753868,"statfs":{"total":773094113280,"available":772867960832,"internally_reserved":0,"allocated":4796416,"data_stored":2731862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.155701"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672404+0000","last_change":"2024-03-30T13:10:57.684501+0000","last_active":"2024-03-30T13:10:58.672404+0000","last_peered":"2024-03-30T13:10:58.672404+0000","last_clean":"2024-03-30T13:10:58.672404+0000","last_became_active":"2024-03-30T13:10:57.684187+0000","last_became_peered":"2024-03-30T13:10:57.684187+0000","last_unstale":"2024-03-30T13:10:58.672404+0000","last_undegraded":"2024-03-30T13:10:58.672404+0000","last_fullsized":"2024-03-30T13:10:58.672404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:07.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672159+0000","last_change":"2024-03-30T13:10:57.681207+0000","last_active":"2024-03-30T13:10:58.672159+0000","last_peered":"2024-03-30T13:10:58.672159+0000","last_clean":"2024-03-30T13:10:58.672159+0000","last_became_active":"2024-03-30T13:10:57.680890+0000","last_became_peered":"2024-03-30T13:10:57.680890+0000","last_unstale":"2024-03-30T13:10:58.672159+0000","last_undegraded":"2024-03-30T13:10:58.672159+0000","last_fullsized":"2024-03-30T13:10:58.672159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:30:42.096995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745282+0000","last_change":"2024-03-30T13:10:57.695761+0000","last_active":"2024-03-30T13:10:59.745282+0000","last_peered":"2024-03-30T13:10:59.745282+0000","last_clean":"2024-03-30T13:10:59.745282+0000","last_became_active":"2024-03-30T13:10:57.695488+0000","last_became_peered":"2024-03-30T13:10:57.695488+0000","last_unstale":"2024-03-30T13:10:59.745282+0000","last_undegraded":"2024-03-30T13:10:59.745282+0000","last_fullsized":"2024-03-30T13:10:59.745282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678726+0000","last_change":"2024-03-30T13:10:57.682241+0000","last_active":"2024-03-30T13:10:58.678726+0000","last_peered":"2024-03-30T13:10:58.678726+0000","last_clean":"2024-03-30T13:10:58.678726+0000","last_became_active":"2024-03-30T13:10:57.681383+0000","last_became_peered":"2024-03-30T13:10:57.681383+0000","last_unstale":"2024-03-30T13:10:58.678726+0000","last_undegraded":"2024-03-30T13:10:58.678726+0000","last_fullsized":"2024-03-30T13:10:58.678726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:43:40.092786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678584+0000","last_change":"2024-03-30T13:10:57.680923+0000","last_active":"2024-03-30T13:10:58.678584+0000","last_peered":"2024-03-30T13:10:58.678584+0000","last_clean":"2024-03-30T13:10:58.678584+0000","last_became_active":"2024-03-30T13:10:57.680625+0000","last_became_peered":"2024-03-30T13:10:57.680625+0000","last_unstale":"2024-03-30T13:10:58.678584+0000","last_undegraded":"2024-03-30T13:10:58.678584+0000","last_fullsized":"2024-03-30T13:10:58.678584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:27:38.309021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676495+0000","last_change":"2024-03-30T13:10:57.687677+0000","last_active":"2024-03-30T13:10:58.676495+0000","last_peered":"2024-03-30T13:10:58.676495+0000","last_clean":"2024-03-30T13:10:58.676495+0000","last_became_active":"2024-03-30T13:10:57.687348+0000","last_became_peered":"2024-03-30T13:10:57.687348+0000","last_unstale":"2024-03-30T13:10:58.676495+0000","last_undegraded":"2024-03-30T13:10:58.676495+0000","last_fullsized":"2024-03-30T13:10:58.676495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:33:20.876411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.677018+0000","last_change":"2024-03-30T13:10:56.677018+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.677018+0000","last_undegraded":"2024-03-30T13:10:56.677018+0000","last_fullsized":"2024-03-30T13:10:56.677018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671840+0000","last_change":"2024-03-30T13:10:57.679497+0000","last_active":"2024-03-30T13:10:58.671840+0000","last_peered":"2024-03-30T13:10:58.671840+0000","last_clean":"2024-03-30T13:10:58.671840+0000","last_became_active":"2024-03-30T13:10:57.679201+0000","last_became_peered":"2024-03-30T13:10:57.679201+0000","last_unstale":"2024-03-30T13:10:58.671840+0000","last_undegraded":"2024-03-30T13:10:58.671840+0000","last_fullsized":"2024-03-30T13:10:58.671840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:04:54.328931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745602+0000","last_change":"2024-03-30T13:10:57.690985+0000","last_active":"2024-03-30T13:10:59.745602+0000","last_peered":"2024-03-30T13:10:59.745602+0000","last_clean":"2024-03-30T13:10:59.745602+0000","last_became_active":"2024-03-30T13:10:57.690657+0000","last_became_peered":"2024-03-30T13:10:57.690657+0000","last_unstale":"2024-03-30T13:10:59.745602+0000","last_undegraded":"2024-03-30T13:10:59.745602+0000","last_fullsized":"2024-03-30T13:10:59.745602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676334+0000","last_change":"2024-03-30T13:10:56.676334+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676334+0000","last_undegraded":"2024-03-30T13:10:56.676334+0000","last_fullsized":"2024-03-30T13:10:56.676334+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.673354+0000","last_change":"2024-03-30T13:10:56.673354+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.673354+0000","last_undegraded":"2024-03-30T13:10:56.673354+0000","last_fullsized":"2024-03-30T13:10:56.673354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745292+0000","last_change":"2024-03-30T13:10:57.691635+0000","last_active":"2024-03-30T13:10:59.745292+0000","last_peered":"2024-03-30T13:10:59.745292+0000","last_clean":"2024-03-30T13:10:59.745292+0000","last_became_active":"2024-03-30T13:10:57.691340+0000","last_became_peered":"2024-03-30T13:10:57.691340+0000","last_unstale":"2024-03-30T13:10:59.745292+0000","last_undegraded":"2024-03-30T13:10:59.745292+0000","last_fullsized":"2024-03-30T13:10:59.745292+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676222+0000","last_change":"2024-03-30T13:10:56.676222+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676222+0000","last_undegraded":"2024-03-30T13:10:56.676222+0000","last_fullsized":"2024-03-30T13:10:56.676222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672408+0000","last_change":"2024-03-30T13:10:57.686031+0000","last_active":"2024-03-30T13:10:58.672408+0000","last_peered":"2024-03-30T13:10:58.672408+0000","last_clean":"2024-03-30T13:10:58.672408+0000","last_became_active":"2024-03-30T13:10:57.685814+0000","last_became_peered":"2024-03-30T13:10:57.685814+0000","last_unstale":"2024-03-30T13:10:58.672408+0000","last_undegraded":"2024-03-30T13:10:58.672408+0000","last_fullsized":"2024-03-30T13:10:58.672408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676235+0000","last_change":"2024-03-30T13:10:56.676235+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676235+0000","last_undegraded":"2024-03-30T13:10:56.676235+0000","last_fullsized":"2024-03-30T13:10:56.676235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032816+0000","last_change":"2024-03-30T13:10:57.681811+0000","last_active":"2024-03-30T13:10:59.032816+0000","last_peered":"2024-03-30T13:10:59.032816+0000","last_clean":"2024-03-30T13:10:59.032816+0000","last_became_active":"2024-03-30T13:10:57.681471+0000","last_became_peered":"2024-03-30T13:10:57.681471+0000","last_unstale":"2024-03-30T13:10:59.032816+0000","last_undegraded":"2024-03-30T13:10:59.032816+0000","last_fullsized":"2024-03-30T13:10:59.032816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033359+0000","last_change":"2024-03-30T13:10:57.687549+0000","last_active":"2024-03-30T13:10:59.033359+0000","last_peered":"2024-03-30T13:10:59.033359+0000","last_clean":"2024-03-30T13:10:59.033359+0000","last_became_active":"2024-03-30T13:10:57.687268+0000","last_became_peered":"2024-03-30T13:10:57.687268+0000","last_unstale":"2024-03-30T13:10:59.033359+0000","last_undegraded":"2024-03-30T13:10:59.033359+0000","last_fullsized":"2024-03-30T13:10:59.033359+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:35.714074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746089+0000","last_change":"2024-03-30T13:10:57.683841+0000","last_active":"2024-03-30T13:10:59.746089+0000","last_peered":"2024-03-30T13:10:59.746089+0000","last_clean":"2024-03-30T13:10:59.746089+0000","last_became_active":"2024-03-30T13:10:57.683568+0000","last_became_peered":"2024-03-30T13:10:57.683568+0000","last_unstale":"2024-03-30T13:10:59.746089+0000","last_undegraded":"2024-03-30T13:10:59.746089+0000","last_fullsized":"2024-03-30T13:10:59.746089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:51:46.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678919+0000","last_change":"2024-03-30T13:10:57.688521+0000","last_active":"2024-03-30T13:10:58.678919+0000","last_peered":"2024-03-30T13:10:58.678919+0000","last_clean":"2024-03-30T13:10:58.678919+0000","last_became_active":"2024-03-30T13:10:57.688188+0000","last_became_peered":"2024-03-30T13:10:57.688188+0000","last_unstale":"2024-03-30T13:10:58.678919+0000","last_undegraded":"2024-03-30T13:10:58.678919+0000","last_fullsized":"2024-03-30T13:10:58.678919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:47.722209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745911+0000","last_change":"2024-03-30T13:10:57.682779+0000","last_active":"2024-03-30T13:10:59.745911+0000","last_peered":"2024-03-30T13:10:59.745911+0000","last_clean":"2024-03-30T13:10:59.745911+0000","last_became_active":"2024-03-30T13:10:57.682444+0000","last_became_peered":"2024-03-30T13:10:57.682444+0000","last_unstale":"2024-03-30T13:10:59.745911+0000","last_undegraded":"2024-03-30T13:10:59.745911+0000","last_fullsized":"2024-03-30T13:10:59.745911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:02:29.613267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672740+0000","last_change":"2024-03-30T13:10:56.672740+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672740+0000","last_undegraded":"2024-03-30T13:10:56.672740+0000","last_fullsized":"2024-03-30T13:10:56.672740+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672797+0000","last_change":"2024-03-30T13:10:57.692104+0000","last_active":"2024-03-30T13:10:58.672797+0000","last_peered":"2024-03-30T13:10:58.672797+0000","last_clean":"2024-03-30T13:10:58.672797+0000","last_became_active":"2024-03-30T13:10:57.691816+0000","last_became_peered":"2024-03-30T13:10:57.691816+0000","last_unstale":"2024-03-30T13:10:58.672797+0000","last_undegraded":"2024-03-30T13:10:58.672797+0000","last_fullsized":"2024-03-30T13:10:58.672797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032948+0000","last_change":"2024-03-30T13:10:57.691507+0000","last_active":"2024-03-30T13:10:59.032948+0000","last_peered":"2024-03-30T13:10:59.032948+0000","last_clean":"2024-03-30T13:10:59.032948+0000","last_became_active":"2024-03-30T13:10:57.691248+0000","last_became_peered":"2024-03-30T13:10:57.691248+0000","last_unstale":"2024-03-30T13:10:59.032948+0000","last_undegraded":"2024-03-30T13:10:59.032948+0000","last_fullsized":"2024-03-30T13:10:59.032948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678498+0000","last_change":"2024-03-30T13:10:57.685316+0000","last_active":"2024-03-30T13:10:58.678498+0000","last_peered":"2024-03-30T13:10:58.678498+0000","last_clean":"2024-03-30T13:10:58.678498+0000","last_became_active":"2024-03-30T13:10:57.685053+0000","last_became_peered":"2024-03-30T13:10:57.685053+0000","last_unstale":"2024-03-30T13:10:58.678498+0000","last_undegraded":"2024-03-30T13:10:58.678498+0000","last_fullsized":"2024-03-30T13:10:58.678498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:43:19.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746090+0000","last_change":"2024-03-30T13:10:57.684225+0000","last_active":"2024-03-30T13:10:59.746090+0000","last_peered":"2024-03-30T13:10:59.746090+0000","last_clean":"2024-03-30T13:10:59.746090+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:59.746090+0000","last_undegraded":"2024-03-30T13:10:59.746090+0000","last_fullsized":"2024-03-30T13:10:59.746090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:44:06.219419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676846+0000","last_change":"2024-03-30T13:10:57.687484+0000","last_active":"2024-03-30T13:10:58.676846+0000","last_peered":"2024-03-30T13:10:58.676846+0000","last_clean":"2024-03-30T13:10:58.676846+0000","last_became_active":"2024-03-30T13:10:57.687191+0000","last_became_peered":"2024-03-30T13:10:57.687191+0000","last_unstale":"2024-03-30T13:10:58.676846+0000","last_undegraded":"2024-03-30T13:10:58.676846+0000","last_fullsized":"2024-03-30T13:10:58.676846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:49:47.246424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746018+0000","last_change":"2024-03-30T13:10:57.687817+0000","last_active":"2024-03-30T13:10:59.746018+0000","last_peered":"2024-03-30T13:10:59.746018+0000","last_clean":"2024-03-30T13:10:59.746018+0000","last_became_active":"2024-03-30T13:10:57.687038+0000","last_became_peered":"2024-03-30T13:10:57.687038+0000","last_unstale":"2024-03-30T13:10:59.746018+0000","last_undegraded":"2024-03-30T13:10:59.746018+0000","last_fullsized":"2024-03-30T13:10:59.746018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:03:06.653896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.675608+0000","last_change":"2024-03-30T13:10:56.675608+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.675608+0000","last_undegraded":"2024-03-30T13:10:56.675608+0000","last_fullsized":"2024-03-30T13:10:56.675608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745905+0000","last_change":"2024-03-30T13:10:57.690320+0000","last_active":"2024-03-30T13:10:59.745905+0000","last_peered":"2024-03-30T13:10:59.745905+0000","last_clean":"2024-03-30T13:10:59.745905+0000","last_became_active":"2024-03-30T13:10:57.689740+0000","last_became_peered":"2024-03-30T13:10:57.689740+0000","last_unstale":"2024-03-30T13:10:59.745905+0000","last_undegraded":"2024-03-30T13:10:59.745905+0000","last_fullsized":"2024-03-30T13:10:59.745905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:55.823728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745918+0000","last_change":"2024-03-30T13:10:57.683564+0000","last_active":"2024-03-30T13:10:59.745918+0000","last_peered":"2024-03-30T13:10:59.745918+0000","last_clean":"2024-03-30T13:10:59.745918+0000","last_became_active":"2024-03-30T13:10:57.683225+0000","last_became_peered":"2024-03-30T13:10:57.683225+0000","last_unstale":"2024-03-30T13:10:59.745918+0000","last_undegraded":"2024-03-30T13:10:59.745918+0000","last_fullsized":"2024-03-30T13:10:59.745918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:36.633906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032986+0000","last_change":"2024-03-30T13:10:57.692664+0000","last_active":"2024-03-30T13:10:59.032986+0000","last_peered":"2024-03-30T13:10:59.032986+0000","last_clean":"2024-03-30T13:10:59.032986+0000","last_became_active":"2024-03-30T13:10:57.692376+0000","last_became_peered":"2024-03-30T13:10:57.692376+0000","last_unstale":"2024-03-30T13:10:59.032986+0000","last_undegraded":"2024-03-30T13:10:59.032986+0000","last_fullsized":"2024-03-30T13:10:59.032986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676356+0000","last_change":"2024-03-30T13:10:57.688812+0000","last_active":"2024-03-30T13:10:58.676356+0000","last_peered":"2024-03-30T13:10:58.676356+0000","last_clean":"2024-03-30T13:10:58.676356+0000","last_became_active":"2024-03-30T13:10:57.688187+0000","last_became_peered":"2024-03-30T13:10:57.688187+0000","last_unstale":"2024-03-30T13:10:58.676356+0000","last_undegraded":"2024-03-30T13:10:58.676356+0000","last_fullsized":"2024-03-30T13:10:58.676356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:13:34.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033219+0000","last_change":"2024-03-30T13:10:57.682984+0000","last_active":"2024-03-30T13:10:59.033219+0000","last_peered":"2024-03-30T13:10:59.033219+0000","last_clean":"2024-03-30T13:10:59.033219+0000","last_became_active":"2024-03-30T13:10:57.682685+0000","last_became_peered":"2024-03-30T13:10:57.682685+0000","last_unstale":"2024-03-30T13:10:59.033219+0000","last_undegraded":"2024-03-30T13:10:59.033219+0000","last_fullsized":"2024-03-30T13:10:59.033219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:28:51.189027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746047+0000","last_change":"2024-03-30T13:10:56.673822+0000","last_active":"2024-03-30T13:10:59.746047+0000","last_peered":"2024-03-30T13:10:59.746047+0000","last_clean":"2024-03-30T13:10:59.746047+0000","last_became_active":"2024-03-30T13:10:56.673533+0000","last_became_peered":"2024-03-30T13:10:56.673533+0000","last_unstale":"2024-03-30T13:10:59.746047+0000","last_undegraded":"2024-03-30T13:10:59.746047+0000","last_fullsized":"2024-03-30T13:10:59.746047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:51:29.821157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676839+0000","last_change":"2024-03-30T13:10:57.688749+0000","last_active":"2024-03-30T13:10:58.676839+0000","last_peered":"2024-03-30T13:10:58.676839+0000","last_clean":"2024-03-30T13:10:58.676839+0000","last_became_active":"2024-03-30T13:10:57.687457+0000","last_became_peered":"2024-03-30T13:10:57.687457+0000","last_unstale":"2024-03-30T13:10:58.676839+0000","last_undegraded":"2024-03-30T13:10:58.676839+0000","last_fullsized":"2024-03-30T13:10:58.676839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:42.131116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745498+0000","last_change":"2024-03-30T13:10:56.673044+0000","last_active":"2024-03-30T13:10:59.745498+0000","last_peered":"2024-03-30T13:10:59.745498+0000","last_clean":"2024-03-30T13:10:59.745498+0000","last_became_active":"2024-03-30T13:10:56.672750+0000","last_became_peered":"2024-03-30T13:10:56.672750+0000","last_unstale":"2024-03-30T13:10:59.745498+0000","last_undegraded":"2024-03-30T13:10:59.745498+0000","last_fullsized":"2024-03-30T13:10:59.745498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672861+0000","last_change":"2024-03-30T13:10:57.685414+0000","last_active":"2024-03-30T13:10:58.672861+0000","last_peered":"2024-03-30T13:10:58.672861+0000","last_clean":"2024-03-30T13:10:58.672861+0000","last_became_active":"2024-03-30T13:10:57.685131+0000","last_became_peered":"2024-03-30T13:10:57.685131+0000","last_unstale":"2024-03-30T13:10:58.672861+0000","last_undegraded":"2024-03-30T13:10:58.672861+0000","last_fullsized":"2024-03-30T13:10:58.672861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676898+0000","last_change":"2024-03-30T13:10:56.679798+0000","last_active":"2024-03-30T13:10:58.676898+0000","last_peered":"2024-03-30T13:10:58.676898+0000","last_clean":"2024-03-30T13:10:58.676898+0000","last_became_active":"2024-03-30T13:10:56.679500+0000","last_became_peered":"2024-03-30T13:10:56.679500+0000","last_unstale":"2024-03-30T13:10:58.676898+0000","last_undegraded":"2024-03-30T13:10:58.676898+0000","last_fullsized":"2024-03-30T13:10:58.676898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672316+0000","last_change":"2024-03-30T13:10:57.688829+0000","last_active":"2024-03-30T13:10:58.672316+0000","last_peered":"2024-03-30T13:10:58.672316+0000","last_clean":"2024-03-30T13:10:58.672316+0000","last_became_active":"2024-03-30T13:10:57.686054+0000","last_became_peered":"2024-03-30T13:10:57.686054+0000","last_unstale":"2024-03-30T13:10:58.672316+0000","last_undegraded":"2024-03-30T13:10:58.672316+0000","last_fullsized":"2024-03-30T13:10:58.672316+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:03:47.898459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672809+0000","last_change":"2024-03-30T13:10:56.674537+0000","last_active":"2024-03-30T13:10:58.672809+0000","last_peered":"2024-03-30T13:10:58.672809+0000","last_clean":"2024-03-30T13:10:58.672809+0000","last_became_active":"2024-03-30T13:10:56.674248+0000","last_became_peered":"2024-03-30T13:10:56.674248+0000","last_unstale":"2024-03-30T13:10:58.672809+0000","last_undegraded":"2024-03-30T13:10:58.672809+0000","last_fullsized":"2024-03-30T13:10:58.672809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745165+0000","last_change":"2024-03-30T13:10:57.693802+0000","last_active":"2024-03-30T13:10:59.745165+0000","last_peered":"2024-03-30T13:10:59.745165+0000","last_clean":"2024-03-30T13:10:59.745165+0000","last_became_active":"2024-03-30T13:10:57.693436+0000","last_became_peered":"2024-03-30T13:10:57.693436+0000","last_unstale":"2024-03-30T13:10:59.745165+0000","last_undegraded":"2024-03-30T13:10:59.745165+0000","last_fullsized":"2024-03-30T13:10:59.745165+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683595+0000","last_change":"2024-03-30T13:10:56.681576+0000","last_active":"2024-03-30T13:10:56.683595+0000","last_peered":"2024-03-30T13:10:56.683595+0000","last_clean":"2024-03-30T13:10:56.683595+0000","last_became_active":"2024-03-30T13:10:56.681081+0000","last_became_peered":"2024-03-30T13:10:56.681081+0000","last_unstale":"2024-03-30T13:10:56.683595+0000","last_undegraded":"2024-03-30T13:10:56.683595+0000","last_fullsized":"2024-03-30T13:10:56.683595+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:51:58.065326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.675811+0000","last_change":"2024-03-30T13:10:56.675811+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.675811+0000","last_undegraded":"2024-03-30T13:10:56.675811+0000","last_fullsized":"2024-03-30T13:10:56.675811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032969+0000","last_change":"2024-03-30T13:10:56.677670+0000","last_active":"2024-03-30T13:10:59.032969+0000","last_peered":"2024-03-30T13:10:59.032969+0000","last_clean":"2024-03-30T13:10:59.032969+0000","last_became_active":"2024-03-30T13:10:56.677367+0000","last_became_peered":"2024-03-30T13:10:56.677367+0000","last_unstale":"2024-03-30T13:10:59.032969+0000","last_undegraded":"2024-03-30T13:10:59.032969+0000","last_fullsized":"2024-03-30T13:10:59.032969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:14:10.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672971+0000","last_change":"2024-03-30T13:10:57.681840+0000","last_active":"2024-03-30T13:10:58.672971+0000","last_peered":"2024-03-30T13:10:58.672971+0000","last_clean":"2024-03-30T13:10:58.672971+0000","last_became_active":"2024-03-30T13:10:57.681548+0000","last_became_peered":"2024-03-30T13:10:57.681548+0000","last_unstale":"2024-03-30T13:10:58.672971+0000","last_undegraded":"2024-03-30T13:10:58.672971+0000","last_fullsized":"2024-03-30T13:10:58.672971+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676449+0000","last_change":"2024-03-30T13:10:56.682810+0000","last_active":"2024-03-30T13:10:58.676449+0000","last_peered":"2024-03-30T13:10:58.676449+0000","last_clean":"2024-03-30T13:10:58.676449+0000","last_became_active":"2024-03-30T13:10:56.682437+0000","last_became_peered":"2024-03-30T13:10:56.682437+0000","last_unstale":"2024-03-30T13:10:58.676449+0000","last_undegraded":"2024-03-30T13:10:58.676449+0000","last_fullsized":"2024-03-30T13:10:58.676449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672038+0000","last_change":"2024-03-30T13:10:57.678420+0000","last_active":"2024-03-30T13:10:58.672038+0000","last_peered":"2024-03-30T13:10:58.672038+0000","last_clean":"2024-03-30T13:10:58.672038+0000","last_became_active":"2024-03-30T13:10:57.678115+0000","last_became_peered":"2024-03-30T13:10:57.678115+0000","last_unstale":"2024-03-30T13:10:58.672038+0000","last_undegraded":"2024-03-30T13:10:58.672038+0000","last_fullsized":"2024-03-30T13:10:58.672038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:42:22.714841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672311+0000","last_change":"2024-03-30T13:10:56.678877+0000","last_active":"2024-03-30T13:10:58.672311+0000","last_peered":"2024-03-30T13:10:58.672311+0000","last_clean":"2024-03-30T13:10:58.672311+0000","last_became_active":"2024-03-30T13:10:56.678573+0000","last_became_peered":"2024-03-30T13:10:56.678573+0000","last_unstale":"2024-03-30T13:10:58.672311+0000","last_undegraded":"2024-03-30T13:10:58.672311+0000","last_fullsized":"2024-03-30T13:10:58.672311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678784+0000","last_change":"2024-03-30T13:10:57.685526+0000","last_active":"2024-03-30T13:10:58.678784+0000","last_peered":"2024-03-30T13:10:58.678784+0000","last_clean":"2024-03-30T13:10:58.678784+0000","last_became_active":"2024-03-30T13:10:57.681079+0000","last_became_peered":"2024-03-30T13:10:57.681079+0000","last_unstale":"2024-03-30T13:10:58.678784+0000","last_undegraded":"2024-03-30T13:10:58.678784+0000","last_fullsized":"2024-03-30T13:10:58.678784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:36:15.323960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672394+0000","last_change":"2024-03-30T13:10:56.680735+0000","last_active":"2024-03-30T13:10:58.672394+0000","last_peered":"2024-03-30T13:10:58.672394+0000","last_clean":"2024-03-30T13:10:58.672394+0000","last_became_active":"2024-03-30T13:10:56.680459+0000","last_became_peered":"2024-03-30T13:10:56.680459+0000","last_unstale":"2024-03-30T13:10:58.672394+0000","last_undegraded":"2024-03-30T13:10:58.672394+0000","last_fullsized":"2024-03-30T13:10:58.672394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:26.601466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033176+0000","last_change":"2024-03-30T13:10:57.687091+0000","last_active":"2024-03-30T13:10:59.033176+0000","last_peered":"2024-03-30T13:10:59.033176+0000","last_clean":"2024-03-30T13:10:59.033176+0000","last_became_active":"2024-03-30T13:10:57.686821+0000","last_became_peered":"2024-03-30T13:10:57.686821+0000","last_unstale":"2024-03-30T13:10:59.033176+0000","last_undegraded":"2024-03-30T13:10:59.033176+0000","last_fullsized":"2024-03-30T13:10:59.033176+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:16:42.538868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683691+0000","last_change":"2024-03-30T13:10:56.682722+0000","last_active":"2024-03-30T13:10:56.683691+0000","last_peered":"2024-03-30T13:10:56.683691+0000","last_clean":"2024-03-30T13:10:56.683691+0000","last_became_active":"2024-03-30T13:10:56.682272+0000","last_became_peered":"2024-03-30T13:10:56.682272+0000","last_unstale":"2024-03-30T13:10:56.683691+0000","last_undegraded":"2024-03-30T13:10:56.683691+0000","last_fullsized":"2024-03-30T13:10:56.683691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:51:40.244956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745459+0000","last_change":"2024-03-30T13:10:57.691155+0000","last_active":"2024-03-30T13:10:59.745459+0000","last_peered":"2024-03-30T13:10:59.745459+0000","last_clean":"2024-03-30T13:10:59.745459+0000","last_became_active":"2024-03-30T13:10:57.690702+0000","last_became_peered":"2024-03-30T13:10:57.690702+0000","last_unstale":"2024-03-30T13:10:59.745459+0000","last_undegraded":"2024-03-30T13:10:59.745459+0000","last_fullsized":"2024-03-30T13:10:59.745459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683944+0000","last_change":"2024-03-30T13:10:56.683050+0000","last_active":"2024-03-30T13:10:56.683944+0000","last_peered":"2024-03-30T13:10:56.683944+0000","last_clean":"2024-03-30T13:10:56.683944+0000","last_became_active":"2024-03-30T13:10:56.682514+0000","last_became_peered":"2024-03-30T13:10:56.682514+0000","last_unstale":"2024-03-30T13:10:56.683944+0000","last_undegraded":"2024-03-30T13:10:56.683944+0000","last_fullsized":"2024-03-30T13:10:56.683944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:40:52.009831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672597+0000","last_change":"2024-03-30T13:10:57.692928+0000","last_active":"2024-03-30T13:10:58.672597+0000","last_peered":"2024-03-30T13:10:58.672597+0000","last_clean":"2024-03-30T13:10:58.672597+0000","last_became_active":"2024-03-30T13:10:57.692672+0000","last_became_peered":"2024-03-30T13:10:57.692672+0000","last_unstale":"2024-03-30T13:10:58.672597+0000","last_undegraded":"2024-03-30T13:10:58.672597+0000","last_fullsized":"2024-03-30T13:10:58.672597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672248+0000","last_change":"2024-03-30T13:10:56.676836+0000","last_active":"2024-03-30T13:10:58.672248+0000","last_peered":"2024-03-30T13:10:58.672248+0000","last_clean":"2024-03-30T13:10:58.672248+0000","last_became_active":"2024-03-30T13:10:56.676586+0000","last_became_peered":"2024-03-30T13:10:56.676586+0000","last_unstale":"2024-03-30T13:10:58.672248+0000","last_undegraded":"2024-03-30T13:10:58.672248+0000","last_fullsized":"2024-03-30T13:10:58.672248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:15.014930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745111+0000","last_change":"2024-03-30T13:10:57.690831+0000","last_active":"2024-03-30T13:10:59.745111+0000","last_peered":"2024-03-30T13:10:59.745111+0000","last_clean":"2024-03-30T13:10:59.745111+0000","last_became_active":"2024-03-30T13:10:57.690522+0000","last_became_peered":"2024-03-30T13:10:57.690522+0000","last_unstale":"2024-03-30T13:10:59.745111+0000","last_undegraded":"2024-03-30T13:10:59.745111+0000","last_fullsized":"2024-03-30T13:10:59.745111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:16.019984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672474+0000","last_change":"2024-03-30T13:10:56.676370+0000","last_active":"2024-03-30T13:10:58.672474+0000","last_peered":"2024-03-30T13:10:58.672474+0000","last_clean":"2024-03-30T13:10:58.672474+0000","last_became_active":"2024-03-30T13:10:56.675812+0000","last_became_peered":"2024-03-30T13:10:56.675812+0000","last_unstale":"2024-03-30T13:10:58.672474+0000","last_undegraded":"2024-03-30T13:10:58.672474+0000","last_fullsized":"2024-03-30T13:10:58.672474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:24.867217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671859+0000","last_change":"2024-03-30T13:10:57.684924+0000","last_active":"2024-03-30T13:10:58.671859+0000","last_peered":"2024-03-30T13:10:58.671859+0000","last_clean":"2024-03-30T13:10:58.671859+0000","last_became_active":"2024-03-30T13:10:57.684588+0000","last_became_peered":"2024-03-30T13:10:57.684588+0000","last_unstale":"2024-03-30T13:10:58.671859+0000","last_undegraded":"2024-03-30T13:10:58.671859+0000","last_fullsized":"2024-03-30T13:10:58.671859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:20:39.043512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676374+0000","last_change":"2024-03-30T13:10:56.683571+0000","last_active":"2024-03-30T13:10:58.676374+0000","last_peered":"2024-03-30T13:10:58.676374+0000","last_clean":"2024-03-30T13:10:58.676374+0000","last_became_active":"2024-03-30T13:10:56.683215+0000","last_became_peered":"2024-03-30T13:10:56.683215+0000","last_unstale":"2024-03-30T13:10:58.676374+0000","last_undegraded":"2024-03-30T13:10:58.676374+0000","last_fullsized":"2024-03-30T13:10:58.676374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033087+0000","last_change":"2024-03-30T13:10:57.690221+0000","last_active":"2024-03-30T13:10:59.033087+0000","last_peered":"2024-03-30T13:10:59.033087+0000","last_clean":"2024-03-30T13:10:59.033087+0000","last_became_active":"2024-03-30T13:10:57.689942+0000","last_became_peered":"2024-03-30T13:10:57.689942+0000","last_unstale":"2024-03-30T13:10:59.033087+0000","last_undegraded":"2024-03-30T13:10:59.033087+0000","last_fullsized":"2024-03-30T13:10:59.033087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745733+0000","last_change":"2024-03-30T13:10:56.674381+0000","last_active":"2024-03-30T13:10:59.745733+0000","last_peered":"2024-03-30T13:10:59.745733+0000","last_clean":"2024-03-30T13:10:59.745733+0000","last_became_active":"2024-03-30T13:10:56.674086+0000","last_became_peered":"2024-03-30T13:10:56.674086+0000","last_unstale":"2024-03-30T13:10:59.745733+0000","last_undegraded":"2024-03-30T13:10:59.745733+0000","last_fullsized":"2024-03-30T13:10:59.745733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:02.130520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672207+0000","last_change":"2024-03-30T13:10:57.689668+0000","last_active":"2024-03-30T13:10:58.672207+0000","last_peered":"2024-03-30T13:10:58.672207+0000","last_clean":"2024-03-30T13:10:58.672207+0000","last_became_active":"2024-03-30T13:10:57.689363+0000","last_became_peered":"2024-03-30T13:10:57.689363+0000","last_unstale":"2024-03-30T13:10:58.672207+0000","last_undegraded":"2024-03-30T13:10:58.672207+0000","last_fullsized":"2024-03-30T13:10:58.672207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:25.435052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032876+0000","last_change":"2024-03-30T13:10:56.676925+0000","last_active":"2024-03-30T13:10:59.032876+0000","last_peered":"2024-03-30T13:10:59.032876+0000","last_clean":"2024-03-30T13:10:59.032876+0000","last_became_active":"2024-03-30T13:10:56.676627+0000","last_became_peered":"2024-03-30T13:10:56.676627+0000","last_unstale":"2024-03-30T13:10:59.032876+0000","last_undegraded":"2024-03-30T13:10:59.032876+0000","last_fullsized":"2024-03-30T13:10:59.032876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672078+0000","last_change":"2024-03-30T13:10:57.686177+0000","last_active":"2024-03-30T13:10:58.672078+0000","last_peered":"2024-03-30T13:10:58.672078+0000","last_clean":"2024-03-30T13:10:58.672078+0000","last_became_active":"2024-03-30T13:10:57.685857+0000","last_became_peered":"2024-03-30T13:10:57.685857+0000","last_unstale":"2024-03-30T13:10:58.672078+0000","last_undegraded":"2024-03-30T13:10:58.672078+0000","last_fullsized":"2024-03-30T13:10:58.672078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:28.503626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683617+0000","last_change":"2024-03-30T13:10:56.681832+0000","last_active":"2024-03-30T13:10:56.683617+0000","last_peered":"2024-03-30T13:10:56.683617+0000","last_clean":"2024-03-30T13:10:56.683617+0000","last_became_active":"2024-03-30T13:10:56.681522+0000","last_became_peered":"2024-03-30T13:10:56.681522+0000","last_unstale":"2024-03-30T13:10:56.683617+0000","last_undegraded":"2024-03-30T13:10:56.683617+0000","last_fullsized":"2024-03-30T13:10:56.683617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:30:19.213692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672924+0000","last_change":"2024-03-30T13:10:56.672924+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672924+0000","last_undegraded":"2024-03-30T13:10:56.672924+0000","last_fullsized":"2024-03-30T13:10:56.672924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672976+0000","last_change":"2024-03-30T13:10:56.681831+0000","last_active":"2024-03-30T13:10:58.672976+0000","last_peered":"2024-03-30T13:10:58.672976+0000","last_clean":"2024-03-30T13:10:58.672976+0000","last_became_active":"2024-03-30T13:10:56.681581+0000","last_became_peered":"2024-03-30T13:10:56.681581+0000","last_unstale":"2024-03-30T13:10:58.672976+0000","last_undegraded":"2024-03-30T13:10:58.672976+0000","last_fullsized":"2024-03-30T13:10:58.672976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:54.818674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.673095+0000","last_change":"2024-03-30T13:10:57.689473+0000","last_active":"2024-03-30T13:10:58.673095+0000","last_peered":"2024-03-30T13:10:58.673095+0000","last_clean":"2024-03-30T13:10:58.673095+0000","last_became_active":"2024-03-30T13:10:57.689184+0000","last_became_peered":"2024-03-30T13:10:57.689184+0000","last_unstale":"2024-03-30T13:10:58.673095+0000","last_undegraded":"2024-03-30T13:10:58.673095+0000","last_fullsized":"2024-03-30T13:10:58.673095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678850+0000","last_change":"2024-03-30T13:10:56.671520+0000","last_active":"2024-03-30T13:10:58.678850+0000","last_peered":"2024-03-30T13:10:58.678850+0000","last_clean":"2024-03-30T13:10:58.678850+0000","last_became_active":"2024-03-30T13:10:56.671198+0000","last_became_peered":"2024-03-30T13:10:56.671198+0000","last_unstale":"2024-03-30T13:10:58.678850+0000","last_undegraded":"2024-03-30T13:10:58.678850+0000","last_fullsized":"2024-03-30T13:10:58.678850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:48:29.323071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.679016+0000","last_change":"2024-03-30T13:10:57.682760+0000","last_active":"2024-03-30T13:10:58.679016+0000","last_peered":"2024-03-30T13:10:58.679016+0000","last_clean":"2024-03-30T13:10:58.679016+0000","last_became_active":"2024-03-30T13:10:57.682495+0000","last_became_peered":"2024-03-30T13:10:57.682495+0000","last_unstale":"2024-03-30T13:10:58.679016+0000","last_undegraded":"2024-03-30T13:10:58.679016+0000","last_fullsized":"2024-03-30T13:10:58.679016+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:03:14.415400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676601+0000","last_change":"2024-03-30T13:10:56.679764+0000","last_active":"2024-03-30T13:10:58.676601+0000","last_peered":"2024-03-30T13:10:58.676601+0000","last_clean":"2024-03-30T13:10:58.676601+0000","last_became_active":"2024-03-30T13:10:56.679483+0000","last_became_peered":"2024-03-30T13:10:56.679483+0000","last_unstale":"2024-03-30T13:10:58.676601+0000","last_undegraded":"2024-03-30T13:10:58.676601+0000","last_fullsized":"2024-03-30T13:10:58.676601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033042+0000","last_change":"2024-03-30T13:10:57.693368+0000","last_active":"2024-03-30T13:10:59.033042+0000","last_peered":"2024-03-30T13:10:59.033042+0000","last_clean":"2024-03-30T13:10:59.033042+0000","last_became_active":"2024-03-30T13:10:57.693076+0000","last_became_peered":"2024-03-30T13:10:57.693076+0000","last_unstale":"2024-03-30T13:10:59.033042+0000","last_undegraded":"2024-03-30T13:10:59.033042+0000","last_fullsized":"2024-03-30T13:10:59.033042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678626+0000","last_change":"2024-03-30T13:10:56.677591+0000","last_active":"2024-03-30T13:10:58.678626+0000","last_peered":"2024-03-30T13:10:58.678626+0000","last_clean":"2024-03-30T13:10:58.678626+0000","last_became_active":"2024-03-30T13:10:56.677289+0000","last_became_peered":"2024-03-30T13:10:56.677289+0000","last_unstale":"2024-03-30T13:10:58.678626+0000","last_undegraded":"2024-03-30T13:10:58.678626+0000","last_fullsized":"2024-03-30T13:10:58.678626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:55:48.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745181+0000","last_change":"2024-03-30T13:10:57.686202+0000","last_active":"2024-03-30T13:10:59.745181+0000","last_peered":"2024-03-30T13:10:59.745181+0000","last_clean":"2024-03-30T13:10:59.745181+0000","last_became_active":"2024-03-30T13:10:57.685942+0000","last_became_peered":"2024-03-30T13:10:57.685942+0000","last_unstale":"2024-03-30T13:10:59.745181+0000","last_undegraded":"2024-03-30T13:10:59.745181+0000","last_fullsized":"2024-03-30T13:10:59.745181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678579+0000","last_change":"2024-03-30T13:10:56.677268+0000","last_active":"2024-03-30T13:10:58.678579+0000","last_peered":"2024-03-30T13:10:58.678579+0000","last_clean":"2024-03-30T13:10:58.678579+0000","last_became_active":"2024-03-30T13:10:56.676985+0000","last_became_peered":"2024-03-30T13:10:56.676985+0000","last_unstale":"2024-03-30T13:10:58.678579+0000","last_undegraded":"2024-03-30T13:10:58.678579+0000","last_fullsized":"2024-03-30T13:10:58.678579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:31:09.416890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672390+0000","last_change":"2024-03-30T13:10:56.672390+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672390+0000","last_undegraded":"2024-03-30T13:10:56.672390+0000","last_fullsized":"2024-03-30T13:10:56.672390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033098+0000","last_change":"2024-03-30T13:10:56.680608+0000","last_active":"2024-03-30T13:10:59.033098+0000","last_peered":"2024-03-30T13:10:59.033098+0000","last_clean":"2024-03-30T13:10:59.033098+0000","last_became_active":"2024-03-30T13:10:56.680346+0000","last_became_peered":"2024-03-30T13:10:56.680346+0000","last_unstale":"2024-03-30T13:10:59.033098+0000","last_undegraded":"2024-03-30T13:10:59.033098+0000","last_fullsized":"2024-03-30T13:10:59.033098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672201+0000","last_change":"2024-03-30T13:10:57.678124+0000","last_active":"2024-03-30T13:10:58.672201+0000","last_peered":"2024-03-30T13:10:58.672201+0000","last_clean":"2024-03-30T13:10:58.672201+0000","last_became_active":"2024-03-30T13:10:57.677826+0000","last_became_peered":"2024-03-30T13:10:57.677826+0000","last_unstale":"2024-03-30T13:10:58.672201+0000","last_undegraded":"2024-03-30T13:10:58.672201+0000","last_fullsized":"2024-03-30T13:10:58.672201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:37.076824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678461+0000","last_change":"2024-03-30T13:10:56.678610+0000","last_active":"2024-03-30T13:10:58.678461+0000","last_peered":"2024-03-30T13:10:58.678461+0000","last_clean":"2024-03-30T13:10:58.678461+0000","last_became_active":"2024-03-30T13:10:56.678357+0000","last_became_peered":"2024-03-30T13:10:56.678357+0000","last_unstale":"2024-03-30T13:10:58.678461+0000","last_undegraded":"2024-03-30T13:10:58.678461+0000","last_fullsized":"2024-03-30T13:10:58.678461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:40.674849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676600+0000","last_change":"2024-03-30T13:10:57.687196+0000","last_active":"2024-03-30T13:10:58.676600+0000","last_peered":"2024-03-30T13:10:58.676600+0000","last_clean":"2024-03-30T13:10:58.676600+0000","last_became_active":"2024-03-30T13:10:57.686683+0000","last_became_peered":"2024-03-30T13:10:57.686683+0000","last_unstale":"2024-03-30T13:10:58.676600+0000","last_undegraded":"2024-03-30T13:10:58.676600+0000","last_fullsized":"2024-03-30T13:10:58.676600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:44:15.354797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683925+0000","last_change":"2024-03-30T13:10:56.682787+0000","last_active":"2024-03-30T13:10:56.683925+0000","last_peered":"2024-03-30T13:10:56.683925+0000","last_clean":"2024-03-30T13:10:56.683925+0000","last_became_active":"2024-03-30T13:10:56.682536+0000","last_became_peered":"2024-03-30T13:10:56.682536+0000","last_unstale":"2024-03-30T13:10:56.683925+0000","last_undegraded":"2024-03-30T13:10:56.683925+0000","last_fullsized":"2024-03-30T13:10:56.683925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:30:59.057923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033243+0000","last_change":"2024-03-30T13:10:57.686653+0000","last_active":"2024-03-30T13:10:59.033243+0000","last_peered":"2024-03-30T13:10:59.033243+0000","last_clean":"2024-03-30T13:10:59.033243+0000","last_became_active":"2024-03-30T13:10:57.686380+0000","last_became_peered":"2024-03-30T13:10:57.686380+0000","last_unstale":"2024-03-30T13:10:59.033243+0000","last_undegraded":"2024-03-30T13:10:59.033243+0000","last_fullsized":"2024-03-30T13:10:59.033243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:13:33.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680418+0000","last_change":"2024-03-30T13:10:56.682042+0000","last_active":"2024-03-30T13:10:58.680418+0000","last_peered":"2024-03-30T13:10:58.680418+0000","last_clean":"2024-03-30T13:10:58.680418+0000","last_became_active":"2024-03-30T13:10:56.681748+0000","last_became_peered":"2024-03-30T13:10:56.681748+0000","last_unstale":"2024-03-30T13:10:58.680418+0000","last_undegraded":"2024-03-30T13:10:58.680418+0000","last_fullsized":"2024-03-30T13:10:58.680418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:55:55.240936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672322+0000","last_change":"2024-03-30T13:10:57.676679+0000","last_active":"2024-03-30T13:10:58.672322+0000","last_peered":"2024-03-30T13:10:58.672322+0000","last_clean":"2024-03-30T13:10:58.672322+0000","last_became_active":"2024-03-30T13:10:57.675136+0000","last_became_peered":"2024-03-30T13:10:57.675136+0000","last_unstale":"2024-03-30T13:10:58.672322+0000","last_undegraded":"2024-03-30T13:10:58.672322+0000","last_fullsized":"2024-03-30T13:10:58.672322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:01:31.125961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745366+0000","last_change":"2024-03-30T13:10:56.682275+0000","last_active":"2024-03-30T13:10:59.745366+0000","last_peered":"2024-03-30T13:10:59.745366+0000","last_clean":"2024-03-30T13:10:59.745366+0000","last_became_active":"2024-03-30T13:10:56.681990+0000","last_became_peered":"2024-03-30T13:10:56.681990+0000","last_unstale":"2024-03-30T13:10:59.745366+0000","last_undegraded":"2024-03-30T13:10:59.745366+0000","last_fullsized":"2024-03-30T13:10:59.745366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745290+0000","last_change":"2024-03-30T13:10:57.692363+0000","last_active":"2024-03-30T13:10:59.745290+0000","last_peered":"2024-03-30T13:10:59.745290+0000","last_clean":"2024-03-30T13:10:59.745290+0000","last_became_active":"2024-03-30T13:10:57.692063+0000","last_became_peered":"2024-03-30T13:10:57.692063+0000","last_unstale":"2024-03-30T13:10:59.745290+0000","last_undegraded":"2024-03-30T13:10:59.745290+0000","last_fullsized":"2024-03-30T13:10:59.745290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745367+0000","last_change":"2024-03-30T13:10:56.680716+0000","last_active":"2024-03-30T13:10:59.745367+0000","last_peered":"2024-03-30T13:10:59.745367+0000","last_clean":"2024-03-30T13:10:59.745367+0000","last_became_active":"2024-03-30T13:10:56.680446+0000","last_became_peered":"2024-03-30T13:10:56.680446+0000","last_unstale":"2024-03-30T13:10:59.745367+0000","last_undegraded":"2024-03-30T13:10:59.745367+0000","last_fullsized":"2024-03-30T13:10:59.745367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676703+0000","last_change":"2024-03-30T13:10:57.684334+0000","last_active":"2024-03-30T13:10:58.676703+0000","last_peered":"2024-03-30T13:10:58.676703+0000","last_clean":"2024-03-30T13:10:58.676703+0000","last_became_active":"2024-03-30T13:10:57.684037+0000","last_became_peered":"2024-03-30T13:10:57.684037+0000","last_unstale":"2024-03-30T13:10:58.676703+0000","last_undegraded":"2024-03-30T13:10:58.676703+0000","last_fullsized":"2024-03-30T13:10:58.676703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:07:21.631648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745812+0000","last_change":"2024-03-30T13:10:56.676202+0000","last_active":"2024-03-30T13:10:59.745812+0000","last_peered":"2024-03-30T13:10:59.745812+0000","last_clean":"2024-03-30T13:10:59.745812+0000","last_became_active":"2024-03-30T13:10:56.675907+0000","last_became_peered":"2024-03-30T13:10:56.675907+0000","last_unstale":"2024-03-30T13:10:59.745812+0000","last_undegraded":"2024-03-30T13:10:59.745812+0000","last_fullsized":"2024-03-30T13:10:59.745812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:58.975900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672618+0000","last_change":"2024-03-30T13:10:57.673049+0000","last_active":"2024-03-30T13:10:58.672618+0000","last_peered":"2024-03-30T13:10:58.672618+0000","last_clean":"2024-03-30T13:10:58.672618+0000","last_became_active":"2024-03-30T13:10:57.672738+0000","last_became_peered":"2024-03-30T13:10:57.672738+0000","last_unstale":"2024-03-30T13:10:58.672618+0000","last_undegraded":"2024-03-30T13:10:58.672618+0000","last_fullsized":"2024-03-30T13:10:58.672618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672426+0000","last_change":"2024-03-30T13:10:56.675234+0000","last_active":"2024-03-30T13:10:58.672426+0000","last_peered":"2024-03-30T13:10:58.672426+0000","last_clean":"2024-03-30T13:10:58.672426+0000","last_became_active":"2024-03-30T13:10:56.674965+0000","last_became_peered":"2024-03-30T13:10:56.674965+0000","last_unstale":"2024-03-30T13:10:58.672426+0000","last_undegraded":"2024-03-30T13:10:58.672426+0000","last_fullsized":"2024-03-30T13:10:58.672426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:35.628852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033158+0000","last_change":"2024-03-30T13:10:57.683624+0000","last_active":"2024-03-30T13:10:59.033158+0000","last_peered":"2024-03-30T13:10:59.033158+0000","last_clean":"2024-03-30T13:10:59.033158+0000","last_became_active":"2024-03-30T13:10:57.683346+0000","last_became_peered":"2024-03-30T13:10:57.683346+0000","last_unstale":"2024-03-30T13:10:59.033158+0000","last_undegraded":"2024-03-30T13:10:59.033158+0000","last_fullsized":"2024-03-30T13:10:59.033158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671837+0000","last_change":"2024-03-30T13:10:56.682711+0000","last_active":"2024-03-30T13:10:58.671837+0000","last_peered":"2024-03-30T13:10:58.671837+0000","last_clean":"2024-03-30T13:10:58.671837+0000","last_became_active":"2024-03-30T13:10:56.682415+0000","last_became_peered":"2024-03-30T13:10:56.682415+0000","last_unstale":"2024-03-30T13:10:58.671837+0000","last_undegraded":"2024-03-30T13:10:58.671837+0000","last_fullsized":"2024-03-30T13:10:58.671837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:10:34.096760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.674446+0000","last_change":"2024-03-30T13:10:56.674446+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.674446+0000","last_undegraded":"2024-03-30T13:10:56.674446+0000","last_fullsized":"2024-03-30T13:10:56.674446+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033155+0000","last_change":"2024-03-30T13:10:56.681620+0000","last_active":"2024-03-30T13:10:59.033155+0000","last_peered":"2024-03-30T13:10:59.033155+0000","last_clean":"2024-03-30T13:10:59.033155+0000","last_became_active":"2024-03-30T13:10:56.681341+0000","last_became_peered":"2024-03-30T13:10:56.681341+0000","last_unstale":"2024-03-30T13:10:59.033155+0000","last_undegraded":"2024-03-30T13:10:59.033155+0000","last_fullsized":"2024-03-30T13:10:59.033155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:46:52.577909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032961+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:10:59.032961+0000","last_peered":"2024-03-30T13:10:59.032961+0000","last_clean":"2024-03-30T13:10:59.032961+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:10:59.032961+0000","last_undegraded":"2024-03-30T13:10:59.032961+0000","last_fullsized":"2024-03-30T13:10:59.032961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":199796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":199818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":782924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":178688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:02.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:02.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.255+0000 7f58c224a640 1 Processor -- start 2024-03-30T13:11:02.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.255+0000 7f58c224a640 1 -- start start 2024-03-30T13:11:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58c224a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58bc15e950 0x7f58bc0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58c224a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58bc0d48c0 0x7f58bc0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58c224a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58bc0d36f0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58c224a640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f58bc0d3cb0 con 0x7f58bc0d48c0 2024-03-30T13:11:02.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58c224a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f58bc0d4d00 con 0x7f58bc15e950 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58baffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58bc0d48c0 0x7f58bc0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58baffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58bc0d48c0 0x7f58bc0d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39538/0 (socket says 172.21.15.66:39538) 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58baffd640 1 -- 172.21.15.66:0/2380841224 learned_addr learned my addr 172.21.15.66:0/2380841224 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.256+0000 7f58bb7fe640 1 --2- 172.21.15.66:0/2380841224 >> v2:172.21.15.117:3300/0 conn(0x7f58bc15e950 0x7f58bc0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.257+0000 7f58baffd640 1 -- 172.21.15.66:0/2380841224 >> v2:172.21.15.117:3300/0 conn(0x7f58bc15e950 msgr2=0x7f58bc0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.257+0000 7f58baffd640 1 --2- 172.21.15.66:0/2380841224 >> v2:172.21.15.117:3300/0 conn(0x7f58bc15e950 0x7f58bc0d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.257+0000 7f58baffd640 1 -- 172.21.15.66:0/2380841224 >> v1:172.21.15.117:6789/0 conn(0x7f58bc0d2f00 legacy=0x7f58bc0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.257+0000 7f58baffd640 1 -- 172.21.15.66:0/2380841224 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc0d39d0 con 0x7f58bc0d48c0 2024-03-30T13:11:02.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.257+0000 7f58baffd640 1 --2- 172.21.15.66:0/2380841224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d48c0 0x7f58bc0d29c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0d3e30 tx=0x7f58a402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3908bd0c7544c8ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.257+0000 7f58ba7fc640 1 -- 172.21.15.66:0/2380841224 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4034070 con 0x7f58bc0d48c0 2024-03-30T13:11:02.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58ba7fc640 1 -- 172.21.15.66:0/2380841224 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a4030860 con 0x7f58bc0d48c0 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58ba7fc640 1 -- 172.21.15.66:0/2380841224 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4030c30 con 0x7f58bc0d48c0 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 -- 172.21.15.66:0/2380841224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d48c0 msgr2=0x7f58bc0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 --2- 172.21.15.66:0/2380841224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d48c0 0x7f58bc0d29c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0d3e30 tx=0x7f58a402b9a0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 -- 172.21.15.66:0/2380841224 shutdown_connections 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 --2- 172.21.15.66:0/2380841224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d48c0 0x7f58bc0d29c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 --2- 172.21.15.66:0/2380841224 >> v2:172.21.15.117:3300/0 conn(0x7f58bc15e950 0x7f58bc0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 -- 172.21.15.66:0/2380841224 >> 172.21.15.66:0/2380841224 conn(0x7f58bc0676e0 msgr2=0x7f58bc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 -- 172.21.15.66:0/2380841224 shutdown_connections 2024-03-30T13:11:02.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.258+0000 7f58c224a640 1 -- 172.21.15.66:0/2380841224 wait complete. 2024-03-30T13:11:02.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.260+0000 7f58c224a640 1 Processor -- start 2024-03-30T13:11:02.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.260+0000 7f58c224a640 1 -- start start 2024-03-30T13:11:02.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58c224a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 0x7f58bc14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58c224a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc0d48c0 0x7f58bc160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58c224a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc15e950 0x7f58bc0b9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58c224a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0d3390 con 0x7f58bc0d2f00 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58c224a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0b7da0 con 0x7f58bc15e950 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58c224a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc15ed30 con 0x7f58bc0d48c0 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 0x7f58bc14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 0x7f58bc14c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39542/0 (socket says 172.21.15.66:39542) 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 -- 172.21.15.66:0/1335961055 learned_addr learned my addr 172.21.15.66:0/1335961055 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58baffd640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc0d48c0 0x7f58bc160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bbfff640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc15e950 0x7f58bc0b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 -- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc0d48c0 msgr2=0x7f58bc160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc0d48c0 0x7f58bc160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 -- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc15e950 msgr2=0x7f58bc0b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc15e950 0x7f58bc0b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 -- 172.21.15.66:0/1335961055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc1615a0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.262+0000 7f58bb7fe640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 0x7f58bc14c190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d5f0 tx=0x7f58ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.263+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0125f0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.263+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac019070 con 0x7f58bc0d2f00 2024-03-30T13:11:02.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.263+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc161880 con 0x7f58bc0d2f00 2024-03-30T13:11:02.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.263+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0129d0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.263+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc0c80c0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.264+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58bc161da0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.264+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f58ac012b70 con 0x7f58bc0d2f00 2024-03-30T13:11:02.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.265+0000 7f58b3fff640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589c06c530 0x7f589c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.265+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f58ac08bad0 con 0x7f58bc0d2f00 2024-03-30T13:11:02.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.265+0000 7f58baffd640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589c06c530 0x7f589c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.265+0000 7f58baffd640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589c06c530 0x7f589c06e9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58a4036000 tx=0x7f58a4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.269+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f58ac059910 con 0x7f58bc0d2f00 2024-03-30T13:11:02.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.450+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58bc0c7f00 con 0x7f589c06c530 2024-03-30T13:11:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.456+0000 7f58b3fff640 1 -- 172.21.15.66:0/1335961055 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251050 (secure 0 0 0) 0x7f58a4030830 con 0x7f589c06c530 2024-03-30T13:11:02.462 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:02.462 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-03-30T13:11:02.457+0000 7fbb1c340640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T13:11:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:11:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589c06c530 msgr2=0x7f589c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589c06c530 0x7f589c06e9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58a4036000 tx=0x7f58a4008030 comp rx=0 tx=0).stop 2024-03-30T13:11:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 msgr2=0x7f58bc14c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 0x7f58bc14c190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d5f0 tx=0x7f58ac00dab0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 shutdown_connections 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589c06c530 0x7f589c06e9f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58bc15e950 0x7f58bc0b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58bc0d48c0 0x7f58bc160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 --2- 172.21.15.66:0/1335961055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc0d2f00 0x7f58bc14c190 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.459+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 >> 172.21.15.66:0/1335961055 conn(0x7f58bc0676e0 msgr2=0x7f58bc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.460+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 shutdown_connections 2024-03-30T13:11:02.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.460+0000 7f58c224a640 1 -- 172.21.15.66:0/1335961055 wait complete. 2024-03-30T13:11:02.474 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-03-30T13:11:00.810218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220852,"kb_used_data":4684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753868,"statfs":{"total":773094113280,"available":772867960832,"internally_reserved":0,"allocated":4796416,"data_stored":2731862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.155701"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672404+0000","last_change":"2024-03-30T13:10:57.684501+0000","last_active":"2024-03-30T13:10:58.672404+0000","last_peered":"2024-03-30T13:10:58.672404+0000","last_clean":"2024-03-30T13:10:58.672404+0000","last_became_active":"2024-03-30T13:10:57.684187+0000","last_became_peered":"2024-03-30T13:10:57.684187+0000","last_unstale":"2024-03-30T13:10:58.672404+0000","last_undegraded":"2024-03-30T13:10:58.672404+0000","last_fullsized":"2024-03-30T13:10:58.672404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:07.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672159+0000","last_change":"2024-03-30T13:10:57.681207+0000","last_active":"2024-03-30T13:10:58.672159+0000","last_peered":"2024-03-30T13:10:58.672159+0000","last_clean":"2024-03-30T13:10:58.672159+0000","last_became_active":"2024-03-30T13:10:57.680890+0000","last_became_peered":"2024-03-30T13:10:57.680890+0000","last_unstale":"2024-03-30T13:10:58.672159+0000","last_undegraded":"2024-03-30T13:10:58.672159+0000","last_fullsized":"2024-03-30T13:10:58.672159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:30:42.096995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745282+0000","last_change":"2024-03-30T13:10:57.695761+0000","last_active":"2024-03-30T13:10:59.745282+0000","last_peered":"2024-03-30T13:10:59.745282+0000","last_clean":"2024-03-30T13:10:59.745282+0000","last_became_active":"2024-03-30T13:10:57.695488+0000","last_became_peered":"2024-03-30T13:10:57.695488+0000","last_unstale":"2024-03-30T13:10:59.745282+0000","last_undegraded":"2024-03-30T13:10:59.745282+0000","last_fullsized":"2024-03-30T13:10:59.745282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678726+0000","last_change":"2024-03-30T13:10:57.682241+0000","last_active":"2024-03-30T13:10:58.678726+0000","last_peered":"2024-03-30T13:10:58.678726+0000","last_clean":"2024-03-30T13:10:58.678726+0000","last_became_active":"2024-03-30T13:10:57.681383+0000","last_became_peered":"2024-03-30T13:10:57.681383+0000","last_unstale":"2024-03-30T13:10:58.678726+0000","last_undegraded":"2024-03-30T13:10:58.678726+0000","last_fullsized":"2024-03-30T13:10:58.678726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:43:40.092786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678584+0000","last_change":"2024-03-30T13:10:57.680923+0000","last_active":"2024-03-30T13:10:58.678584+0000","last_peered":"2024-03-30T13:10:58.678584+0000","last_clean":"2024-03-30T13:10:58.678584+0000","last_became_active":"2024-03-30T13:10:57.680625+0000","last_became_peered":"2024-03-30T13:10:57.680625+0000","last_unstale":"2024-03-30T13:10:58.678584+0000","last_undegraded":"2024-03-30T13:10:58.678584+0000","last_fullsized":"2024-03-30T13:10:58.678584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:27:38.309021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676495+0000","last_change":"2024-03-30T13:10:57.687677+0000","last_active":"2024-03-30T13:10:58.676495+0000","last_peered":"2024-03-30T13:10:58.676495+0000","last_clean":"2024-03-30T13:10:58.676495+0000","last_became_active":"2024-03-30T13:10:57.687348+0000","last_became_peered":"2024-03-30T13:10:57.687348+0000","last_unstale":"2024-03-30T13:10:58.676495+0000","last_undegraded":"2024-03-30T13:10:58.676495+0000","last_fullsized":"2024-03-30T13:10:58.676495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:33:20.876411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.677018+0000","last_change":"2024-03-30T13:10:56.677018+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.677018+0000","last_undegraded":"2024-03-30T13:10:56.677018+0000","last_fullsized":"2024-03-30T13:10:56.677018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671840+0000","last_change":"2024-03-30T13:10:57.679497+0000","last_active":"2024-03-30T13:10:58.671840+0000","last_peered":"2024-03-30T13:10:58.671840+0000","last_clean":"2024-03-30T13:10:58.671840+0000","last_became_active":"2024-03-30T13:10:57.679201+0000","last_became_peered":"2024-03-30T13:10:57.679201+0000","last_unstale":"2024-03-30T13:10:58.671840+0000","last_undegraded":"2024-03-30T13:10:58.671840+0000","last_fullsized":"2024-03-30T13:10:58.671840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:04:54.328931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745602+0000","last_change":"2024-03-30T13:10:57.690985+0000","last_active":"2024-03-30T13:10:59.745602+0000","last_peered":"2024-03-30T13:10:59.745602+0000","last_clean":"2024-03-30T13:10:59.745602+0000","last_became_active":"2024-03-30T13:10:57.690657+0000","last_became_peered":"2024-03-30T13:10:57.690657+0000","last_unstale":"2024-03-30T13:10:59.745602+0000","last_undegraded":"2024-03-30T13:10:59.745602+0000","last_fullsized":"2024-03-30T13:10:59.745602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676334+0000","last_change":"2024-03-30T13:10:56.676334+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676334+0000","last_undegraded":"2024-03-30T13:10:56.676334+0000","last_fullsized":"2024-03-30T13:10:56.676334+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.673354+0000","last_change":"2024-03-30T13:10:56.673354+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.673354+0000","last_undegraded":"2024-03-30T13:10:56.673354+0000","last_fullsized":"2024-03-30T13:10:56.673354+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745292+0000","last_change":"2024-03-30T13:10:57.691635+0000","last_active":"2024-03-30T13:10:59.745292+0000","last_peered":"2024-03-30T13:10:59.745292+0000","last_clean":"2024-03-30T13:10:59.745292+0000","last_became_active":"2024-03-30T13:10:57.691340+0000","last_became_peered":"2024-03-30T13:10:57.691340+0000","last_unstale":"2024-03-30T13:10:59.745292+0000","last_undegraded":"2024-03-30T13:10:59.745292+0000","last_fullsized":"2024-03-30T13:10:59.745292+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676222+0000","last_change":"2024-03-30T13:10:56.676222+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676222+0000","last_undegraded":"2024-03-30T13:10:56.676222+0000","last_fullsized":"2024-03-30T13:10:56.676222+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672408+0000","last_change":"2024-03-30T13:10:57.686031+0000","last_active":"2024-03-30T13:10:58.672408+0000","last_peered":"2024-03-30T13:10:58.672408+0000","last_clean":"2024-03-30T13:10:58.672408+0000","last_became_active":"2024-03-30T13:10:57.685814+0000","last_became_peered":"2024-03-30T13:10:57.685814+0000","last_unstale":"2024-03-30T13:10:58.672408+0000","last_undegraded":"2024-03-30T13:10:58.672408+0000","last_fullsized":"2024-03-30T13:10:58.672408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.676235+0000","last_change":"2024-03-30T13:10:56.676235+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.676235+0000","last_undegraded":"2024-03-30T13:10:56.676235+0000","last_fullsized":"2024-03-30T13:10:56.676235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032816+0000","last_change":"2024-03-30T13:10:57.681811+0000","last_active":"2024-03-30T13:10:59.032816+0000","last_peered":"2024-03-30T13:10:59.032816+0000","last_clean":"2024-03-30T13:10:59.032816+0000","last_became_active":"2024-03-30T13:10:57.681471+0000","last_became_peered":"2024-03-30T13:10:57.681471+0000","last_unstale":"2024-03-30T13:10:59.032816+0000","last_undegraded":"2024-03-30T13:10:59.032816+0000","last_fullsized":"2024-03-30T13:10:59.032816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033359+0000","last_change":"2024-03-30T13:10:57.687549+0000","last_active":"2024-03-30T13:10:59.033359+0000","last_peered":"2024-03-30T13:10:59.033359+0000","last_clean":"2024-03-30T13:10:59.033359+0000","last_became_active":"2024-03-30T13:10:57.687268+0000","last_became_peered":"2024-03-30T13:10:57.687268+0000","last_unstale":"2024-03-30T13:10:59.033359+0000","last_undegraded":"2024-03-30T13:10:59.033359+0000","last_fullsized":"2024-03-30T13:10:59.033359+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:35.714074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746089+0000","last_change":"2024-03-30T13:10:57.683841+0000","last_active":"2024-03-30T13:10:59.746089+0000","last_peered":"2024-03-30T13:10:59.746089+0000","last_clean":"2024-03-30T13:10:59.746089+0000","last_became_active":"2024-03-30T13:10:57.683568+0000","last_became_peered":"2024-03-30T13:10:57.683568+0000","last_unstale":"2024-03-30T13:10:59.746089+0000","last_undegraded":"2024-03-30T13:10:59.746089+0000","last_fullsized":"2024-03-30T13:10:59.746089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:51:46.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678919+0000","last_change":"2024-03-30T13:10:57.688521+0000","last_active":"2024-03-30T13:10:58.678919+0000","last_peered":"2024-03-30T13:10:58.678919+0000","last_clean":"2024-03-30T13:10:58.678919+0000","last_became_active":"2024-03-30T13:10:57.688188+0000","last_became_peered":"2024-03-30T13:10:57.688188+0000","last_unstale":"2024-03-30T13:10:58.678919+0000","last_undegraded":"2024-03-30T13:10:58.678919+0000","last_fullsized":"2024-03-30T13:10:58.678919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:47.722209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745911+0000","last_change":"2024-03-30T13:10:57.682779+0000","last_active":"2024-03-30T13:10:59.745911+0000","last_peered":"2024-03-30T13:10:59.745911+0000","last_clean":"2024-03-30T13:10:59.745911+0000","last_became_active":"2024-03-30T13:10:57.682444+0000","last_became_peered":"2024-03-30T13:10:57.682444+0000","last_unstale":"2024-03-30T13:10:59.745911+0000","last_undegraded":"2024-03-30T13:10:59.745911+0000","last_fullsized":"2024-03-30T13:10:59.745911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:02:29.613267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672740+0000","last_change":"2024-03-30T13:10:56.672740+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672740+0000","last_undegraded":"2024-03-30T13:10:56.672740+0000","last_fullsized":"2024-03-30T13:10:56.672740+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672797+0000","last_change":"2024-03-30T13:10:57.692104+0000","last_active":"2024-03-30T13:10:58.672797+0000","last_peered":"2024-03-30T13:10:58.672797+0000","last_clean":"2024-03-30T13:10:58.672797+0000","last_became_active":"2024-03-30T13:10:57.691816+0000","last_became_peered":"2024-03-30T13:10:57.691816+0000","last_unstale":"2024-03-30T13:10:58.672797+0000","last_undegraded":"2024-03-30T13:10:58.672797+0000","last_fullsized":"2024-03-30T13:10:58.672797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032948+0000","last_change":"2024-03-30T13:10:57.691507+0000","last_active":"2024-03-30T13:10:59.032948+0000","last_peered":"2024-03-30T13:10:59.032948+0000","last_clean":"2024-03-30T13:10:59.032948+0000","last_became_active":"2024-03-30T13:10:57.691248+0000","last_became_peered":"2024-03-30T13:10:57.691248+0000","last_unstale":"2024-03-30T13:10:59.032948+0000","last_undegraded":"2024-03-30T13:10:59.032948+0000","last_fullsized":"2024-03-30T13:10:59.032948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678498+0000","last_change":"2024-03-30T13:10:57.685316+0000","last_active":"2024-03-30T13:10:58.678498+0000","last_peered":"2024-03-30T13:10:58.678498+0000","last_clean":"2024-03-30T13:10:58.678498+0000","last_became_active":"2024-03-30T13:10:57.685053+0000","last_became_peered":"2024-03-30T13:10:57.685053+0000","last_unstale":"2024-03-30T13:10:58.678498+0000","last_undegraded":"2024-03-30T13:10:58.678498+0000","last_fullsized":"2024-03-30T13:10:58.678498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:43:19.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746090+0000","last_change":"2024-03-30T13:10:57.684225+0000","last_active":"2024-03-30T13:10:59.746090+0000","last_peered":"2024-03-30T13:10:59.746090+0000","last_clean":"2024-03-30T13:10:59.746090+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:59.746090+0000","last_undegraded":"2024-03-30T13:10:59.746090+0000","last_fullsized":"2024-03-30T13:10:59.746090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:44:06.219419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676846+0000","last_change":"2024-03-30T13:10:57.687484+0000","last_active":"2024-03-30T13:10:58.676846+0000","last_peered":"2024-03-30T13:10:58.676846+0000","last_clean":"2024-03-30T13:10:58.676846+0000","last_became_active":"2024-03-30T13:10:57.687191+0000","last_became_peered":"2024-03-30T13:10:57.687191+0000","last_unstale":"2024-03-30T13:10:58.676846+0000","last_undegraded":"2024-03-30T13:10:58.676846+0000","last_fullsized":"2024-03-30T13:10:58.676846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:49:47.246424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746018+0000","last_change":"2024-03-30T13:10:57.687817+0000","last_active":"2024-03-30T13:10:59.746018+0000","last_peered":"2024-03-30T13:10:59.746018+0000","last_clean":"2024-03-30T13:10:59.746018+0000","last_became_active":"2024-03-30T13:10:57.687038+0000","last_became_peered":"2024-03-30T13:10:57.687038+0000","last_unstale":"2024-03-30T13:10:59.746018+0000","last_undegraded":"2024-03-30T13:10:59.746018+0000","last_fullsized":"2024-03-30T13:10:59.746018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:03:06.653896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.675608+0000","last_change":"2024-03-30T13:10:56.675608+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.675608+0000","last_undegraded":"2024-03-30T13:10:56.675608+0000","last_fullsized":"2024-03-30T13:10:56.675608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745905+0000","last_change":"2024-03-30T13:10:57.690320+0000","last_active":"2024-03-30T13:10:59.745905+0000","last_peered":"2024-03-30T13:10:59.745905+0000","last_clean":"2024-03-30T13:10:59.745905+0000","last_became_active":"2024-03-30T13:10:57.689740+0000","last_became_peered":"2024-03-30T13:10:57.689740+0000","last_unstale":"2024-03-30T13:10:59.745905+0000","last_undegraded":"2024-03-30T13:10:59.745905+0000","last_fullsized":"2024-03-30T13:10:59.745905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:55.823728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745918+0000","last_change":"2024-03-30T13:10:57.683564+0000","last_active":"2024-03-30T13:10:59.745918+0000","last_peered":"2024-03-30T13:10:59.745918+0000","last_clean":"2024-03-30T13:10:59.745918+0000","last_became_active":"2024-03-30T13:10:57.683225+0000","last_became_peered":"2024-03-30T13:10:57.683225+0000","last_unstale":"2024-03-30T13:10:59.745918+0000","last_undegraded":"2024-03-30T13:10:59.745918+0000","last_fullsized":"2024-03-30T13:10:59.745918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:36.633906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032986+0000","last_change":"2024-03-30T13:10:57.692664+0000","last_active":"2024-03-30T13:10:59.032986+0000","last_peered":"2024-03-30T13:10:59.032986+0000","last_clean":"2024-03-30T13:10:59.032986+0000","last_became_active":"2024-03-30T13:10:57.692376+0000","last_became_peered":"2024-03-30T13:10:57.692376+0000","last_unstale":"2024-03-30T13:10:59.032986+0000","last_undegraded":"2024-03-30T13:10:59.032986+0000","last_fullsized":"2024-03-30T13:10:59.032986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676356+0000","last_change":"2024-03-30T13:10:57.688812+0000","last_active":"2024-03-30T13:10:58.676356+0000","last_peered":"2024-03-30T13:10:58.676356+0000","last_clean":"2024-03-30T13:10:58.676356+0000","last_became_active":"2024-03-30T13:10:57.688187+0000","last_became_peered":"2024-03-30T13:10:57.688187+0000","last_unstale":"2024-03-30T13:10:58.676356+0000","last_undegraded":"2024-03-30T13:10:58.676356+0000","last_fullsized":"2024-03-30T13:10:58.676356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:13:34.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033219+0000","last_change":"2024-03-30T13:10:57.682984+0000","last_active":"2024-03-30T13:10:59.033219+0000","last_peered":"2024-03-30T13:10:59.033219+0000","last_clean":"2024-03-30T13:10:59.033219+0000","last_became_active":"2024-03-30T13:10:57.682685+0000","last_became_peered":"2024-03-30T13:10:57.682685+0000","last_unstale":"2024-03-30T13:10:59.033219+0000","last_undegraded":"2024-03-30T13:10:59.033219+0000","last_fullsized":"2024-03-30T13:10:59.033219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:28:51.189027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746047+0000","last_change":"2024-03-30T13:10:56.673822+0000","last_active":"2024-03-30T13:10:59.746047+0000","last_peered":"2024-03-30T13:10:59.746047+0000","last_clean":"2024-03-30T13:10:59.746047+0000","last_became_active":"2024-03-30T13:10:56.673533+0000","last_became_peered":"2024-03-30T13:10:56.673533+0000","last_unstale":"2024-03-30T13:10:59.746047+0000","last_undegraded":"2024-03-30T13:10:59.746047+0000","last_fullsized":"2024-03-30T13:10:59.746047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:51:29.821157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676839+0000","last_change":"2024-03-30T13:10:57.688749+0000","last_active":"2024-03-30T13:10:58.676839+0000","last_peered":"2024-03-30T13:10:58.676839+0000","last_clean":"2024-03-30T13:10:58.676839+0000","last_became_active":"2024-03-30T13:10:57.687457+0000","last_became_peered":"2024-03-30T13:10:57.687457+0000","last_unstale":"2024-03-30T13:10:58.676839+0000","last_undegraded":"2024-03-30T13:10:58.676839+0000","last_fullsized":"2024-03-30T13:10:58.676839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:42.131116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745498+0000","last_change":"2024-03-30T13:10:56.673044+0000","last_active":"2024-03-30T13:10:59.745498+0000","last_peered":"2024-03-30T13:10:59.745498+0000","last_clean":"2024-03-30T13:10:59.745498+0000","last_became_active":"2024-03-30T13:10:56.672750+0000","last_became_peered":"2024-03-30T13:10:56.672750+0000","last_unstale":"2024-03-30T13:10:59.745498+0000","last_undegraded":"2024-03-30T13:10:59.745498+0000","last_fullsized":"2024-03-30T13:10:59.745498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672861+0000","last_change":"2024-03-30T13:10:57.685414+0000","last_active":"2024-03-30T13:10:58.672861+0000","last_peered":"2024-03-30T13:10:58.672861+0000","last_clean":"2024-03-30T13:10:58.672861+0000","last_became_active":"2024-03-30T13:10:57.685131+0000","last_became_peered":"2024-03-30T13:10:57.685131+0000","last_unstale":"2024-03-30T13:10:58.672861+0000","last_undegraded":"2024-03-30T13:10:58.672861+0000","last_fullsized":"2024-03-30T13:10:58.672861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676898+0000","last_change":"2024-03-30T13:10:56.679798+0000","last_active":"2024-03-30T13:10:58.676898+0000","last_peered":"2024-03-30T13:10:58.676898+0000","last_clean":"2024-03-30T13:10:58.676898+0000","last_became_active":"2024-03-30T13:10:56.679500+0000","last_became_peered":"2024-03-30T13:10:56.679500+0000","last_unstale":"2024-03-30T13:10:58.676898+0000","last_undegraded":"2024-03-30T13:10:58.676898+0000","last_fullsized":"2024-03-30T13:10:58.676898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672316+0000","last_change":"2024-03-30T13:10:57.688829+0000","last_active":"2024-03-30T13:10:58.672316+0000","last_peered":"2024-03-30T13:10:58.672316+0000","last_clean":"2024-03-30T13:10:58.672316+0000","last_became_active":"2024-03-30T13:10:57.686054+0000","last_became_peered":"2024-03-30T13:10:57.686054+0000","last_unstale":"2024-03-30T13:10:58.672316+0000","last_undegraded":"2024-03-30T13:10:58.672316+0000","last_fullsized":"2024-03-30T13:10:58.672316+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:03:47.898459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672809+0000","last_change":"2024-03-30T13:10:56.674537+0000","last_active":"2024-03-30T13:10:58.672809+0000","last_peered":"2024-03-30T13:10:58.672809+0000","last_clean":"2024-03-30T13:10:58.672809+0000","last_became_active":"2024-03-30T13:10:56.674248+0000","last_became_peered":"2024-03-30T13:10:56.674248+0000","last_unstale":"2024-03-30T13:10:58.672809+0000","last_undegraded":"2024-03-30T13:10:58.672809+0000","last_fullsized":"2024-03-30T13:10:58.672809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745165+0000","last_change":"2024-03-30T13:10:57.693802+0000","last_active":"2024-03-30T13:10:59.745165+0000","last_peered":"2024-03-30T13:10:59.745165+0000","last_clean":"2024-03-30T13:10:59.745165+0000","last_became_active":"2024-03-30T13:10:57.693436+0000","last_became_peered":"2024-03-30T13:10:57.693436+0000","last_unstale":"2024-03-30T13:10:59.745165+0000","last_undegraded":"2024-03-30T13:10:59.745165+0000","last_fullsized":"2024-03-30T13:10:59.745165+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683595+0000","last_change":"2024-03-30T13:10:56.681576+0000","last_active":"2024-03-30T13:10:56.683595+0000","last_peered":"2024-03-30T13:10:56.683595+0000","last_clean":"2024-03-30T13:10:56.683595+0000","last_became_active":"2024-03-30T13:10:56.681081+0000","last_became_peered":"2024-03-30T13:10:56.681081+0000","last_unstale":"2024-03-30T13:10:56.683595+0000","last_undegraded":"2024-03-30T13:10:56.683595+0000","last_fullsized":"2024-03-30T13:10:56.683595+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:51:58.065326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.675811+0000","last_change":"2024-03-30T13:10:56.675811+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.675811+0000","last_undegraded":"2024-03-30T13:10:56.675811+0000","last_fullsized":"2024-03-30T13:10:56.675811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032969+0000","last_change":"2024-03-30T13:10:56.677670+0000","last_active":"2024-03-30T13:10:59.032969+0000","last_peered":"2024-03-30T13:10:59.032969+0000","last_clean":"2024-03-30T13:10:59.032969+0000","last_became_active":"2024-03-30T13:10:56.677367+0000","last_became_peered":"2024-03-30T13:10:56.677367+0000","last_unstale":"2024-03-30T13:10:59.032969+0000","last_undegraded":"2024-03-30T13:10:59.032969+0000","last_fullsized":"2024-03-30T13:10:59.032969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:14:10.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672971+0000","last_change":"2024-03-30T13:10:57.681840+0000","last_active":"2024-03-30T13:10:58.672971+0000","last_peered":"2024-03-30T13:10:58.672971+0000","last_clean":"2024-03-30T13:10:58.672971+0000","last_became_active":"2024-03-30T13:10:57.681548+0000","last_became_peered":"2024-03-30T13:10:57.681548+0000","last_unstale":"2024-03-30T13:10:58.672971+0000","last_undegraded":"2024-03-30T13:10:58.672971+0000","last_fullsized":"2024-03-30T13:10:58.672971+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676449+0000","last_change":"2024-03-30T13:10:56.682810+0000","last_active":"2024-03-30T13:10:58.676449+0000","last_peered":"2024-03-30T13:10:58.676449+0000","last_clean":"2024-03-30T13:10:58.676449+0000","last_became_active":"2024-03-30T13:10:56.682437+0000","last_became_peered":"2024-03-30T13:10:56.682437+0000","last_unstale":"2024-03-30T13:10:58.676449+0000","last_undegraded":"2024-03-30T13:10:58.676449+0000","last_fullsized":"2024-03-30T13:10:58.676449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672038+0000","last_change":"2024-03-30T13:10:57.678420+0000","last_active":"2024-03-30T13:10:58.672038+0000","last_peered":"2024-03-30T13:10:58.672038+0000","last_clean":"2024-03-30T13:10:58.672038+0000","last_became_active":"2024-03-30T13:10:57.678115+0000","last_became_peered":"2024-03-30T13:10:57.678115+0000","last_unstale":"2024-03-30T13:10:58.672038+0000","last_undegraded":"2024-03-30T13:10:58.672038+0000","last_fullsized":"2024-03-30T13:10:58.672038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:42:22.714841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672311+0000","last_change":"2024-03-30T13:10:56.678877+0000","last_active":"2024-03-30T13:10:58.672311+0000","last_peered":"2024-03-30T13:10:58.672311+0000","last_clean":"2024-03-30T13:10:58.672311+0000","last_became_active":"2024-03-30T13:10:56.678573+0000","last_became_peered":"2024-03-30T13:10:56.678573+0000","last_unstale":"2024-03-30T13:10:58.672311+0000","last_undegraded":"2024-03-30T13:10:58.672311+0000","last_fullsized":"2024-03-30T13:10:58.672311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678784+0000","last_change":"2024-03-30T13:10:57.685526+0000","last_active":"2024-03-30T13:10:58.678784+0000","last_peered":"2024-03-30T13:10:58.678784+0000","last_clean":"2024-03-30T13:10:58.678784+0000","last_became_active":"2024-03-30T13:10:57.681079+0000","last_became_peered":"2024-03-30T13:10:57.681079+0000","last_unstale":"2024-03-30T13:10:58.678784+0000","last_undegraded":"2024-03-30T13:10:58.678784+0000","last_fullsized":"2024-03-30T13:10:58.678784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:36:15.323960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672394+0000","last_change":"2024-03-30T13:10:56.680735+0000","last_active":"2024-03-30T13:10:58.672394+0000","last_peered":"2024-03-30T13:10:58.672394+0000","last_clean":"2024-03-30T13:10:58.672394+0000","last_became_active":"2024-03-30T13:10:56.680459+0000","last_became_peered":"2024-03-30T13:10:56.680459+0000","last_unstale":"2024-03-30T13:10:58.672394+0000","last_undegraded":"2024-03-30T13:10:58.672394+0000","last_fullsized":"2024-03-30T13:10:58.672394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:26.601466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033176+0000","last_change":"2024-03-30T13:10:57.687091+0000","last_active":"2024-03-30T13:10:59.033176+0000","last_peered":"2024-03-30T13:10:59.033176+0000","last_clean":"2024-03-30T13:10:59.033176+0000","last_became_active":"2024-03-30T13:10:57.686821+0000","last_became_peered":"2024-03-30T13:10:57.686821+0000","last_unstale":"2024-03-30T13:10:59.033176+0000","last_undegraded":"2024-03-30T13:10:59.033176+0000","last_fullsized":"2024-03-30T13:10:59.033176+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:16:42.538868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683691+0000","last_change":"2024-03-30T13:10:56.682722+0000","last_active":"2024-03-30T13:10:56.683691+0000","last_peered":"2024-03-30T13:10:56.683691+0000","last_clean":"2024-03-30T13:10:56.683691+0000","last_became_active":"2024-03-30T13:10:56.682272+0000","last_became_peered":"2024-03-30T13:10:56.682272+0000","last_unstale":"2024-03-30T13:10:56.683691+0000","last_undegraded":"2024-03-30T13:10:56.683691+0000","last_fullsized":"2024-03-30T13:10:56.683691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:51:40.244956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745459+0000","last_change":"2024-03-30T13:10:57.691155+0000","last_active":"2024-03-30T13:10:59.745459+0000","last_peered":"2024-03-30T13:10:59.745459+0000","last_clean":"2024-03-30T13:10:59.745459+0000","last_became_active":"2024-03-30T13:10:57.690702+0000","last_became_peered":"2024-03-30T13:10:57.690702+0000","last_unstale":"2024-03-30T13:10:59.745459+0000","last_undegraded":"2024-03-30T13:10:59.745459+0000","last_fullsized":"2024-03-30T13:10:59.745459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683944+0000","last_change":"2024-03-30T13:10:56.683050+0000","last_active":"2024-03-30T13:10:56.683944+0000","last_peered":"2024-03-30T13:10:56.683944+0000","last_clean":"2024-03-30T13:10:56.683944+0000","last_became_active":"2024-03-30T13:10:56.682514+0000","last_became_peered":"2024-03-30T13:10:56.682514+0000","last_unstale":"2024-03-30T13:10:56.683944+0000","last_undegraded":"2024-03-30T13:10:56.683944+0000","last_fullsized":"2024-03-30T13:10:56.683944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:40:52.009831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672597+0000","last_change":"2024-03-30T13:10:57.692928+0000","last_active":"2024-03-30T13:10:58.672597+0000","last_peered":"2024-03-30T13:10:58.672597+0000","last_clean":"2024-03-30T13:10:58.672597+0000","last_became_active":"2024-03-30T13:10:57.692672+0000","last_became_peered":"2024-03-30T13:10:57.692672+0000","last_unstale":"2024-03-30T13:10:58.672597+0000","last_undegraded":"2024-03-30T13:10:58.672597+0000","last_fullsized":"2024-03-30T13:10:58.672597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672248+0000","last_change":"2024-03-30T13:10:56.676836+0000","last_active":"2024-03-30T13:10:58.672248+0000","last_peered":"2024-03-30T13:10:58.672248+0000","last_clean":"2024-03-30T13:10:58.672248+0000","last_became_active":"2024-03-30T13:10:56.676586+0000","last_became_peered":"2024-03-30T13:10:56.676586+0000","last_unstale":"2024-03-30T13:10:58.672248+0000","last_undegraded":"2024-03-30T13:10:58.672248+0000","last_fullsized":"2024-03-30T13:10:58.672248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:15.014930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745111+0000","last_change":"2024-03-30T13:10:57.690831+0000","last_active":"2024-03-30T13:10:59.745111+0000","last_peered":"2024-03-30T13:10:59.745111+0000","last_clean":"2024-03-30T13:10:59.745111+0000","last_became_active":"2024-03-30T13:10:57.690522+0000","last_became_peered":"2024-03-30T13:10:57.690522+0000","last_unstale":"2024-03-30T13:10:59.745111+0000","last_undegraded":"2024-03-30T13:10:59.745111+0000","last_fullsized":"2024-03-30T13:10:59.745111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:16.019984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672474+0000","last_change":"2024-03-30T13:10:56.676370+0000","last_active":"2024-03-30T13:10:58.672474+0000","last_peered":"2024-03-30T13:10:58.672474+0000","last_clean":"2024-03-30T13:10:58.672474+0000","last_became_active":"2024-03-30T13:10:56.675812+0000","last_became_peered":"2024-03-30T13:10:56.675812+0000","last_unstale":"2024-03-30T13:10:58.672474+0000","last_undegraded":"2024-03-30T13:10:58.672474+0000","last_fullsized":"2024-03-30T13:10:58.672474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:24.867217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671859+0000","last_change":"2024-03-30T13:10:57.684924+0000","last_active":"2024-03-30T13:10:58.671859+0000","last_peered":"2024-03-30T13:10:58.671859+0000","last_clean":"2024-03-30T13:10:58.671859+0000","last_became_active":"2024-03-30T13:10:57.684588+0000","last_became_peered":"2024-03-30T13:10:57.684588+0000","last_unstale":"2024-03-30T13:10:58.671859+0000","last_undegraded":"2024-03-30T13:10:58.671859+0000","last_fullsized":"2024-03-30T13:10:58.671859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:20:39.043512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676374+0000","last_change":"2024-03-30T13:10:56.683571+0000","last_active":"2024-03-30T13:10:58.676374+0000","last_peered":"2024-03-30T13:10:58.676374+0000","last_clean":"2024-03-30T13:10:58.676374+0000","last_became_active":"2024-03-30T13:10:56.683215+0000","last_became_peered":"2024-03-30T13:10:56.683215+0000","last_unstale":"2024-03-30T13:10:58.676374+0000","last_undegraded":"2024-03-30T13:10:58.676374+0000","last_fullsized":"2024-03-30T13:10:58.676374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033087+0000","last_change":"2024-03-30T13:10:57.690221+0000","last_active":"2024-03-30T13:10:59.033087+0000","last_peered":"2024-03-30T13:10:59.033087+0000","last_clean":"2024-03-30T13:10:59.033087+0000","last_became_active":"2024-03-30T13:10:57.689942+0000","last_became_peered":"2024-03-30T13:10:57.689942+0000","last_unstale":"2024-03-30T13:10:59.033087+0000","last_undegraded":"2024-03-30T13:10:59.033087+0000","last_fullsized":"2024-03-30T13:10:59.033087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745733+0000","last_change":"2024-03-30T13:10:56.674381+0000","last_active":"2024-03-30T13:10:59.745733+0000","last_peered":"2024-03-30T13:10:59.745733+0000","last_clean":"2024-03-30T13:10:59.745733+0000","last_became_active":"2024-03-30T13:10:56.674086+0000","last_became_peered":"2024-03-30T13:10:56.674086+0000","last_unstale":"2024-03-30T13:10:59.745733+0000","last_undegraded":"2024-03-30T13:10:59.745733+0000","last_fullsized":"2024-03-30T13:10:59.745733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:02.130520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672207+0000","last_change":"2024-03-30T13:10:57.689668+0000","last_active":"2024-03-30T13:10:58.672207+0000","last_peered":"2024-03-30T13:10:58.672207+0000","last_clean":"2024-03-30T13:10:58.672207+0000","last_became_active":"2024-03-30T13:10:57.689363+0000","last_became_peered":"2024-03-30T13:10:57.689363+0000","last_unstale":"2024-03-30T13:10:58.672207+0000","last_undegraded":"2024-03-30T13:10:58.672207+0000","last_fullsized":"2024-03-30T13:10:58.672207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:25.435052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032876+0000","last_change":"2024-03-30T13:10:56.676925+0000","last_active":"2024-03-30T13:10:59.032876+0000","last_peered":"2024-03-30T13:10:59.032876+0000","last_clean":"2024-03-30T13:10:59.032876+0000","last_became_active":"2024-03-30T13:10:56.676627+0000","last_became_peered":"2024-03-30T13:10:56.676627+0000","last_unstale":"2024-03-30T13:10:59.032876+0000","last_undegraded":"2024-03-30T13:10:59.032876+0000","last_fullsized":"2024-03-30T13:10:59.032876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672078+0000","last_change":"2024-03-30T13:10:57.686177+0000","last_active":"2024-03-30T13:10:58.672078+0000","last_peered":"2024-03-30T13:10:58.672078+0000","last_clean":"2024-03-30T13:10:58.672078+0000","last_became_active":"2024-03-30T13:10:57.685857+0000","last_became_peered":"2024-03-30T13:10:57.685857+0000","last_unstale":"2024-03-30T13:10:58.672078+0000","last_undegraded":"2024-03-30T13:10:58.672078+0000","last_fullsized":"2024-03-30T13:10:58.672078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:28.503626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683617+0000","last_change":"2024-03-30T13:10:56.681832+0000","last_active":"2024-03-30T13:10:56.683617+0000","last_peered":"2024-03-30T13:10:56.683617+0000","last_clean":"2024-03-30T13:10:56.683617+0000","last_became_active":"2024-03-30T13:10:56.681522+0000","last_became_peered":"2024-03-30T13:10:56.681522+0000","last_unstale":"2024-03-30T13:10:56.683617+0000","last_undegraded":"2024-03-30T13:10:56.683617+0000","last_fullsized":"2024-03-30T13:10:56.683617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:30:19.213692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672924+0000","last_change":"2024-03-30T13:10:56.672924+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672924+0000","last_undegraded":"2024-03-30T13:10:56.672924+0000","last_fullsized":"2024-03-30T13:10:56.672924+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672976+0000","last_change":"2024-03-30T13:10:56.681831+0000","last_active":"2024-03-30T13:10:58.672976+0000","last_peered":"2024-03-30T13:10:58.672976+0000","last_clean":"2024-03-30T13:10:58.672976+0000","last_became_active":"2024-03-30T13:10:56.681581+0000","last_became_peered":"2024-03-30T13:10:56.681581+0000","last_unstale":"2024-03-30T13:10:58.672976+0000","last_undegraded":"2024-03-30T13:10:58.672976+0000","last_fullsized":"2024-03-30T13:10:58.672976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:54.818674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.673095+0000","last_change":"2024-03-30T13:10:57.689473+0000","last_active":"2024-03-30T13:10:58.673095+0000","last_peered":"2024-03-30T13:10:58.673095+0000","last_clean":"2024-03-30T13:10:58.673095+0000","last_became_active":"2024-03-30T13:10:57.689184+0000","last_became_peered":"2024-03-30T13:10:57.689184+0000","last_unstale":"2024-03-30T13:10:58.673095+0000","last_undegraded":"2024-03-30T13:10:58.673095+0000","last_fullsized":"2024-03-30T13:10:58.673095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678850+0000","last_change":"2024-03-30T13:10:56.671520+0000","last_active":"2024-03-30T13:10:58.678850+0000","last_peered":"2024-03-30T13:10:58.678850+0000","last_clean":"2024-03-30T13:10:58.678850+0000","last_became_active":"2024-03-30T13:10:56.671198+0000","last_became_peered":"2024-03-30T13:10:56.671198+0000","last_unstale":"2024-03-30T13:10:58.678850+0000","last_undegraded":"2024-03-30T13:10:58.678850+0000","last_fullsized":"2024-03-30T13:10:58.678850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:48:29.323071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.679016+0000","last_change":"2024-03-30T13:10:57.682760+0000","last_active":"2024-03-30T13:10:58.679016+0000","last_peered":"2024-03-30T13:10:58.679016+0000","last_clean":"2024-03-30T13:10:58.679016+0000","last_became_active":"2024-03-30T13:10:57.682495+0000","last_became_peered":"2024-03-30T13:10:57.682495+0000","last_unstale":"2024-03-30T13:10:58.679016+0000","last_undegraded":"2024-03-30T13:10:58.679016+0000","last_fullsized":"2024-03-30T13:10:58.679016+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:03:14.415400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676601+0000","last_change":"2024-03-30T13:10:56.679764+0000","last_active":"2024-03-30T13:10:58.676601+0000","last_peered":"2024-03-30T13:10:58.676601+0000","last_clean":"2024-03-30T13:10:58.676601+0000","last_became_active":"2024-03-30T13:10:56.679483+0000","last_became_peered":"2024-03-30T13:10:56.679483+0000","last_unstale":"2024-03-30T13:10:58.676601+0000","last_undegraded":"2024-03-30T13:10:58.676601+0000","last_fullsized":"2024-03-30T13:10:58.676601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033042+0000","last_change":"2024-03-30T13:10:57.693368+0000","last_active":"2024-03-30T13:10:59.033042+0000","last_peered":"2024-03-30T13:10:59.033042+0000","last_clean":"2024-03-30T13:10:59.033042+0000","last_became_active":"2024-03-30T13:10:57.693076+0000","last_became_peered":"2024-03-30T13:10:57.693076+0000","last_unstale":"2024-03-30T13:10:59.033042+0000","last_undegraded":"2024-03-30T13:10:59.033042+0000","last_fullsized":"2024-03-30T13:10:59.033042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678626+0000","last_change":"2024-03-30T13:10:56.677591+0000","last_active":"2024-03-30T13:10:58.678626+0000","last_peered":"2024-03-30T13:10:58.678626+0000","last_clean":"2024-03-30T13:10:58.678626+0000","last_became_active":"2024-03-30T13:10:56.677289+0000","last_became_peered":"2024-03-30T13:10:56.677289+0000","last_unstale":"2024-03-30T13:10:58.678626+0000","last_undegraded":"2024-03-30T13:10:58.678626+0000","last_fullsized":"2024-03-30T13:10:58.678626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:55:48.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745181+0000","last_change":"2024-03-30T13:10:57.686202+0000","last_active":"2024-03-30T13:10:59.745181+0000","last_peered":"2024-03-30T13:10:59.745181+0000","last_clean":"2024-03-30T13:10:59.745181+0000","last_became_active":"2024-03-30T13:10:57.685942+0000","last_became_peered":"2024-03-30T13:10:57.685942+0000","last_unstale":"2024-03-30T13:10:59.745181+0000","last_undegraded":"2024-03-30T13:10:59.745181+0000","last_fullsized":"2024-03-30T13:10:59.745181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678579+0000","last_change":"2024-03-30T13:10:56.677268+0000","last_active":"2024-03-30T13:10:58.678579+0000","last_peered":"2024-03-30T13:10:58.678579+0000","last_clean":"2024-03-30T13:10:58.678579+0000","last_became_active":"2024-03-30T13:10:56.676985+0000","last_became_peered":"2024-03-30T13:10:56.676985+0000","last_unstale":"2024-03-30T13:10:58.678579+0000","last_undegraded":"2024-03-30T13:10:58.678579+0000","last_fullsized":"2024-03-30T13:10:58.678579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:31:09.416890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.672390+0000","last_change":"2024-03-30T13:10:56.672390+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.672390+0000","last_undegraded":"2024-03-30T13:10:56.672390+0000","last_fullsized":"2024-03-30T13:10:56.672390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033098+0000","last_change":"2024-03-30T13:10:56.680608+0000","last_active":"2024-03-30T13:10:59.033098+0000","last_peered":"2024-03-30T13:10:59.033098+0000","last_clean":"2024-03-30T13:10:59.033098+0000","last_became_active":"2024-03-30T13:10:56.680346+0000","last_became_peered":"2024-03-30T13:10:56.680346+0000","last_unstale":"2024-03-30T13:10:59.033098+0000","last_undegraded":"2024-03-30T13:10:59.033098+0000","last_fullsized":"2024-03-30T13:10:59.033098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672201+0000","last_change":"2024-03-30T13:10:57.678124+0000","last_active":"2024-03-30T13:10:58.672201+0000","last_peered":"2024-03-30T13:10:58.672201+0000","last_clean":"2024-03-30T13:10:58.672201+0000","last_became_active":"2024-03-30T13:10:57.677826+0000","last_became_peered":"2024-03-30T13:10:57.677826+0000","last_unstale":"2024-03-30T13:10:58.672201+0000","last_undegraded":"2024-03-30T13:10:58.672201+0000","last_fullsized":"2024-03-30T13:10:58.672201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:37.076824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678461+0000","last_change":"2024-03-30T13:10:56.678610+0000","last_active":"2024-03-30T13:10:58.678461+0000","last_peered":"2024-03-30T13:10:58.678461+0000","last_clean":"2024-03-30T13:10:58.678461+0000","last_became_active":"2024-03-30T13:10:56.678357+0000","last_became_peered":"2024-03-30T13:10:56.678357+0000","last_unstale":"2024-03-30T13:10:58.678461+0000","last_undegraded":"2024-03-30T13:10:58.678461+0000","last_fullsized":"2024-03-30T13:10:58.678461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:40.674849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676600+0000","last_change":"2024-03-30T13:10:57.687196+0000","last_active":"2024-03-30T13:10:58.676600+0000","last_peered":"2024-03-30T13:10:58.676600+0000","last_clean":"2024-03-30T13:10:58.676600+0000","last_became_active":"2024-03-30T13:10:57.686683+0000","last_became_peered":"2024-03-30T13:10:57.686683+0000","last_unstale":"2024-03-30T13:10:58.676600+0000","last_undegraded":"2024-03-30T13:10:58.676600+0000","last_fullsized":"2024-03-30T13:10:58.676600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:44:15.354797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T13:10:56.683925+0000","last_change":"2024-03-30T13:10:56.682787+0000","last_active":"2024-03-30T13:10:56.683925+0000","last_peered":"2024-03-30T13:10:56.683925+0000","last_clean":"2024-03-30T13:10:56.683925+0000","last_became_active":"2024-03-30T13:10:56.682536+0000","last_became_peered":"2024-03-30T13:10:56.682536+0000","last_unstale":"2024-03-30T13:10:56.683925+0000","last_undegraded":"2024-03-30T13:10:56.683925+0000","last_fullsized":"2024-03-30T13:10:56.683925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:30:59.057923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033243+0000","last_change":"2024-03-30T13:10:57.686653+0000","last_active":"2024-03-30T13:10:59.033243+0000","last_peered":"2024-03-30T13:10:59.033243+0000","last_clean":"2024-03-30T13:10:59.033243+0000","last_became_active":"2024-03-30T13:10:57.686380+0000","last_became_peered":"2024-03-30T13:10:57.686380+0000","last_unstale":"2024-03-30T13:10:59.033243+0000","last_undegraded":"2024-03-30T13:10:59.033243+0000","last_fullsized":"2024-03-30T13:10:59.033243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:13:33.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680418+0000","last_change":"2024-03-30T13:10:56.682042+0000","last_active":"2024-03-30T13:10:58.680418+0000","last_peered":"2024-03-30T13:10:58.680418+0000","last_clean":"2024-03-30T13:10:58.680418+0000","last_became_active":"2024-03-30T13:10:56.681748+0000","last_became_peered":"2024-03-30T13:10:56.681748+0000","last_unstale":"2024-03-30T13:10:58.680418+0000","last_undegraded":"2024-03-30T13:10:58.680418+0000","last_fullsized":"2024-03-30T13:10:58.680418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:55:55.240936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672322+0000","last_change":"2024-03-30T13:10:57.676679+0000","last_active":"2024-03-30T13:10:58.672322+0000","last_peered":"2024-03-30T13:10:58.672322+0000","last_clean":"2024-03-30T13:10:58.672322+0000","last_became_active":"2024-03-30T13:10:57.675136+0000","last_became_peered":"2024-03-30T13:10:57.675136+0000","last_unstale":"2024-03-30T13:10:58.672322+0000","last_undegraded":"2024-03-30T13:10:58.672322+0000","last_fullsized":"2024-03-30T13:10:58.672322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:01:31.125961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745366+0000","last_change":"2024-03-30T13:10:56.682275+0000","last_active":"2024-03-30T13:10:59.745366+0000","last_peered":"2024-03-30T13:10:59.745366+0000","last_clean":"2024-03-30T13:10:59.745366+0000","last_became_active":"2024-03-30T13:10:56.681990+0000","last_became_peered":"2024-03-30T13:10:56.681990+0000","last_unstale":"2024-03-30T13:10:59.745366+0000","last_undegraded":"2024-03-30T13:10:59.745366+0000","last_fullsized":"2024-03-30T13:10:59.745366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745290+0000","last_change":"2024-03-30T13:10:57.692363+0000","last_active":"2024-03-30T13:10:59.745290+0000","last_peered":"2024-03-30T13:10:59.745290+0000","last_clean":"2024-03-30T13:10:59.745290+0000","last_became_active":"2024-03-30T13:10:57.692063+0000","last_became_peered":"2024-03-30T13:10:57.692063+0000","last_unstale":"2024-03-30T13:10:59.745290+0000","last_undegraded":"2024-03-30T13:10:59.745290+0000","last_fullsized":"2024-03-30T13:10:59.745290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745367+0000","last_change":"2024-03-30T13:10:56.680716+0000","last_active":"2024-03-30T13:10:59.745367+0000","last_peered":"2024-03-30T13:10:59.745367+0000","last_clean":"2024-03-30T13:10:59.745367+0000","last_became_active":"2024-03-30T13:10:56.680446+0000","last_became_peered":"2024-03-30T13:10:56.680446+0000","last_unstale":"2024-03-30T13:10:59.745367+0000","last_undegraded":"2024-03-30T13:10:59.745367+0000","last_fullsized":"2024-03-30T13:10:59.745367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676703+0000","last_change":"2024-03-30T13:10:57.684334+0000","last_active":"2024-03-30T13:10:58.676703+0000","last_peered":"2024-03-30T13:10:58.676703+0000","last_clean":"2024-03-30T13:10:58.676703+0000","last_became_active":"2024-03-30T13:10:57.684037+0000","last_became_peered":"2024-03-30T13:10:57.684037+0000","last_unstale":"2024-03-30T13:10:58.676703+0000","last_undegraded":"2024-03-30T13:10:58.676703+0000","last_fullsized":"2024-03-30T13:10:58.676703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:07:21.631648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745812+0000","last_change":"2024-03-30T13:10:56.676202+0000","last_active":"2024-03-30T13:10:59.745812+0000","last_peered":"2024-03-30T13:10:59.745812+0000","last_clean":"2024-03-30T13:10:59.745812+0000","last_became_active":"2024-03-30T13:10:56.675907+0000","last_became_peered":"2024-03-30T13:10:56.675907+0000","last_unstale":"2024-03-30T13:10:59.745812+0000","last_undegraded":"2024-03-30T13:10:59.745812+0000","last_fullsized":"2024-03-30T13:10:59.745812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:58.975900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672618+0000","last_change":"2024-03-30T13:10:57.673049+0000","last_active":"2024-03-30T13:10:58.672618+0000","last_peered":"2024-03-30T13:10:58.672618+0000","last_clean":"2024-03-30T13:10:58.672618+0000","last_became_active":"2024-03-30T13:10:57.672738+0000","last_became_peered":"2024-03-30T13:10:57.672738+0000","last_unstale":"2024-03-30T13:10:58.672618+0000","last_undegraded":"2024-03-30T13:10:58.672618+0000","last_fullsized":"2024-03-30T13:10:58.672618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672426+0000","last_change":"2024-03-30T13:10:56.675234+0000","last_active":"2024-03-30T13:10:58.672426+0000","last_peered":"2024-03-30T13:10:58.672426+0000","last_clean":"2024-03-30T13:10:58.672426+0000","last_became_active":"2024-03-30T13:10:56.674965+0000","last_became_peered":"2024-03-30T13:10:56.674965+0000","last_unstale":"2024-03-30T13:10:58.672426+0000","last_undegraded":"2024-03-30T13:10:58.672426+0000","last_fullsized":"2024-03-30T13:10:58.672426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:35.628852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033158+0000","last_change":"2024-03-30T13:10:57.683624+0000","last_active":"2024-03-30T13:10:59.033158+0000","last_peered":"2024-03-30T13:10:59.033158+0000","last_clean":"2024-03-30T13:10:59.033158+0000","last_became_active":"2024-03-30T13:10:57.683346+0000","last_became_peered":"2024-03-30T13:10:57.683346+0000","last_unstale":"2024-03-30T13:10:59.033158+0000","last_undegraded":"2024-03-30T13:10:59.033158+0000","last_fullsized":"2024-03-30T13:10:59.033158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671837+0000","last_change":"2024-03-30T13:10:56.682711+0000","last_active":"2024-03-30T13:10:58.671837+0000","last_peered":"2024-03-30T13:10:58.671837+0000","last_clean":"2024-03-30T13:10:58.671837+0000","last_became_active":"2024-03-30T13:10:56.682415+0000","last_became_peered":"2024-03-30T13:10:56.682415+0000","last_unstale":"2024-03-30T13:10:58.671837+0000","last_undegraded":"2024-03-30T13:10:58.671837+0000","last_fullsized":"2024-03-30T13:10:58.671837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:10:34.096760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-03-30T13:10:56.674446+0000","last_change":"2024-03-30T13:10:56.674446+0000","last_active":"2024-03-30T13:10:56.651517+0000","last_peered":"2024-03-30T13:10:56.651517+0000","last_clean":"2024-03-30T13:10:56.651517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:10:56.674446+0000","last_undegraded":"2024-03-30T13:10:56.674446+0000","last_fullsized":"2024-03-30T13:10:56.674446+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033155+0000","last_change":"2024-03-30T13:10:56.681620+0000","last_active":"2024-03-30T13:10:59.033155+0000","last_peered":"2024-03-30T13:10:59.033155+0000","last_clean":"2024-03-30T13:10:59.033155+0000","last_became_active":"2024-03-30T13:10:56.681341+0000","last_became_peered":"2024-03-30T13:10:56.681341+0000","last_unstale":"2024-03-30T13:10:59.033155+0000","last_undegraded":"2024-03-30T13:10:59.033155+0000","last_fullsized":"2024-03-30T13:10:59.033155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:46:52.577909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032961+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:10:59.032961+0000","last_peered":"2024-03-30T13:10:59.032961+0000","last_clean":"2024-03-30T13:10:59.032961+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:10:59.032961+0000","last_undegraded":"2024-03-30T13:10:59.032961+0000","last_fullsized":"2024-03-30T13:10:59.032961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":199796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":199818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":782924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":178688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-30T13:11:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.677018+0000', 'last_change': '2024-03-30T13:10:56.677018+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.677018+0000', 'last_undegraded': '2024-03-30T13:10:56.677018+0000', 'last_fullsized': '2024-03-30T13:10:56.677018+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-30T13:11:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.676334+0000', 'last_change': '2024-03-30T13:10:56.676334+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.676334+0000', 'last_undegraded': '2024-03-30T13:10:56.676334+0000', 'last_fullsized': '2024-03-30T13:10:56.676334+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-30T13:11:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.673354+0000', 'last_change': '2024-03-30T13:10:56.673354+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.673354+0000', 'last_undegraded': '2024-03-30T13:10:56.673354+0000', 'last_fullsized': '2024-03-30T13:10:56.673354+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.676222+0000', 'last_change': '2024-03-30T13:10:56.676222+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.676222+0000', 'last_undegraded': '2024-03-30T13:10:56.676222+0000', 'last_fullsized': '2024-03-30T13:10:56.676222+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.676235+0000', 'last_change': '2024-03-30T13:10:56.676235+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.676235+0000', 'last_undegraded': '2024-03-30T13:10:56.676235+0000', 'last_fullsized': '2024-03-30T13:10:56.676235+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.672740+0000', 'last_change': '2024-03-30T13:10:56.672740+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.672740+0000', 'last_undegraded': '2024-03-30T13:10:56.672740+0000', 'last_fullsized': '2024-03-30T13:10:56.672740+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.675608+0000', 'last_change': '2024-03-30T13:10:56.675608+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.675608+0000', 'last_undegraded': '2024-03-30T13:10:56.675608+0000', 'last_fullsized': '2024-03-30T13:10:56.675608+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.675811+0000', 'last_change': '2024-03-30T13:10:56.675811+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.675811+0000', 'last_undegraded': '2024-03-30T13:10:56.675811+0000', 'last_fullsized': '2024-03-30T13:10:56.675811+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-30T13:11:02.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.672924+0000', 'last_change': '2024-03-30T13:10:56.672924+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.672924+0000', 'last_undegraded': '2024-03-30T13:10:56.672924+0000', 'last_fullsized': '2024-03-30T13:10:56.672924+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-30T13:11:02.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.672390+0000', 'last_change': '2024-03-30T13:10:56.672390+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.672390+0000', 'last_undegraded': '2024-03-30T13:10:56.672390+0000', 'last_fullsized': '2024-03-30T13:10:56.672390+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-30T13:11:02.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:10:56.674446+0000', 'last_change': '2024-03-30T13:10:56.674446+0000', 'last_active': '2024-03-30T13:10:56.651517+0000', 'last_peered': '2024-03-30T13:10:56.651517+0000', 'last_clean': '2024-03-30T13:10:56.651517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:10:56.674446+0000', 'last_undegraded': '2024-03-30T13:10:56.674446+0000', 'last_fullsized': '2024-03-30T13:10:56.674446+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'last_clean_scrub_stamp': '2024-03-30T13:10:56.651517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T13:11:02.480 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-30T13:11:02.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.617+0000 7fb66aaa5640 1 Processor -- start 2024-03-30T13:11:02.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.617+0000 7fb66aaa5640 1 -- start start 2024-03-30T13:11:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb66aaa5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb66aaa5640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6640d36f0 con 0x7fb6640d48c0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb66aaa5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6640d39d0 con 0x7fb6640d30d0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb66aaa5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb6640d3cb0 con 0x7fb66415e950 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb663fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb6640d48c0 0x7fb6640d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60696/0 (socket says 172.21.15.66:60696) 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb663fff640 1 -- 172.21.15.66:0/2990285314 learned_addr learned my addr 172.21.15.66:0/2990285314 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb668841640 1 --2- 172.21.15.66:0/2990285314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46040205 0 0) 0x7fb6640d36f0 con 0x7fb6640d48c0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.618+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654003520 con 0x7fb6640d48c0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939856241 0 0) 0x7fb6640d39d0 con 0x7fb6640d30d0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6640d36f0 con 0x7fb6640d30d0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4072418573 0 0) 0x7fb654003520 con 0x7fb6640d48c0 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb668841640 1 -- 172.21.15.66:0/2990285314 >> v1:172.21.15.117:6789/0 conn(0x7fb6640d30d0 legacy=0x7fb6640b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb668841640 1 -- 172.21.15.66:0/2990285314 >> v1:172.21.15.66:6789/0 conn(0x7fb6640d48c0 legacy=0x7fb6640d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb668841640 1 -- 172.21.15.66:0/2990285314 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6640b7ac0 con 0x7fb66415e950 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.619+0000 7fb668841640 1 --2- 172.21.15.66:0/2990285314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640d4380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097d0 tx=0x7fb65802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85a06b2fb76e6be server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650001670 con 0x7fb66415e950 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658037070 con 0x7fb66415e950 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb6637fe640 1 -- 172.21.15.66:0/2990285314 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580317a0 con 0x7fb66415e950 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 -- 172.21.15.66:0/2990285314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 msgr2=0x7fb6640d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/2990285314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640d4380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097d0 tx=0x7fb65802b8a0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 -- 172.21.15.66:0/2990285314 shutdown_connections 2024-03-30T13:11:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/2990285314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640d4380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 -- 172.21.15.66:0/2990285314 >> 172.21.15.66:0/2990285314 conn(0x7fb6640676e0 msgr2=0x7fb664067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 -- 172.21.15.66:0/2990285314 shutdown_connections 2024-03-30T13:11:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.620+0000 7fb66aaa5640 1 -- 172.21.15.66:0/2990285314 wait complete. 2024-03-30T13:11:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.621+0000 7fb66aaa5640 1 Processor -- start 2024-03-30T13:11:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.621+0000 7fb66aaa5640 1 -- start start 2024-03-30T13:11:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb66aaa5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6640d30d0 0x7fb664161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb66aaa5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 0x7fb6640cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb66aaa5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb66aaa5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb66415ed30 con 0x7fb6640d48c0 2024-03-30T13:11:02.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb66aaa5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6640d4d00 con 0x7fb6640d30d0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb66aaa5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb66415eeb0 con 0x7fb66415e950 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb663fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 0x7fb6640cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb663fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 0x7fb6640cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39556/0 (socket says 172.21.15.66:39556) 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb663fff640 1 -- 172.21.15.66:0/3497897526 learned_addr learned my addr 172.21.15.66:0/3497897526 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb668841640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6640d30d0 0x7fb664161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.622+0000 7fb669042640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb663fff640 1 -- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 msgr2=0x7fb6640ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb663fff640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb663fff640 1 -- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6640d30d0 msgr2=0x7fb664161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb663fff640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6640d30d0 0x7fb664161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb663fff640 1 -- 172.21.15.66:0/3497897526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb664162190 con 0x7fb6640d48c0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb663fff640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 0x7fb6640cca60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c780 tx=0x7fb65000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650017070 con 0x7fb6640d48c0 2024-03-30T13:11:02.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb650005af0 con 0x7fb6640d48c0 2024-03-30T13:11:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6641583c0 con 0x7fb6640d48c0 2024-03-30T13:11:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650007070 con 0x7fb6640d48c0 2024-03-30T13:11:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.623+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb664158980 con 0x7fb6640d48c0 2024-03-30T13:11:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.625+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb65001a070 con 0x7fb6640d48c0 2024-03-30T13:11:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.625+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb664075a10 con 0x7fb6640d48c0 2024-03-30T13:11:02.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.625+0000 7fb6617fa640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb64806c530 0x7fb64806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.625+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fb65008ace0 con 0x7fb6640d48c0 2024-03-30T13:11:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.625+0000 7fb668841640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb64806c530 0x7fb64806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.626+0000 7fb668841640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb64806c530 0x7fb64806e9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb658036000 tx=0x7fb658008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.629+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb650058bb0 con 0x7fb6640d48c0 2024-03-30T13:11:02.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.928+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb664067e30 con 0x7fb6640d48c0 2024-03-30T13:11:02.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.929+0000 7fb6617fa640 1 -- 172.21.15.66:0/3497897526 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1099 (secure 0 0 0) 0x7fb65005ceb0 con 0x7fb6640d48c0 2024-03-30T13:11:02.933 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:02.933 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":104,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1711804169,"num_in_osds":8,"osd_in_since":1711804160,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":86},{"state_name":"creating+peering","count":11}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592666,"bytes_used":226152448,"bytes_avail":772867960832,"bytes_total":773094113280,"inactive_pgs_ratio":0.11340206116437912,"write_bytes_sec":2383,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5382}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-30T13:10:46.808155+0000","services":{}},"progress_events":{}} 2024-03-30T13:11:02.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb64806c530 msgr2=0x7fb64806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb64806c530 0x7fb64806e9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb658036000 tx=0x7fb658008030 comp rx=0 tx=0).stop 2024-03-30T13:11:02.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 msgr2=0x7fb6640cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 0x7fb6640cca60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c780 tx=0x7fb65000cc40 comp rx=0 tx=0).stop 2024-03-30T13:11:02.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 shutdown_connections 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb64806c530 0x7fb64806e9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb66415e950 0x7fb6640ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6640d48c0 0x7fb6640cca60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 --2- 172.21.15.66:0/3497897526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6640d30d0 0x7fb664161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 >> 172.21.15.66:0/3497897526 conn(0x7fb6640676e0 msgr2=0x7fb6640b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 shutdown_connections 2024-03-30T13:11:02.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:02.931+0000 7fb66aaa5640 1 -- 172.21.15.66:0/3497897526 wait complete. 2024-03-30T13:11:02.946 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-30T13:11:02.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:03.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.050+0000 7f21c14eb640 1 Processor -- start 2024-03-30T13:11:03.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.050+0000 7f21c14eb640 1 -- start start 2024-03-30T13:11:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21c14eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21c14eb640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21bc0cedb0 0x7f21bc0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21c14eb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21bc0cf970 con 0x7f21bc0b63e0 2024-03-30T13:11:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21c14eb640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f21bc0b3c80 con 0x7f21bc0cedb0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21c14eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f21bc0d2560 con 0x7f21bc15e9c0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21ba7fc640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21bc0cedb0 0x7f21bc0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.051+0000 7f21ba7fc640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21bc0cedb0 0x7f21bc0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39566/0 (socket says 172.21.15.66:39566) 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.052+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1981393287 learned_addr learned my addr 172.21.15.66:0/1981393287 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.052+0000 7f21baffd640 1 --2- 172.21.15.66:0/1981393287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.052+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1981393287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 msgr2=0x7f21bc0b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.052+0000 7f21ba7fc640 1 --2- 172.21.15.66:0/1981393287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.052+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1981393287 >> v1:172.21.15.117:6789/0 conn(0x7f21bc0b63e0 legacy=0x7f21bc0d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.052+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1981393287 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc0cfc50 con 0x7f21bc0cedb0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21ba7fc640 1 --2- 172.21.15.66:0/1981393287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cf1f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009870 tx=0x7f21ac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e701f9a44954d47f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21b9ffb640 1 -- 172.21.15.66:0/1981393287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac034070 con 0x7f21bc0cedb0 2024-03-30T13:11:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21b9ffb640 1 -- 172.21.15.66:0/1981393287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac030840 con 0x7f21bc0cedb0 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21b9ffb640 1 -- 172.21.15.66:0/1981393287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac030c10 con 0x7f21bc0cedb0 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21c14eb640 1 -- 172.21.15.66:0/1981393287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 msgr2=0x7f21bc0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1981393287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cf1f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009870 tx=0x7f21ac02b980 comp rx=0 tx=0).stop 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21c14eb640 1 -- 172.21.15.66:0/1981393287 shutdown_connections 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1981393287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cf1f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1981393287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.053+0000 7f21c14eb640 1 -- 172.21.15.66:0/1981393287 >> 172.21.15.66:0/1981393287 conn(0x7f21bc0676e0 msgr2=0x7f21bc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.054+0000 7f21c14eb640 1 -- 172.21.15.66:0/1981393287 shutdown_connections 2024-03-30T13:11:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.054+0000 7f21c14eb640 1 -- 172.21.15.66:0/1981393287 wait complete. 2024-03-30T13:11:03.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.055+0000 7f21c14eb640 1 Processor -- start 2024-03-30T13:11:03.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.055+0000 7f21c14eb640 1 -- start start 2024-03-30T13:11:03.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21bc0b63e0 0x7f21bc157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc0d2a80 con 0x7f21bc0cedb0 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc0cf730 con 0x7f21bc0b63e0 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f21bc0b7d80 con 0x7f21bc15e9c0 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39570/0 (socket says 172.21.15.66:39570) 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1566532818 learned_addr learned my addr 172.21.15.66:0/1566532818 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21baffd640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21bc0b63e0 0x7f21bc157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21bb7fe640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 msgr2=0x7f21bc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21bc0b63e0 msgr2=0x7f21bc157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21bc0b63e0 0x7f21bc157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 -- 172.21.15.66:0/1566532818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc158d80 con 0x7f21bc0cedb0 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21baffd640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21bc0b63e0 0x7f21bc157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21ba7fc640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cd510 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac048070 con 0x7f21bc0cedb0 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac030640 con 0x7f21bc0cedb0 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc1f8040 con 0x7f21bc0cedb0 2024-03-30T13:11:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.056+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac034040 con 0x7f21bc0cedb0 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.057+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1f8600 con 0x7f21bc0cedb0 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.058+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21bc075a10 con 0x7f21bc0cedb0 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.058+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21ac037070 con 0x7f21bc0cedb0 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.058+0000 7f21a37fe640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21a806c530 0x7f21a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.058+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f21ac0ad760 con 0x7f21bc0cedb0 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.058+0000 7f21baffd640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21a806c530 0x7f21a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.059+0000 7f21baffd640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21a806c530 0x7f21a806e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009a20 tx=0x7f21b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.063+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f21ac07bc90 con 0x7f21bc0cedb0 2024-03-30T13:11:03.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.243+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21bc1577c0 con 0x7f21a806c530 2024-03-30T13:11:03.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.249+0000 7f21a37fe640 1 -- 172.21.15.66:0/1566532818 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251958 (secure 0 0 0) 0x7f21bc1577c0 con 0x7f21a806c530 2024-03-30T13:11:03.256 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:03.258 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:11:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21a806c530 msgr2=0x7f21a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21a806c530 0x7f21a806e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009a20 tx=0x7f21b0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 msgr2=0x7f21bc0cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cd510 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac004730 comp rx=0 tx=0).stop 2024-03-30T13:11:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 shutdown_connections 2024-03-30T13:11:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21a806c530 0x7f21a806e9f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21bc15e9c0 0x7f21bc0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21bc0cedb0 0x7f21bc0cd510 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 --2- 172.21.15.66:0/1566532818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21bc0b63e0 0x7f21bc157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 >> 172.21.15.66:0/1566532818 conn(0x7f21bc0676e0 msgr2=0x7f21bc0b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 shutdown_connections 2024-03-30T13:11:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:03.252+0000 7f21c14eb640 1 -- 172.21.15.66:0/1566532818 wait complete. 2024-03-30T13:11:03.270 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-03-30T13:11:02.810744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220900,"kb_used_data":4728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753820,"statfs":{"total":773094113280,"available":772867911680,"internally_reserved":0,"allocated":4841472,"data_stored":2752500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.148667"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672404+0000","last_change":"2024-03-30T13:10:57.684501+0000","last_active":"2024-03-30T13:10:58.672404+0000","last_peered":"2024-03-30T13:10:58.672404+0000","last_clean":"2024-03-30T13:10:58.672404+0000","last_became_active":"2024-03-30T13:10:57.684187+0000","last_became_peered":"2024-03-30T13:10:57.684187+0000","last_unstale":"2024-03-30T13:10:58.672404+0000","last_undegraded":"2024-03-30T13:10:58.672404+0000","last_fullsized":"2024-03-30T13:10:58.672404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:07.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672159+0000","last_change":"2024-03-30T13:10:57.681207+0000","last_active":"2024-03-30T13:10:58.672159+0000","last_peered":"2024-03-30T13:10:58.672159+0000","last_clean":"2024-03-30T13:10:58.672159+0000","last_became_active":"2024-03-30T13:10:57.680890+0000","last_became_peered":"2024-03-30T13:10:57.680890+0000","last_unstale":"2024-03-30T13:10:58.672159+0000","last_undegraded":"2024-03-30T13:10:58.672159+0000","last_fullsized":"2024-03-30T13:10:58.672159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:30:42.096995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745282+0000","last_change":"2024-03-30T13:10:57.695761+0000","last_active":"2024-03-30T13:10:59.745282+0000","last_peered":"2024-03-30T13:10:59.745282+0000","last_clean":"2024-03-30T13:10:59.745282+0000","last_became_active":"2024-03-30T13:10:57.695488+0000","last_became_peered":"2024-03-30T13:10:57.695488+0000","last_unstale":"2024-03-30T13:10:59.745282+0000","last_undegraded":"2024-03-30T13:10:59.745282+0000","last_fullsized":"2024-03-30T13:10:59.745282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678726+0000","last_change":"2024-03-30T13:10:57.682241+0000","last_active":"2024-03-30T13:10:58.678726+0000","last_peered":"2024-03-30T13:10:58.678726+0000","last_clean":"2024-03-30T13:10:58.678726+0000","last_became_active":"2024-03-30T13:10:57.681383+0000","last_became_peered":"2024-03-30T13:10:57.681383+0000","last_unstale":"2024-03-30T13:10:58.678726+0000","last_undegraded":"2024-03-30T13:10:58.678726+0000","last_fullsized":"2024-03-30T13:10:58.678726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:43:40.092786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678584+0000","last_change":"2024-03-30T13:10:57.680923+0000","last_active":"2024-03-30T13:10:58.678584+0000","last_peered":"2024-03-30T13:10:58.678584+0000","last_clean":"2024-03-30T13:10:58.678584+0000","last_became_active":"2024-03-30T13:10:57.680625+0000","last_became_peered":"2024-03-30T13:10:57.680625+0000","last_unstale":"2024-03-30T13:10:58.678584+0000","last_undegraded":"2024-03-30T13:10:58.678584+0000","last_fullsized":"2024-03-30T13:10:58.678584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:27:38.309021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676495+0000","last_change":"2024-03-30T13:10:57.687677+0000","last_active":"2024-03-30T13:10:58.676495+0000","last_peered":"2024-03-30T13:10:58.676495+0000","last_clean":"2024-03-30T13:10:58.676495+0000","last_became_active":"2024-03-30T13:10:57.687348+0000","last_became_peered":"2024-03-30T13:10:57.687348+0000","last_unstale":"2024-03-30T13:10:58.676495+0000","last_undegraded":"2024-03-30T13:10:58.676495+0000","last_fullsized":"2024-03-30T13:10:58.676495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:33:20.876411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680914+0000","last_change":"2024-03-30T13:10:57.680450+0000","last_active":"2024-03-30T13:10:58.680914+0000","last_peered":"2024-03-30T13:10:58.680914+0000","last_clean":"2024-03-30T13:10:58.680914+0000","last_became_active":"2024-03-30T13:10:57.680147+0000","last_became_peered":"2024-03-30T13:10:57.680147+0000","last_unstale":"2024-03-30T13:10:58.680914+0000","last_undegraded":"2024-03-30T13:10:58.680914+0000","last_fullsized":"2024-03-30T13:10:58.680914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:56.711272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671840+0000","last_change":"2024-03-30T13:10:57.679497+0000","last_active":"2024-03-30T13:10:58.671840+0000","last_peered":"2024-03-30T13:10:58.671840+0000","last_clean":"2024-03-30T13:10:58.671840+0000","last_became_active":"2024-03-30T13:10:57.679201+0000","last_became_peered":"2024-03-30T13:10:57.679201+0000","last_unstale":"2024-03-30T13:10:58.671840+0000","last_undegraded":"2024-03-30T13:10:58.671840+0000","last_fullsized":"2024-03-30T13:10:58.671840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:04:54.328931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745602+0000","last_change":"2024-03-30T13:10:57.690985+0000","last_active":"2024-03-30T13:10:59.745602+0000","last_peered":"2024-03-30T13:10:59.745602+0000","last_clean":"2024-03-30T13:10:59.745602+0000","last_became_active":"2024-03-30T13:10:57.690657+0000","last_became_peered":"2024-03-30T13:10:57.690657+0000","last_unstale":"2024-03-30T13:10:59.745602+0000","last_undegraded":"2024-03-30T13:10:59.745602+0000","last_fullsized":"2024-03-30T13:10:59.745602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681200+0000","last_change":"2024-03-30T13:10:57.680642+0000","last_active":"2024-03-30T13:10:58.681200+0000","last_peered":"2024-03-30T13:10:58.681200+0000","last_clean":"2024-03-30T13:10:58.681200+0000","last_became_active":"2024-03-30T13:10:57.680287+0000","last_became_peered":"2024-03-30T13:10:57.680287+0000","last_unstale":"2024-03-30T13:10:58.681200+0000","last_undegraded":"2024-03-30T13:10:58.681200+0000","last_fullsized":"2024-03-30T13:10:58.681200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:34:29.439469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681467+0000","last_change":"2024-03-30T13:10:57.681398+0000","last_active":"2024-03-30T13:10:58.681467+0000","last_peered":"2024-03-30T13:10:58.681467+0000","last_clean":"2024-03-30T13:10:58.681467+0000","last_became_active":"2024-03-30T13:10:57.681055+0000","last_became_peered":"2024-03-30T13:10:57.681055+0000","last_unstale":"2024-03-30T13:10:58.681467+0000","last_undegraded":"2024-03-30T13:10:58.681467+0000","last_fullsized":"2024-03-30T13:10:58.681467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:12.152970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745292+0000","last_change":"2024-03-30T13:10:57.691635+0000","last_active":"2024-03-30T13:10:59.745292+0000","last_peered":"2024-03-30T13:10:59.745292+0000","last_clean":"2024-03-30T13:10:59.745292+0000","last_became_active":"2024-03-30T13:10:57.691340+0000","last_became_peered":"2024-03-30T13:10:57.691340+0000","last_unstale":"2024-03-30T13:10:59.745292+0000","last_undegraded":"2024-03-30T13:10:59.745292+0000","last_fullsized":"2024-03-30T13:10:59.745292+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680976+0000","last_change":"2024-03-30T13:10:57.680573+0000","last_active":"2024-03-30T13:10:58.680976+0000","last_peered":"2024-03-30T13:10:58.680976+0000","last_clean":"2024-03-30T13:10:58.680976+0000","last_became_active":"2024-03-30T13:10:57.680250+0000","last_became_peered":"2024-03-30T13:10:57.680250+0000","last_unstale":"2024-03-30T13:10:58.680976+0000","last_undegraded":"2024-03-30T13:10:58.680976+0000","last_fullsized":"2024-03-30T13:10:58.680976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:22:10.463077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672408+0000","last_change":"2024-03-30T13:10:57.686031+0000","last_active":"2024-03-30T13:10:58.672408+0000","last_peered":"2024-03-30T13:10:58.672408+0000","last_clean":"2024-03-30T13:10:58.672408+0000","last_became_active":"2024-03-30T13:10:57.685814+0000","last_became_peered":"2024-03-30T13:10:57.685814+0000","last_unstale":"2024-03-30T13:10:58.672408+0000","last_undegraded":"2024-03-30T13:10:58.672408+0000","last_fullsized":"2024-03-30T13:10:58.672408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680965+0000","last_change":"2024-03-30T13:10:57.691786+0000","last_active":"2024-03-30T13:10:58.680965+0000","last_peered":"2024-03-30T13:10:58.680965+0000","last_clean":"2024-03-30T13:10:58.680965+0000","last_became_active":"2024-03-30T13:10:57.690961+0000","last_became_peered":"2024-03-30T13:10:57.690961+0000","last_unstale":"2024-03-30T13:10:58.680965+0000","last_undegraded":"2024-03-30T13:10:58.680965+0000","last_fullsized":"2024-03-30T13:10:58.680965+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:47.380589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032816+0000","last_change":"2024-03-30T13:10:57.681811+0000","last_active":"2024-03-30T13:10:59.032816+0000","last_peered":"2024-03-30T13:10:59.032816+0000","last_clean":"2024-03-30T13:10:59.032816+0000","last_became_active":"2024-03-30T13:10:57.681471+0000","last_became_peered":"2024-03-30T13:10:57.681471+0000","last_unstale":"2024-03-30T13:10:59.032816+0000","last_undegraded":"2024-03-30T13:10:59.032816+0000","last_fullsized":"2024-03-30T13:10:59.032816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033359+0000","last_change":"2024-03-30T13:10:57.687549+0000","last_active":"2024-03-30T13:10:59.033359+0000","last_peered":"2024-03-30T13:10:59.033359+0000","last_clean":"2024-03-30T13:10:59.033359+0000","last_became_active":"2024-03-30T13:10:57.687268+0000","last_became_peered":"2024-03-30T13:10:57.687268+0000","last_unstale":"2024-03-30T13:10:59.033359+0000","last_undegraded":"2024-03-30T13:10:59.033359+0000","last_fullsized":"2024-03-30T13:10:59.033359+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:35.714074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746089+0000","last_change":"2024-03-30T13:10:57.683841+0000","last_active":"2024-03-30T13:10:59.746089+0000","last_peered":"2024-03-30T13:10:59.746089+0000","last_clean":"2024-03-30T13:10:59.746089+0000","last_became_active":"2024-03-30T13:10:57.683568+0000","last_became_peered":"2024-03-30T13:10:57.683568+0000","last_unstale":"2024-03-30T13:10:59.746089+0000","last_undegraded":"2024-03-30T13:10:59.746089+0000","last_fullsized":"2024-03-30T13:10:59.746089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:51:46.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678919+0000","last_change":"2024-03-30T13:10:57.688521+0000","last_active":"2024-03-30T13:10:58.678919+0000","last_peered":"2024-03-30T13:10:58.678919+0000","last_clean":"2024-03-30T13:10:58.678919+0000","last_became_active":"2024-03-30T13:10:57.688188+0000","last_became_peered":"2024-03-30T13:10:57.688188+0000","last_unstale":"2024-03-30T13:10:58.678919+0000","last_undegraded":"2024-03-30T13:10:58.678919+0000","last_fullsized":"2024-03-30T13:10:58.678919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:47.722209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745911+0000","last_change":"2024-03-30T13:10:57.682779+0000","last_active":"2024-03-30T13:10:59.745911+0000","last_peered":"2024-03-30T13:10:59.745911+0000","last_clean":"2024-03-30T13:10:59.745911+0000","last_became_active":"2024-03-30T13:10:57.682444+0000","last_became_peered":"2024-03-30T13:10:57.682444+0000","last_unstale":"2024-03-30T13:10:59.745911+0000","last_undegraded":"2024-03-30T13:10:59.745911+0000","last_fullsized":"2024-03-30T13:10:59.745911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:02:29.613267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681089+0000","last_change":"2024-03-30T13:10:57.692912+0000","last_active":"2024-03-30T13:10:58.681089+0000","last_peered":"2024-03-30T13:10:58.681089+0000","last_clean":"2024-03-30T13:10:58.681089+0000","last_became_active":"2024-03-30T13:10:57.692639+0000","last_became_peered":"2024-03-30T13:10:57.692639+0000","last_unstale":"2024-03-30T13:10:58.681089+0000","last_undegraded":"2024-03-30T13:10:58.681089+0000","last_fullsized":"2024-03-30T13:10:58.681089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:01:30.821857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672797+0000","last_change":"2024-03-30T13:10:57.692104+0000","last_active":"2024-03-30T13:10:58.672797+0000","last_peered":"2024-03-30T13:10:58.672797+0000","last_clean":"2024-03-30T13:10:58.672797+0000","last_became_active":"2024-03-30T13:10:57.691816+0000","last_became_peered":"2024-03-30T13:10:57.691816+0000","last_unstale":"2024-03-30T13:10:58.672797+0000","last_undegraded":"2024-03-30T13:10:58.672797+0000","last_fullsized":"2024-03-30T13:10:58.672797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032948+0000","last_change":"2024-03-30T13:10:57.691507+0000","last_active":"2024-03-30T13:10:59.032948+0000","last_peered":"2024-03-30T13:10:59.032948+0000","last_clean":"2024-03-30T13:10:59.032948+0000","last_became_active":"2024-03-30T13:10:57.691248+0000","last_became_peered":"2024-03-30T13:10:57.691248+0000","last_unstale":"2024-03-30T13:10:59.032948+0000","last_undegraded":"2024-03-30T13:10:59.032948+0000","last_fullsized":"2024-03-30T13:10:59.032948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678498+0000","last_change":"2024-03-30T13:10:57.685316+0000","last_active":"2024-03-30T13:10:58.678498+0000","last_peered":"2024-03-30T13:10:58.678498+0000","last_clean":"2024-03-30T13:10:58.678498+0000","last_became_active":"2024-03-30T13:10:57.685053+0000","last_became_peered":"2024-03-30T13:10:57.685053+0000","last_unstale":"2024-03-30T13:10:58.678498+0000","last_undegraded":"2024-03-30T13:10:58.678498+0000","last_fullsized":"2024-03-30T13:10:58.678498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:43:19.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746090+0000","last_change":"2024-03-30T13:10:57.684225+0000","last_active":"2024-03-30T13:10:59.746090+0000","last_peered":"2024-03-30T13:10:59.746090+0000","last_clean":"2024-03-30T13:10:59.746090+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:59.746090+0000","last_undegraded":"2024-03-30T13:10:59.746090+0000","last_fullsized":"2024-03-30T13:10:59.746090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:44:06.219419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676846+0000","last_change":"2024-03-30T13:10:57.687484+0000","last_active":"2024-03-30T13:10:58.676846+0000","last_peered":"2024-03-30T13:10:58.676846+0000","last_clean":"2024-03-30T13:10:58.676846+0000","last_became_active":"2024-03-30T13:10:57.687191+0000","last_became_peered":"2024-03-30T13:10:57.687191+0000","last_unstale":"2024-03-30T13:10:58.676846+0000","last_undegraded":"2024-03-30T13:10:58.676846+0000","last_fullsized":"2024-03-30T13:10:58.676846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:49:47.246424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746018+0000","last_change":"2024-03-30T13:10:57.687817+0000","last_active":"2024-03-30T13:10:59.746018+0000","last_peered":"2024-03-30T13:10:59.746018+0000","last_clean":"2024-03-30T13:10:59.746018+0000","last_became_active":"2024-03-30T13:10:57.687038+0000","last_became_peered":"2024-03-30T13:10:57.687038+0000","last_unstale":"2024-03-30T13:10:59.746018+0000","last_undegraded":"2024-03-30T13:10:59.746018+0000","last_fullsized":"2024-03-30T13:10:59.746018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:03:06.653896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681394+0000","last_change":"2024-03-30T13:10:57.681266+0000","last_active":"2024-03-30T13:10:58.681394+0000","last_peered":"2024-03-30T13:10:58.681394+0000","last_clean":"2024-03-30T13:10:58.681394+0000","last_became_active":"2024-03-30T13:10:57.680923+0000","last_became_peered":"2024-03-30T13:10:57.680923+0000","last_unstale":"2024-03-30T13:10:58.681394+0000","last_undegraded":"2024-03-30T13:10:58.681394+0000","last_fullsized":"2024-03-30T13:10:58.681394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:43.701244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745905+0000","last_change":"2024-03-30T13:10:57.690320+0000","last_active":"2024-03-30T13:10:59.745905+0000","last_peered":"2024-03-30T13:10:59.745905+0000","last_clean":"2024-03-30T13:10:59.745905+0000","last_became_active":"2024-03-30T13:10:57.689740+0000","last_became_peered":"2024-03-30T13:10:57.689740+0000","last_unstale":"2024-03-30T13:10:59.745905+0000","last_undegraded":"2024-03-30T13:10:59.745905+0000","last_fullsized":"2024-03-30T13:10:59.745905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:55.823728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745918+0000","last_change":"2024-03-30T13:10:57.683564+0000","last_active":"2024-03-30T13:10:59.745918+0000","last_peered":"2024-03-30T13:10:59.745918+0000","last_clean":"2024-03-30T13:10:59.745918+0000","last_became_active":"2024-03-30T13:10:57.683225+0000","last_became_peered":"2024-03-30T13:10:57.683225+0000","last_unstale":"2024-03-30T13:10:59.745918+0000","last_undegraded":"2024-03-30T13:10:59.745918+0000","last_fullsized":"2024-03-30T13:10:59.745918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:36.633906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032986+0000","last_change":"2024-03-30T13:10:57.692664+0000","last_active":"2024-03-30T13:10:59.032986+0000","last_peered":"2024-03-30T13:10:59.032986+0000","last_clean":"2024-03-30T13:10:59.032986+0000","last_became_active":"2024-03-30T13:10:57.692376+0000","last_became_peered":"2024-03-30T13:10:57.692376+0000","last_unstale":"2024-03-30T13:10:59.032986+0000","last_undegraded":"2024-03-30T13:10:59.032986+0000","last_fullsized":"2024-03-30T13:10:59.032986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676356+0000","last_change":"2024-03-30T13:10:57.688812+0000","last_active":"2024-03-30T13:10:58.676356+0000","last_peered":"2024-03-30T13:10:58.676356+0000","last_clean":"2024-03-30T13:10:58.676356+0000","last_became_active":"2024-03-30T13:10:57.688187+0000","last_became_peered":"2024-03-30T13:10:57.688187+0000","last_unstale":"2024-03-30T13:10:58.676356+0000","last_undegraded":"2024-03-30T13:10:58.676356+0000","last_fullsized":"2024-03-30T13:10:58.676356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:13:34.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033219+0000","last_change":"2024-03-30T13:10:57.682984+0000","last_active":"2024-03-30T13:10:59.033219+0000","last_peered":"2024-03-30T13:10:59.033219+0000","last_clean":"2024-03-30T13:10:59.033219+0000","last_became_active":"2024-03-30T13:10:57.682685+0000","last_became_peered":"2024-03-30T13:10:57.682685+0000","last_unstale":"2024-03-30T13:10:59.033219+0000","last_undegraded":"2024-03-30T13:10:59.033219+0000","last_fullsized":"2024-03-30T13:10:59.033219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:28:51.189027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746047+0000","last_change":"2024-03-30T13:10:56.673822+0000","last_active":"2024-03-30T13:10:59.746047+0000","last_peered":"2024-03-30T13:10:59.746047+0000","last_clean":"2024-03-30T13:10:59.746047+0000","last_became_active":"2024-03-30T13:10:56.673533+0000","last_became_peered":"2024-03-30T13:10:56.673533+0000","last_unstale":"2024-03-30T13:10:59.746047+0000","last_undegraded":"2024-03-30T13:10:59.746047+0000","last_fullsized":"2024-03-30T13:10:59.746047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:51:29.821157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676839+0000","last_change":"2024-03-30T13:10:57.688749+0000","last_active":"2024-03-30T13:10:58.676839+0000","last_peered":"2024-03-30T13:10:58.676839+0000","last_clean":"2024-03-30T13:10:58.676839+0000","last_became_active":"2024-03-30T13:10:57.687457+0000","last_became_peered":"2024-03-30T13:10:57.687457+0000","last_unstale":"2024-03-30T13:10:58.676839+0000","last_undegraded":"2024-03-30T13:10:58.676839+0000","last_fullsized":"2024-03-30T13:10:58.676839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:42.131116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745498+0000","last_change":"2024-03-30T13:10:56.673044+0000","last_active":"2024-03-30T13:10:59.745498+0000","last_peered":"2024-03-30T13:10:59.745498+0000","last_clean":"2024-03-30T13:10:59.745498+0000","last_became_active":"2024-03-30T13:10:56.672750+0000","last_became_peered":"2024-03-30T13:10:56.672750+0000","last_unstale":"2024-03-30T13:10:59.745498+0000","last_undegraded":"2024-03-30T13:10:59.745498+0000","last_fullsized":"2024-03-30T13:10:59.745498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672861+0000","last_change":"2024-03-30T13:10:57.685414+0000","last_active":"2024-03-30T13:10:58.672861+0000","last_peered":"2024-03-30T13:10:58.672861+0000","last_clean":"2024-03-30T13:10:58.672861+0000","last_became_active":"2024-03-30T13:10:57.685131+0000","last_became_peered":"2024-03-30T13:10:57.685131+0000","last_unstale":"2024-03-30T13:10:58.672861+0000","last_undegraded":"2024-03-30T13:10:58.672861+0000","last_fullsized":"2024-03-30T13:10:58.672861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676898+0000","last_change":"2024-03-30T13:10:56.679798+0000","last_active":"2024-03-30T13:10:58.676898+0000","last_peered":"2024-03-30T13:10:58.676898+0000","last_clean":"2024-03-30T13:10:58.676898+0000","last_became_active":"2024-03-30T13:10:56.679500+0000","last_became_peered":"2024-03-30T13:10:56.679500+0000","last_unstale":"2024-03-30T13:10:58.676898+0000","last_undegraded":"2024-03-30T13:10:58.676898+0000","last_fullsized":"2024-03-30T13:10:58.676898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672316+0000","last_change":"2024-03-30T13:10:57.688829+0000","last_active":"2024-03-30T13:10:58.672316+0000","last_peered":"2024-03-30T13:10:58.672316+0000","last_clean":"2024-03-30T13:10:58.672316+0000","last_became_active":"2024-03-30T13:10:57.686054+0000","last_became_peered":"2024-03-30T13:10:57.686054+0000","last_unstale":"2024-03-30T13:10:58.672316+0000","last_undegraded":"2024-03-30T13:10:58.672316+0000","last_fullsized":"2024-03-30T13:10:58.672316+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:03:47.898459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672809+0000","last_change":"2024-03-30T13:10:56.674537+0000","last_active":"2024-03-30T13:10:58.672809+0000","last_peered":"2024-03-30T13:10:58.672809+0000","last_clean":"2024-03-30T13:10:58.672809+0000","last_became_active":"2024-03-30T13:10:56.674248+0000","last_became_peered":"2024-03-30T13:10:56.674248+0000","last_unstale":"2024-03-30T13:10:58.672809+0000","last_undegraded":"2024-03-30T13:10:58.672809+0000","last_fullsized":"2024-03-30T13:10:58.672809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745165+0000","last_change":"2024-03-30T13:10:57.693802+0000","last_active":"2024-03-30T13:10:59.745165+0000","last_peered":"2024-03-30T13:10:59.745165+0000","last_clean":"2024-03-30T13:10:59.745165+0000","last_became_active":"2024-03-30T13:10:57.693436+0000","last_became_peered":"2024-03-30T13:10:57.693436+0000","last_unstale":"2024-03-30T13:10:59.745165+0000","last_undegraded":"2024-03-30T13:10:59.745165+0000","last_fullsized":"2024-03-30T13:10:59.745165+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681179+0000","last_change":"2024-03-30T13:10:56.681576+0000","last_active":"2024-03-30T13:10:58.681179+0000","last_peered":"2024-03-30T13:10:58.681179+0000","last_clean":"2024-03-30T13:10:58.681179+0000","last_became_active":"2024-03-30T13:10:56.681081+0000","last_became_peered":"2024-03-30T13:10:56.681081+0000","last_unstale":"2024-03-30T13:10:58.681179+0000","last_undegraded":"2024-03-30T13:10:58.681179+0000","last_fullsized":"2024-03-30T13:10:58.681179+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681212+0000","last_change":"2024-03-30T13:10:57.691276+0000","last_active":"2024-03-30T13:10:58.681212+0000","last_peered":"2024-03-30T13:10:58.681212+0000","last_clean":"2024-03-30T13:10:58.681212+0000","last_became_active":"2024-03-30T13:10:57.690846+0000","last_became_peered":"2024-03-30T13:10:57.690846+0000","last_unstale":"2024-03-30T13:10:58.681212+0000","last_undegraded":"2024-03-30T13:10:58.681212+0000","last_fullsized":"2024-03-30T13:10:58.681212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:29.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032969+0000","last_change":"2024-03-30T13:10:56.677670+0000","last_active":"2024-03-30T13:10:59.032969+0000","last_peered":"2024-03-30T13:10:59.032969+0000","last_clean":"2024-03-30T13:10:59.032969+0000","last_became_active":"2024-03-30T13:10:56.677367+0000","last_became_peered":"2024-03-30T13:10:56.677367+0000","last_unstale":"2024-03-30T13:10:59.032969+0000","last_undegraded":"2024-03-30T13:10:59.032969+0000","last_fullsized":"2024-03-30T13:10:59.032969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:14:10.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672971+0000","last_change":"2024-03-30T13:10:57.681840+0000","last_active":"2024-03-30T13:10:58.672971+0000","last_peered":"2024-03-30T13:10:58.672971+0000","last_clean":"2024-03-30T13:10:58.672971+0000","last_became_active":"2024-03-30T13:10:57.681548+0000","last_became_peered":"2024-03-30T13:10:57.681548+0000","last_unstale":"2024-03-30T13:10:58.672971+0000","last_undegraded":"2024-03-30T13:10:58.672971+0000","last_fullsized":"2024-03-30T13:10:58.672971+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676449+0000","last_change":"2024-03-30T13:10:56.682810+0000","last_active":"2024-03-30T13:10:58.676449+0000","last_peered":"2024-03-30T13:10:58.676449+0000","last_clean":"2024-03-30T13:10:58.676449+0000","last_became_active":"2024-03-30T13:10:56.682437+0000","last_became_peered":"2024-03-30T13:10:56.682437+0000","last_unstale":"2024-03-30T13:10:58.676449+0000","last_undegraded":"2024-03-30T13:10:58.676449+0000","last_fullsized":"2024-03-30T13:10:58.676449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672038+0000","last_change":"2024-03-30T13:10:57.678420+0000","last_active":"2024-03-30T13:10:58.672038+0000","last_peered":"2024-03-30T13:10:58.672038+0000","last_clean":"2024-03-30T13:10:58.672038+0000","last_became_active":"2024-03-30T13:10:57.678115+0000","last_became_peered":"2024-03-30T13:10:57.678115+0000","last_unstale":"2024-03-30T13:10:58.672038+0000","last_undegraded":"2024-03-30T13:10:58.672038+0000","last_fullsized":"2024-03-30T13:10:58.672038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:42:22.714841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672311+0000","last_change":"2024-03-30T13:10:56.678877+0000","last_active":"2024-03-30T13:10:58.672311+0000","last_peered":"2024-03-30T13:10:58.672311+0000","last_clean":"2024-03-30T13:10:58.672311+0000","last_became_active":"2024-03-30T13:10:56.678573+0000","last_became_peered":"2024-03-30T13:10:56.678573+0000","last_unstale":"2024-03-30T13:10:58.672311+0000","last_undegraded":"2024-03-30T13:10:58.672311+0000","last_fullsized":"2024-03-30T13:10:58.672311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678784+0000","last_change":"2024-03-30T13:10:57.685526+0000","last_active":"2024-03-30T13:10:58.678784+0000","last_peered":"2024-03-30T13:10:58.678784+0000","last_clean":"2024-03-30T13:10:58.678784+0000","last_became_active":"2024-03-30T13:10:57.681079+0000","last_became_peered":"2024-03-30T13:10:57.681079+0000","last_unstale":"2024-03-30T13:10:58.678784+0000","last_undegraded":"2024-03-30T13:10:58.678784+0000","last_fullsized":"2024-03-30T13:10:58.678784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:36:15.323960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672394+0000","last_change":"2024-03-30T13:10:56.680735+0000","last_active":"2024-03-30T13:10:58.672394+0000","last_peered":"2024-03-30T13:10:58.672394+0000","last_clean":"2024-03-30T13:10:58.672394+0000","last_became_active":"2024-03-30T13:10:56.680459+0000","last_became_peered":"2024-03-30T13:10:56.680459+0000","last_unstale":"2024-03-30T13:10:58.672394+0000","last_undegraded":"2024-03-30T13:10:58.672394+0000","last_fullsized":"2024-03-30T13:10:58.672394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:26.601466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033176+0000","last_change":"2024-03-30T13:10:57.687091+0000","last_active":"2024-03-30T13:10:59.033176+0000","last_peered":"2024-03-30T13:10:59.033176+0000","last_clean":"2024-03-30T13:10:59.033176+0000","last_became_active":"2024-03-30T13:10:57.686821+0000","last_became_peered":"2024-03-30T13:10:57.686821+0000","last_unstale":"2024-03-30T13:10:59.033176+0000","last_undegraded":"2024-03-30T13:10:59.033176+0000","last_fullsized":"2024-03-30T13:10:59.033176+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:16:42.538868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681476+0000","last_change":"2024-03-30T13:10:56.682722+0000","last_active":"2024-03-30T13:10:58.681476+0000","last_peered":"2024-03-30T13:10:58.681476+0000","last_clean":"2024-03-30T13:10:58.681476+0000","last_became_active":"2024-03-30T13:10:56.682272+0000","last_became_peered":"2024-03-30T13:10:56.682272+0000","last_unstale":"2024-03-30T13:10:58.681476+0000","last_undegraded":"2024-03-30T13:10:58.681476+0000","last_fullsized":"2024-03-30T13:10:58.681476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745459+0000","last_change":"2024-03-30T13:10:57.691155+0000","last_active":"2024-03-30T13:10:59.745459+0000","last_peered":"2024-03-30T13:10:59.745459+0000","last_clean":"2024-03-30T13:10:59.745459+0000","last_became_active":"2024-03-30T13:10:57.690702+0000","last_became_peered":"2024-03-30T13:10:57.690702+0000","last_unstale":"2024-03-30T13:10:59.745459+0000","last_undegraded":"2024-03-30T13:10:59.745459+0000","last_fullsized":"2024-03-30T13:10:59.745459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681550+0000","last_change":"2024-03-30T13:10:56.683050+0000","last_active":"2024-03-30T13:10:58.681550+0000","last_peered":"2024-03-30T13:10:58.681550+0000","last_clean":"2024-03-30T13:10:58.681550+0000","last_became_active":"2024-03-30T13:10:56.682514+0000","last_became_peered":"2024-03-30T13:10:56.682514+0000","last_unstale":"2024-03-30T13:10:58.681550+0000","last_undegraded":"2024-03-30T13:10:58.681550+0000","last_fullsized":"2024-03-30T13:10:58.681550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672597+0000","last_change":"2024-03-30T13:10:57.692928+0000","last_active":"2024-03-30T13:10:58.672597+0000","last_peered":"2024-03-30T13:10:58.672597+0000","last_clean":"2024-03-30T13:10:58.672597+0000","last_became_active":"2024-03-30T13:10:57.692672+0000","last_became_peered":"2024-03-30T13:10:57.692672+0000","last_unstale":"2024-03-30T13:10:58.672597+0000","last_undegraded":"2024-03-30T13:10:58.672597+0000","last_fullsized":"2024-03-30T13:10:58.672597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672248+0000","last_change":"2024-03-30T13:10:56.676836+0000","last_active":"2024-03-30T13:10:58.672248+0000","last_peered":"2024-03-30T13:10:58.672248+0000","last_clean":"2024-03-30T13:10:58.672248+0000","last_became_active":"2024-03-30T13:10:56.676586+0000","last_became_peered":"2024-03-30T13:10:56.676586+0000","last_unstale":"2024-03-30T13:10:58.672248+0000","last_undegraded":"2024-03-30T13:10:58.672248+0000","last_fullsized":"2024-03-30T13:10:58.672248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:15.014930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745111+0000","last_change":"2024-03-30T13:10:57.690831+0000","last_active":"2024-03-30T13:10:59.745111+0000","last_peered":"2024-03-30T13:10:59.745111+0000","last_clean":"2024-03-30T13:10:59.745111+0000","last_became_active":"2024-03-30T13:10:57.690522+0000","last_became_peered":"2024-03-30T13:10:57.690522+0000","last_unstale":"2024-03-30T13:10:59.745111+0000","last_undegraded":"2024-03-30T13:10:59.745111+0000","last_fullsized":"2024-03-30T13:10:59.745111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:16.019984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672474+0000","last_change":"2024-03-30T13:10:56.676370+0000","last_active":"2024-03-30T13:10:58.672474+0000","last_peered":"2024-03-30T13:10:58.672474+0000","last_clean":"2024-03-30T13:10:58.672474+0000","last_became_active":"2024-03-30T13:10:56.675812+0000","last_became_peered":"2024-03-30T13:10:56.675812+0000","last_unstale":"2024-03-30T13:10:58.672474+0000","last_undegraded":"2024-03-30T13:10:58.672474+0000","last_fullsized":"2024-03-30T13:10:58.672474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:24.867217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671859+0000","last_change":"2024-03-30T13:10:57.684924+0000","last_active":"2024-03-30T13:10:58.671859+0000","last_peered":"2024-03-30T13:10:58.671859+0000","last_clean":"2024-03-30T13:10:58.671859+0000","last_became_active":"2024-03-30T13:10:57.684588+0000","last_became_peered":"2024-03-30T13:10:57.684588+0000","last_unstale":"2024-03-30T13:10:58.671859+0000","last_undegraded":"2024-03-30T13:10:58.671859+0000","last_fullsized":"2024-03-30T13:10:58.671859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:20:39.043512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676374+0000","last_change":"2024-03-30T13:10:56.683571+0000","last_active":"2024-03-30T13:10:58.676374+0000","last_peered":"2024-03-30T13:10:58.676374+0000","last_clean":"2024-03-30T13:10:58.676374+0000","last_became_active":"2024-03-30T13:10:56.683215+0000","last_became_peered":"2024-03-30T13:10:56.683215+0000","last_unstale":"2024-03-30T13:10:58.676374+0000","last_undegraded":"2024-03-30T13:10:58.676374+0000","last_fullsized":"2024-03-30T13:10:58.676374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033087+0000","last_change":"2024-03-30T13:10:57.690221+0000","last_active":"2024-03-30T13:10:59.033087+0000","last_peered":"2024-03-30T13:10:59.033087+0000","last_clean":"2024-03-30T13:10:59.033087+0000","last_became_active":"2024-03-30T13:10:57.689942+0000","last_became_peered":"2024-03-30T13:10:57.689942+0000","last_unstale":"2024-03-30T13:10:59.033087+0000","last_undegraded":"2024-03-30T13:10:59.033087+0000","last_fullsized":"2024-03-30T13:10:59.033087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745733+0000","last_change":"2024-03-30T13:10:56.674381+0000","last_active":"2024-03-30T13:10:59.745733+0000","last_peered":"2024-03-30T13:10:59.745733+0000","last_clean":"2024-03-30T13:10:59.745733+0000","last_became_active":"2024-03-30T13:10:56.674086+0000","last_became_peered":"2024-03-30T13:10:56.674086+0000","last_unstale":"2024-03-30T13:10:59.745733+0000","last_undegraded":"2024-03-30T13:10:59.745733+0000","last_fullsized":"2024-03-30T13:10:59.745733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:02.130520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672207+0000","last_change":"2024-03-30T13:10:57.689668+0000","last_active":"2024-03-30T13:10:58.672207+0000","last_peered":"2024-03-30T13:10:58.672207+0000","last_clean":"2024-03-30T13:10:58.672207+0000","last_became_active":"2024-03-30T13:10:57.689363+0000","last_became_peered":"2024-03-30T13:10:57.689363+0000","last_unstale":"2024-03-30T13:10:58.672207+0000","last_undegraded":"2024-03-30T13:10:58.672207+0000","last_fullsized":"2024-03-30T13:10:58.672207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:25.435052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032876+0000","last_change":"2024-03-30T13:10:56.676925+0000","last_active":"2024-03-30T13:10:59.032876+0000","last_peered":"2024-03-30T13:10:59.032876+0000","last_clean":"2024-03-30T13:10:59.032876+0000","last_became_active":"2024-03-30T13:10:56.676627+0000","last_became_peered":"2024-03-30T13:10:56.676627+0000","last_unstale":"2024-03-30T13:10:59.032876+0000","last_undegraded":"2024-03-30T13:10:59.032876+0000","last_fullsized":"2024-03-30T13:10:59.032876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672078+0000","last_change":"2024-03-30T13:10:57.686177+0000","last_active":"2024-03-30T13:10:58.672078+0000","last_peered":"2024-03-30T13:10:58.672078+0000","last_clean":"2024-03-30T13:10:58.672078+0000","last_became_active":"2024-03-30T13:10:57.685857+0000","last_became_peered":"2024-03-30T13:10:57.685857+0000","last_unstale":"2024-03-30T13:10:58.672078+0000","last_undegraded":"2024-03-30T13:10:58.672078+0000","last_fullsized":"2024-03-30T13:10:58.672078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:28.503626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681448+0000","last_change":"2024-03-30T13:10:56.681832+0000","last_active":"2024-03-30T13:10:58.681448+0000","last_peered":"2024-03-30T13:10:58.681448+0000","last_clean":"2024-03-30T13:10:58.681448+0000","last_became_active":"2024-03-30T13:10:56.681522+0000","last_became_peered":"2024-03-30T13:10:56.681522+0000","last_unstale":"2024-03-30T13:10:58.681448+0000","last_undegraded":"2024-03-30T13:10:58.681448+0000","last_fullsized":"2024-03-30T13:10:58.681448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681440+0000","last_change":"2024-03-30T13:10:57.693544+0000","last_active":"2024-03-30T13:10:58.681440+0000","last_peered":"2024-03-30T13:10:58.681440+0000","last_clean":"2024-03-30T13:10:58.681440+0000","last_became_active":"2024-03-30T13:10:57.693276+0000","last_became_peered":"2024-03-30T13:10:57.693276+0000","last_unstale":"2024-03-30T13:10:58.681440+0000","last_undegraded":"2024-03-30T13:10:58.681440+0000","last_fullsized":"2024-03-30T13:10:58.681440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:31.858312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672976+0000","last_change":"2024-03-30T13:10:56.681831+0000","last_active":"2024-03-30T13:10:58.672976+0000","last_peered":"2024-03-30T13:10:58.672976+0000","last_clean":"2024-03-30T13:10:58.672976+0000","last_became_active":"2024-03-30T13:10:56.681581+0000","last_became_peered":"2024-03-30T13:10:56.681581+0000","last_unstale":"2024-03-30T13:10:58.672976+0000","last_undegraded":"2024-03-30T13:10:58.672976+0000","last_fullsized":"2024-03-30T13:10:58.672976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:54.818674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.673095+0000","last_change":"2024-03-30T13:10:57.689473+0000","last_active":"2024-03-30T13:10:58.673095+0000","last_peered":"2024-03-30T13:10:58.673095+0000","last_clean":"2024-03-30T13:10:58.673095+0000","last_became_active":"2024-03-30T13:10:57.689184+0000","last_became_peered":"2024-03-30T13:10:57.689184+0000","last_unstale":"2024-03-30T13:10:58.673095+0000","last_undegraded":"2024-03-30T13:10:58.673095+0000","last_fullsized":"2024-03-30T13:10:58.673095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678850+0000","last_change":"2024-03-30T13:10:56.671520+0000","last_active":"2024-03-30T13:10:58.678850+0000","last_peered":"2024-03-30T13:10:58.678850+0000","last_clean":"2024-03-30T13:10:58.678850+0000","last_became_active":"2024-03-30T13:10:56.671198+0000","last_became_peered":"2024-03-30T13:10:56.671198+0000","last_unstale":"2024-03-30T13:10:58.678850+0000","last_undegraded":"2024-03-30T13:10:58.678850+0000","last_fullsized":"2024-03-30T13:10:58.678850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:48:29.323071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.679016+0000","last_change":"2024-03-30T13:10:57.682760+0000","last_active":"2024-03-30T13:10:58.679016+0000","last_peered":"2024-03-30T13:10:58.679016+0000","last_clean":"2024-03-30T13:10:58.679016+0000","last_became_active":"2024-03-30T13:10:57.682495+0000","last_became_peered":"2024-03-30T13:10:57.682495+0000","last_unstale":"2024-03-30T13:10:58.679016+0000","last_undegraded":"2024-03-30T13:10:58.679016+0000","last_fullsized":"2024-03-30T13:10:58.679016+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:03:14.415400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676601+0000","last_change":"2024-03-30T13:10:56.679764+0000","last_active":"2024-03-30T13:10:58.676601+0000","last_peered":"2024-03-30T13:10:58.676601+0000","last_clean":"2024-03-30T13:10:58.676601+0000","last_became_active":"2024-03-30T13:10:56.679483+0000","last_became_peered":"2024-03-30T13:10:56.679483+0000","last_unstale":"2024-03-30T13:10:58.676601+0000","last_undegraded":"2024-03-30T13:10:58.676601+0000","last_fullsized":"2024-03-30T13:10:58.676601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033042+0000","last_change":"2024-03-30T13:10:57.693368+0000","last_active":"2024-03-30T13:10:59.033042+0000","last_peered":"2024-03-30T13:10:59.033042+0000","last_clean":"2024-03-30T13:10:59.033042+0000","last_became_active":"2024-03-30T13:10:57.693076+0000","last_became_peered":"2024-03-30T13:10:57.693076+0000","last_unstale":"2024-03-30T13:10:59.033042+0000","last_undegraded":"2024-03-30T13:10:59.033042+0000","last_fullsized":"2024-03-30T13:10:59.033042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678626+0000","last_change":"2024-03-30T13:10:56.677591+0000","last_active":"2024-03-30T13:10:58.678626+0000","last_peered":"2024-03-30T13:10:58.678626+0000","last_clean":"2024-03-30T13:10:58.678626+0000","last_became_active":"2024-03-30T13:10:56.677289+0000","last_became_peered":"2024-03-30T13:10:56.677289+0000","last_unstale":"2024-03-30T13:10:58.678626+0000","last_undegraded":"2024-03-30T13:10:58.678626+0000","last_fullsized":"2024-03-30T13:10:58.678626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:55:48.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745181+0000","last_change":"2024-03-30T13:10:57.686202+0000","last_active":"2024-03-30T13:10:59.745181+0000","last_peered":"2024-03-30T13:10:59.745181+0000","last_clean":"2024-03-30T13:10:59.745181+0000","last_became_active":"2024-03-30T13:10:57.685942+0000","last_became_peered":"2024-03-30T13:10:57.685942+0000","last_unstale":"2024-03-30T13:10:59.745181+0000","last_undegraded":"2024-03-30T13:10:59.745181+0000","last_fullsized":"2024-03-30T13:10:59.745181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678579+0000","last_change":"2024-03-30T13:10:56.677268+0000","last_active":"2024-03-30T13:10:58.678579+0000","last_peered":"2024-03-30T13:10:58.678579+0000","last_clean":"2024-03-30T13:10:58.678579+0000","last_became_active":"2024-03-30T13:10:56.676985+0000","last_became_peered":"2024-03-30T13:10:56.676985+0000","last_unstale":"2024-03-30T13:10:58.678579+0000","last_undegraded":"2024-03-30T13:10:58.678579+0000","last_fullsized":"2024-03-30T13:10:58.678579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:31:09.416890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681045+0000","last_change":"2024-03-30T13:10:57.686046+0000","last_active":"2024-03-30T13:10:58.681045+0000","last_peered":"2024-03-30T13:10:58.681045+0000","last_clean":"2024-03-30T13:10:58.681045+0000","last_became_active":"2024-03-30T13:10:57.685805+0000","last_became_peered":"2024-03-30T13:10:57.685805+0000","last_unstale":"2024-03-30T13:10:58.681045+0000","last_undegraded":"2024-03-30T13:10:58.681045+0000","last_fullsized":"2024-03-30T13:10:58.681045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:21:38.368777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033098+0000","last_change":"2024-03-30T13:10:56.680608+0000","last_active":"2024-03-30T13:10:59.033098+0000","last_peered":"2024-03-30T13:10:59.033098+0000","last_clean":"2024-03-30T13:10:59.033098+0000","last_became_active":"2024-03-30T13:10:56.680346+0000","last_became_peered":"2024-03-30T13:10:56.680346+0000","last_unstale":"2024-03-30T13:10:59.033098+0000","last_undegraded":"2024-03-30T13:10:59.033098+0000","last_fullsized":"2024-03-30T13:10:59.033098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672201+0000","last_change":"2024-03-30T13:10:57.678124+0000","last_active":"2024-03-30T13:10:58.672201+0000","last_peered":"2024-03-30T13:10:58.672201+0000","last_clean":"2024-03-30T13:10:58.672201+0000","last_became_active":"2024-03-30T13:10:57.677826+0000","last_became_peered":"2024-03-30T13:10:57.677826+0000","last_unstale":"2024-03-30T13:10:58.672201+0000","last_undegraded":"2024-03-30T13:10:58.672201+0000","last_fullsized":"2024-03-30T13:10:58.672201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:37.076824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678461+0000","last_change":"2024-03-30T13:10:56.678610+0000","last_active":"2024-03-30T13:10:58.678461+0000","last_peered":"2024-03-30T13:10:58.678461+0000","last_clean":"2024-03-30T13:10:58.678461+0000","last_became_active":"2024-03-30T13:10:56.678357+0000","last_became_peered":"2024-03-30T13:10:56.678357+0000","last_unstale":"2024-03-30T13:10:58.678461+0000","last_undegraded":"2024-03-30T13:10:58.678461+0000","last_fullsized":"2024-03-30T13:10:58.678461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:40.674849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676600+0000","last_change":"2024-03-30T13:10:57.687196+0000","last_active":"2024-03-30T13:10:58.676600+0000","last_peered":"2024-03-30T13:10:58.676600+0000","last_clean":"2024-03-30T13:10:58.676600+0000","last_became_active":"2024-03-30T13:10:57.686683+0000","last_became_peered":"2024-03-30T13:10:57.686683+0000","last_unstale":"2024-03-30T13:10:58.676600+0000","last_undegraded":"2024-03-30T13:10:58.676600+0000","last_fullsized":"2024-03-30T13:10:58.676600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:44:15.354797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681577+0000","last_change":"2024-03-30T13:10:56.682787+0000","last_active":"2024-03-30T13:10:58.681577+0000","last_peered":"2024-03-30T13:10:58.681577+0000","last_clean":"2024-03-30T13:10:58.681577+0000","last_became_active":"2024-03-30T13:10:56.682536+0000","last_became_peered":"2024-03-30T13:10:56.682536+0000","last_unstale":"2024-03-30T13:10:58.681577+0000","last_undegraded":"2024-03-30T13:10:58.681577+0000","last_fullsized":"2024-03-30T13:10:58.681577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:28:07.167637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033243+0000","last_change":"2024-03-30T13:10:57.686653+0000","last_active":"2024-03-30T13:10:59.033243+0000","last_peered":"2024-03-30T13:10:59.033243+0000","last_clean":"2024-03-30T13:10:59.033243+0000","last_became_active":"2024-03-30T13:10:57.686380+0000","last_became_peered":"2024-03-30T13:10:57.686380+0000","last_unstale":"2024-03-30T13:10:59.033243+0000","last_undegraded":"2024-03-30T13:10:59.033243+0000","last_fullsized":"2024-03-30T13:10:59.033243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:13:33.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680418+0000","last_change":"2024-03-30T13:10:56.682042+0000","last_active":"2024-03-30T13:10:58.680418+0000","last_peered":"2024-03-30T13:10:58.680418+0000","last_clean":"2024-03-30T13:10:58.680418+0000","last_became_active":"2024-03-30T13:10:56.681748+0000","last_became_peered":"2024-03-30T13:10:56.681748+0000","last_unstale":"2024-03-30T13:10:58.680418+0000","last_undegraded":"2024-03-30T13:10:58.680418+0000","last_fullsized":"2024-03-30T13:10:58.680418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:55:55.240936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672322+0000","last_change":"2024-03-30T13:10:57.676679+0000","last_active":"2024-03-30T13:10:58.672322+0000","last_peered":"2024-03-30T13:10:58.672322+0000","last_clean":"2024-03-30T13:10:58.672322+0000","last_became_active":"2024-03-30T13:10:57.675136+0000","last_became_peered":"2024-03-30T13:10:57.675136+0000","last_unstale":"2024-03-30T13:10:58.672322+0000","last_undegraded":"2024-03-30T13:10:58.672322+0000","last_fullsized":"2024-03-30T13:10:58.672322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:01:31.125961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745366+0000","last_change":"2024-03-30T13:10:56.682275+0000","last_active":"2024-03-30T13:10:59.745366+0000","last_peered":"2024-03-30T13:10:59.745366+0000","last_clean":"2024-03-30T13:10:59.745366+0000","last_became_active":"2024-03-30T13:10:56.681990+0000","last_became_peered":"2024-03-30T13:10:56.681990+0000","last_unstale":"2024-03-30T13:10:59.745366+0000","last_undegraded":"2024-03-30T13:10:59.745366+0000","last_fullsized":"2024-03-30T13:10:59.745366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745290+0000","last_change":"2024-03-30T13:10:57.692363+0000","last_active":"2024-03-30T13:10:59.745290+0000","last_peered":"2024-03-30T13:10:59.745290+0000","last_clean":"2024-03-30T13:10:59.745290+0000","last_became_active":"2024-03-30T13:10:57.692063+0000","last_became_peered":"2024-03-30T13:10:57.692063+0000","last_unstale":"2024-03-30T13:10:59.745290+0000","last_undegraded":"2024-03-30T13:10:59.745290+0000","last_fullsized":"2024-03-30T13:10:59.745290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745367+0000","last_change":"2024-03-30T13:10:56.680716+0000","last_active":"2024-03-30T13:10:59.745367+0000","last_peered":"2024-03-30T13:10:59.745367+0000","last_clean":"2024-03-30T13:10:59.745367+0000","last_became_active":"2024-03-30T13:10:56.680446+0000","last_became_peered":"2024-03-30T13:10:56.680446+0000","last_unstale":"2024-03-30T13:10:59.745367+0000","last_undegraded":"2024-03-30T13:10:59.745367+0000","last_fullsized":"2024-03-30T13:10:59.745367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676703+0000","last_change":"2024-03-30T13:10:57.684334+0000","last_active":"2024-03-30T13:10:58.676703+0000","last_peered":"2024-03-30T13:10:58.676703+0000","last_clean":"2024-03-30T13:10:58.676703+0000","last_became_active":"2024-03-30T13:10:57.684037+0000","last_became_peered":"2024-03-30T13:10:57.684037+0000","last_unstale":"2024-03-30T13:10:58.676703+0000","last_undegraded":"2024-03-30T13:10:58.676703+0000","last_fullsized":"2024-03-30T13:10:58.676703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:07:21.631648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745812+0000","last_change":"2024-03-30T13:10:56.676202+0000","last_active":"2024-03-30T13:10:59.745812+0000","last_peered":"2024-03-30T13:10:59.745812+0000","last_clean":"2024-03-30T13:10:59.745812+0000","last_became_active":"2024-03-30T13:10:56.675907+0000","last_became_peered":"2024-03-30T13:10:56.675907+0000","last_unstale":"2024-03-30T13:10:59.745812+0000","last_undegraded":"2024-03-30T13:10:59.745812+0000","last_fullsized":"2024-03-30T13:10:59.745812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:58.975900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672618+0000","last_change":"2024-03-30T13:10:57.673049+0000","last_active":"2024-03-30T13:10:58.672618+0000","last_peered":"2024-03-30T13:10:58.672618+0000","last_clean":"2024-03-30T13:10:58.672618+0000","last_became_active":"2024-03-30T13:10:57.672738+0000","last_became_peered":"2024-03-30T13:10:57.672738+0000","last_unstale":"2024-03-30T13:10:58.672618+0000","last_undegraded":"2024-03-30T13:10:58.672618+0000","last_fullsized":"2024-03-30T13:10:58.672618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672426+0000","last_change":"2024-03-30T13:10:56.675234+0000","last_active":"2024-03-30T13:10:58.672426+0000","last_peered":"2024-03-30T13:10:58.672426+0000","last_clean":"2024-03-30T13:10:58.672426+0000","last_became_active":"2024-03-30T13:10:56.674965+0000","last_became_peered":"2024-03-30T13:10:56.674965+0000","last_unstale":"2024-03-30T13:10:58.672426+0000","last_undegraded":"2024-03-30T13:10:58.672426+0000","last_fullsized":"2024-03-30T13:10:58.672426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:35.628852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033158+0000","last_change":"2024-03-30T13:10:57.683624+0000","last_active":"2024-03-30T13:10:59.033158+0000","last_peered":"2024-03-30T13:10:59.033158+0000","last_clean":"2024-03-30T13:10:59.033158+0000","last_became_active":"2024-03-30T13:10:57.683346+0000","last_became_peered":"2024-03-30T13:10:57.683346+0000","last_unstale":"2024-03-30T13:10:59.033158+0000","last_undegraded":"2024-03-30T13:10:59.033158+0000","last_fullsized":"2024-03-30T13:10:59.033158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671837+0000","last_change":"2024-03-30T13:10:56.682711+0000","last_active":"2024-03-30T13:10:58.671837+0000","last_peered":"2024-03-30T13:10:58.671837+0000","last_clean":"2024-03-30T13:10:58.671837+0000","last_became_active":"2024-03-30T13:10:56.682415+0000","last_became_peered":"2024-03-30T13:10:56.682415+0000","last_unstale":"2024-03-30T13:10:58.671837+0000","last_undegraded":"2024-03-30T13:10:58.671837+0000","last_fullsized":"2024-03-30T13:10:58.671837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:10:34.096760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681340+0000","last_change":"2024-03-30T13:10:57.684219+0000","last_active":"2024-03-30T13:10:58.681340+0000","last_peered":"2024-03-30T13:10:58.681340+0000","last_clean":"2024-03-30T13:10:58.681340+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:58.681340+0000","last_undegraded":"2024-03-30T13:10:58.681340+0000","last_fullsized":"2024-03-30T13:10:58.681340+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:53:48.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033155+0000","last_change":"2024-03-30T13:10:56.681620+0000","last_active":"2024-03-30T13:10:59.033155+0000","last_peered":"2024-03-30T13:10:59.033155+0000","last_clean":"2024-03-30T13:10:59.033155+0000","last_became_active":"2024-03-30T13:10:56.681341+0000","last_became_peered":"2024-03-30T13:10:56.681341+0000","last_unstale":"2024-03-30T13:10:59.033155+0000","last_undegraded":"2024-03-30T13:10:59.033155+0000","last_fullsized":"2024-03-30T13:10:59.033155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:46:52.577909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032961+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:10:59.032961+0000","last_peered":"2024-03-30T13:10:59.032961+0000","last_clean":"2024-03-30T13:10:59.032961+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:10:59.032961+0000","last_undegraded":"2024-03-30T13:10:59.032961+0000","last_fullsized":"2024-03-30T13:10:59.032961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":199796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":199818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":782924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":462848,"data_stored":199326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:06.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.381+0000 7f2861d38640 1 Processor -- start 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.381+0000 7f2861d38640 1 -- start start 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f2861d38640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f285c0d4a90 0x7f285c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f2861d38640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c0d36f0 con 0x7f285c0d2f00 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f2861d38640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c0d39d0 con 0x7f285c15e950 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f2861d38640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f285c0d3cb0 con 0x7f285c0d4a90 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285bfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f285c0d2f00 0x7f285c0b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60702/0 (socket says 172.21.15.66:60702) 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285bfff640 1 -- 172.21.15.66:0/2914893165 learned_addr learned my addr 172.21.15.66:0/2914893165 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2054912193 0 0) 0x7f285c0d36f0 con 0x7f285c0d2f00 2024-03-30T13:11:06.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285affd640 1 --2- 172.21.15.66:0/2914893165 >> v2:172.21.15.117:3300/0 conn(0x7f285c0d4a90 0x7f285c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2844003520 con 0x7f285c0d2f00 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1355378043 0 0) 0x7f2844003520 con 0x7f285c0d2f00 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 >> v2:172.21.15.117:3300/0 conn(0x7f285c0d4a90 msgr2=0x7f285c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 --2- 172.21.15.66:0/2914893165 >> v2:172.21.15.117:3300/0 conn(0x7f285c0d4a90 0x7f285c0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 >> v1:172.21.15.117:6789/0 conn(0x7f285c15e950 legacy=0x7f285c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0b7ac0 con 0x7f285c0d2f00 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f28540019f0 con 0x7f285c0d2f00 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 >> v1:172.21.15.66:6789/0 conn(0x7f285c0d2f00 legacy=0x7f285c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28440039b0 0x7f2844003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2844004800 0x7f2844004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.382+0000 7f285a7fc640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28440051c0 0x7f284400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0d3cb0 con 0x7f28440039b0 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f284400c070 con 0x7f28440051c0 2024-03-30T13:11:06.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f284400c270 con 0x7f2844004800 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285b7fe640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28440039b0 0x7f2844003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285bfff640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28440051c0 0x7f284400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285affd640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2844004800 0x7f2844004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285b7fe640 1 -- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2844004800 msgr2=0x7f2844004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285b7fe640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2844004800 0x7f2844004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285b7fe640 1 -- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28440051c0 msgr2=0x7f284400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285b7fe640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28440051c0 0x7f284400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285b7fe640 1 -- 172.21.15.66:0/2914893165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2844003520 con 0x7f28440039b0 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285bfff640 1 -- 172.21.15.66:0/2914893165 reap_dead start 2024-03-30T13:11:06.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.383+0000 7f285affd640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2844004800 0x7f2844004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.384+0000 7f285b7fe640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28440039b0 0x7f2844003dd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2850001750 tx=0x7f2844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2feb84ff5f5f21a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.384+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850001370 con 0x7f28440039b0 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.384+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28500052d0 con 0x7f28440039b0 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.384+0000 7f285a7fc640 1 -- 172.21.15.66:0/2914893165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850003830 con 0x7f28440039b0 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 -- 172.21.15.66:0/2914893165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28440039b0 msgr2=0x7f2844003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28440039b0 0x7f2844003dd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2850001750 tx=0x7f2844005830 comp rx=0 tx=0).stop 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 -- 172.21.15.66:0/2914893165 shutdown_connections 2024-03-30T13:11:06.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 --2- 172.21.15.66:0/2914893165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28440039b0 0x7f2844003dd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 -- 172.21.15.66:0/2914893165 >> 172.21.15.66:0/2914893165 conn(0x7f285c0676e0 msgr2=0x7f285c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 -- 172.21.15.66:0/2914893165 shutdown_connections 2024-03-30T13:11:06.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.385+0000 7f2861d38640 1 -- 172.21.15.66:0/2914893165 wait complete. 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.387+0000 7f2861d38640 1 Processor -- start 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.387+0000 7f2861d38640 1 -- start start 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f2861d38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28440039b0 0x7f285c1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f2861d38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c159dc0 0x7f285c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f2861d38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 0x7f285c154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f2861d38640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f285c15ed30 con 0x7f285c1549a0 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f2861d38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0d33b0 con 0x7f285c159dc0 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f2861d38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f285c15eeb0 con 0x7f28440039b0 2024-03-30T13:11:06.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f285bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 0x7f285c154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f285bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 0x7f285c154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39588/0 (socket says 172.21.15.66:39588) 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f285bfff640 1 -- 172.21.15.66:0/4031161369 learned_addr learned my addr 172.21.15.66:0/4031161369 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f285b7fe640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28440039b0 0x7f285c1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.388+0000 7f285affd640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c159dc0 0x7f285c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285bfff640 1 -- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28440039b0 msgr2=0x7f285c1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285bfff640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28440039b0 0x7f285c1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285bfff640 1 -- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c159dc0 msgr2=0x7f285c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285bfff640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c159dc0 0x7f285c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285bfff640 1 -- 172.21.15.66:0/4031161369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c15a2f0 con 0x7f285c1549a0 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285b7fe640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28440039b0 0x7f285c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:06.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f285bfff640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 0x7f285c154e20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f285c0d1f30 tx=0x7f2854007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400ab00 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2854019070 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c15a5d0 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540133f0 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.389+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c15ab90 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.390+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c075a10 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.391+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2854013590 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.391+0000 7f2843fff640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f284806c530 0x7f284806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.391+0000 7f285b7fe640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f284806c530 0x7f284806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.391+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f285408bd20 con 0x7f285c1549a0 2024-03-30T13:11:06.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.392+0000 7f285b7fe640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f284806c530 0x7f284806e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2850033000 tx=0x7f2850009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.396+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f285405a1b0 con 0x7f285c1549a0 2024-03-30T13:11:06.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.577+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f285c0cd1b0 con 0x7f284806c530 2024-03-30T13:11:06.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.583+0000 7f2843fff640 1 -- 172.21.15.66:0/4031161369 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f285c0cd1b0 con 0x7f284806c530 2024-03-30T13:11:06.589 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:06.590 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:11:06.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f284806c530 msgr2=0x7f284806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f284806c530 0x7f284806e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2850033000 tx=0x7f2850009030 comp rx=0 tx=0).stop 2024-03-30T13:11:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 msgr2=0x7f285c154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 0x7f285c154e20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f285c0d1f30 tx=0x7f2854007b00 comp rx=0 tx=0).stop 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 shutdown_connections 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f284806c530 0x7f284806e9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f285c1549a0 0x7f285c154e20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c159dc0 0x7f285c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 --2- 172.21.15.66:0/4031161369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28440039b0 0x7f285c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.586+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 >> 172.21.15.66:0/4031161369 conn(0x7f285c0676e0 msgr2=0x7f285c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.587+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 shutdown_connections 2024-03-30T13:11:06.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.587+0000 7f2861d38640 1 -- 172.21.15.66:0/4031161369 wait complete. 2024-03-30T13:11:06.602 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-03-30T13:11:04.811091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220932,"kb_used_data":4752,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753788,"statfs":{"total":773094113280,"available":772867878912,"internally_reserved":0,"allocated":4866048,"data_stored":2765884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001376"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672404+0000","last_change":"2024-03-30T13:10:57.684501+0000","last_active":"2024-03-30T13:10:58.672404+0000","last_peered":"2024-03-30T13:10:58.672404+0000","last_clean":"2024-03-30T13:10:58.672404+0000","last_became_active":"2024-03-30T13:10:57.684187+0000","last_became_peered":"2024-03-30T13:10:57.684187+0000","last_unstale":"2024-03-30T13:10:58.672404+0000","last_undegraded":"2024-03-30T13:10:58.672404+0000","last_fullsized":"2024-03-30T13:10:58.672404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:07.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672159+0000","last_change":"2024-03-30T13:10:57.681207+0000","last_active":"2024-03-30T13:10:58.672159+0000","last_peered":"2024-03-30T13:10:58.672159+0000","last_clean":"2024-03-30T13:10:58.672159+0000","last_became_active":"2024-03-30T13:10:57.680890+0000","last_became_peered":"2024-03-30T13:10:57.680890+0000","last_unstale":"2024-03-30T13:10:58.672159+0000","last_undegraded":"2024-03-30T13:10:58.672159+0000","last_fullsized":"2024-03-30T13:10:58.672159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:30:42.096995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745282+0000","last_change":"2024-03-30T13:10:57.695761+0000","last_active":"2024-03-30T13:10:59.745282+0000","last_peered":"2024-03-30T13:10:59.745282+0000","last_clean":"2024-03-30T13:10:59.745282+0000","last_became_active":"2024-03-30T13:10:57.695488+0000","last_became_peered":"2024-03-30T13:10:57.695488+0000","last_unstale":"2024-03-30T13:10:59.745282+0000","last_undegraded":"2024-03-30T13:10:59.745282+0000","last_fullsized":"2024-03-30T13:10:59.745282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678726+0000","last_change":"2024-03-30T13:10:57.682241+0000","last_active":"2024-03-30T13:10:58.678726+0000","last_peered":"2024-03-30T13:10:58.678726+0000","last_clean":"2024-03-30T13:10:58.678726+0000","last_became_active":"2024-03-30T13:10:57.681383+0000","last_became_peered":"2024-03-30T13:10:57.681383+0000","last_unstale":"2024-03-30T13:10:58.678726+0000","last_undegraded":"2024-03-30T13:10:58.678726+0000","last_fullsized":"2024-03-30T13:10:58.678726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:43:40.092786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678584+0000","last_change":"2024-03-30T13:10:57.680923+0000","last_active":"2024-03-30T13:10:58.678584+0000","last_peered":"2024-03-30T13:10:58.678584+0000","last_clean":"2024-03-30T13:10:58.678584+0000","last_became_active":"2024-03-30T13:10:57.680625+0000","last_became_peered":"2024-03-30T13:10:57.680625+0000","last_unstale":"2024-03-30T13:10:58.678584+0000","last_undegraded":"2024-03-30T13:10:58.678584+0000","last_fullsized":"2024-03-30T13:10:58.678584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:27:38.309021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676495+0000","last_change":"2024-03-30T13:10:57.687677+0000","last_active":"2024-03-30T13:10:58.676495+0000","last_peered":"2024-03-30T13:10:58.676495+0000","last_clean":"2024-03-30T13:10:58.676495+0000","last_became_active":"2024-03-30T13:10:57.687348+0000","last_became_peered":"2024-03-30T13:10:57.687348+0000","last_unstale":"2024-03-30T13:10:58.676495+0000","last_undegraded":"2024-03-30T13:10:58.676495+0000","last_fullsized":"2024-03-30T13:10:58.676495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:33:20.876411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680914+0000","last_change":"2024-03-30T13:10:57.680450+0000","last_active":"2024-03-30T13:10:58.680914+0000","last_peered":"2024-03-30T13:10:58.680914+0000","last_clean":"2024-03-30T13:10:58.680914+0000","last_became_active":"2024-03-30T13:10:57.680147+0000","last_became_peered":"2024-03-30T13:10:57.680147+0000","last_unstale":"2024-03-30T13:10:58.680914+0000","last_undegraded":"2024-03-30T13:10:58.680914+0000","last_fullsized":"2024-03-30T13:10:58.680914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:56.711272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671840+0000","last_change":"2024-03-30T13:10:57.679497+0000","last_active":"2024-03-30T13:10:58.671840+0000","last_peered":"2024-03-30T13:10:58.671840+0000","last_clean":"2024-03-30T13:10:58.671840+0000","last_became_active":"2024-03-30T13:10:57.679201+0000","last_became_peered":"2024-03-30T13:10:57.679201+0000","last_unstale":"2024-03-30T13:10:58.671840+0000","last_undegraded":"2024-03-30T13:10:58.671840+0000","last_fullsized":"2024-03-30T13:10:58.671840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:04:54.328931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745602+0000","last_change":"2024-03-30T13:10:57.690985+0000","last_active":"2024-03-30T13:10:59.745602+0000","last_peered":"2024-03-30T13:10:59.745602+0000","last_clean":"2024-03-30T13:10:59.745602+0000","last_became_active":"2024-03-30T13:10:57.690657+0000","last_became_peered":"2024-03-30T13:10:57.690657+0000","last_unstale":"2024-03-30T13:10:59.745602+0000","last_undegraded":"2024-03-30T13:10:59.745602+0000","last_fullsized":"2024-03-30T13:10:59.745602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681200+0000","last_change":"2024-03-30T13:10:57.680642+0000","last_active":"2024-03-30T13:10:58.681200+0000","last_peered":"2024-03-30T13:10:58.681200+0000","last_clean":"2024-03-30T13:10:58.681200+0000","last_became_active":"2024-03-30T13:10:57.680287+0000","last_became_peered":"2024-03-30T13:10:57.680287+0000","last_unstale":"2024-03-30T13:10:58.681200+0000","last_undegraded":"2024-03-30T13:10:58.681200+0000","last_fullsized":"2024-03-30T13:10:58.681200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:34:29.439469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681467+0000","last_change":"2024-03-30T13:10:57.681398+0000","last_active":"2024-03-30T13:10:58.681467+0000","last_peered":"2024-03-30T13:10:58.681467+0000","last_clean":"2024-03-30T13:10:58.681467+0000","last_became_active":"2024-03-30T13:10:57.681055+0000","last_became_peered":"2024-03-30T13:10:57.681055+0000","last_unstale":"2024-03-30T13:10:58.681467+0000","last_undegraded":"2024-03-30T13:10:58.681467+0000","last_fullsized":"2024-03-30T13:10:58.681467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:12.152970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745292+0000","last_change":"2024-03-30T13:10:57.691635+0000","last_active":"2024-03-30T13:10:59.745292+0000","last_peered":"2024-03-30T13:10:59.745292+0000","last_clean":"2024-03-30T13:10:59.745292+0000","last_became_active":"2024-03-30T13:10:57.691340+0000","last_became_peered":"2024-03-30T13:10:57.691340+0000","last_unstale":"2024-03-30T13:10:59.745292+0000","last_undegraded":"2024-03-30T13:10:59.745292+0000","last_fullsized":"2024-03-30T13:10:59.745292+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680976+0000","last_change":"2024-03-30T13:10:57.680573+0000","last_active":"2024-03-30T13:10:58.680976+0000","last_peered":"2024-03-30T13:10:58.680976+0000","last_clean":"2024-03-30T13:10:58.680976+0000","last_became_active":"2024-03-30T13:10:57.680250+0000","last_became_peered":"2024-03-30T13:10:57.680250+0000","last_unstale":"2024-03-30T13:10:58.680976+0000","last_undegraded":"2024-03-30T13:10:58.680976+0000","last_fullsized":"2024-03-30T13:10:58.680976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:22:10.463077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672408+0000","last_change":"2024-03-30T13:10:57.686031+0000","last_active":"2024-03-30T13:10:58.672408+0000","last_peered":"2024-03-30T13:10:58.672408+0000","last_clean":"2024-03-30T13:10:58.672408+0000","last_became_active":"2024-03-30T13:10:57.685814+0000","last_became_peered":"2024-03-30T13:10:57.685814+0000","last_unstale":"2024-03-30T13:10:58.672408+0000","last_undegraded":"2024-03-30T13:10:58.672408+0000","last_fullsized":"2024-03-30T13:10:58.672408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680965+0000","last_change":"2024-03-30T13:10:57.691786+0000","last_active":"2024-03-30T13:10:58.680965+0000","last_peered":"2024-03-30T13:10:58.680965+0000","last_clean":"2024-03-30T13:10:58.680965+0000","last_became_active":"2024-03-30T13:10:57.690961+0000","last_became_peered":"2024-03-30T13:10:57.690961+0000","last_unstale":"2024-03-30T13:10:58.680965+0000","last_undegraded":"2024-03-30T13:10:58.680965+0000","last_fullsized":"2024-03-30T13:10:58.680965+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:47.380589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032816+0000","last_change":"2024-03-30T13:10:57.681811+0000","last_active":"2024-03-30T13:10:59.032816+0000","last_peered":"2024-03-30T13:10:59.032816+0000","last_clean":"2024-03-30T13:10:59.032816+0000","last_became_active":"2024-03-30T13:10:57.681471+0000","last_became_peered":"2024-03-30T13:10:57.681471+0000","last_unstale":"2024-03-30T13:10:59.032816+0000","last_undegraded":"2024-03-30T13:10:59.032816+0000","last_fullsized":"2024-03-30T13:10:59.032816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033359+0000","last_change":"2024-03-30T13:10:57.687549+0000","last_active":"2024-03-30T13:10:59.033359+0000","last_peered":"2024-03-30T13:10:59.033359+0000","last_clean":"2024-03-30T13:10:59.033359+0000","last_became_active":"2024-03-30T13:10:57.687268+0000","last_became_peered":"2024-03-30T13:10:57.687268+0000","last_unstale":"2024-03-30T13:10:59.033359+0000","last_undegraded":"2024-03-30T13:10:59.033359+0000","last_fullsized":"2024-03-30T13:10:59.033359+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:28:35.714074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746089+0000","last_change":"2024-03-30T13:10:57.683841+0000","last_active":"2024-03-30T13:10:59.746089+0000","last_peered":"2024-03-30T13:10:59.746089+0000","last_clean":"2024-03-30T13:10:59.746089+0000","last_became_active":"2024-03-30T13:10:57.683568+0000","last_became_peered":"2024-03-30T13:10:57.683568+0000","last_unstale":"2024-03-30T13:10:59.746089+0000","last_undegraded":"2024-03-30T13:10:59.746089+0000","last_fullsized":"2024-03-30T13:10:59.746089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:51:46.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678919+0000","last_change":"2024-03-30T13:10:57.688521+0000","last_active":"2024-03-30T13:10:58.678919+0000","last_peered":"2024-03-30T13:10:58.678919+0000","last_clean":"2024-03-30T13:10:58.678919+0000","last_became_active":"2024-03-30T13:10:57.688188+0000","last_became_peered":"2024-03-30T13:10:57.688188+0000","last_unstale":"2024-03-30T13:10:58.678919+0000","last_undegraded":"2024-03-30T13:10:58.678919+0000","last_fullsized":"2024-03-30T13:10:58.678919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:47.722209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745911+0000","last_change":"2024-03-30T13:10:57.682779+0000","last_active":"2024-03-30T13:10:59.745911+0000","last_peered":"2024-03-30T13:10:59.745911+0000","last_clean":"2024-03-30T13:10:59.745911+0000","last_became_active":"2024-03-30T13:10:57.682444+0000","last_became_peered":"2024-03-30T13:10:57.682444+0000","last_unstale":"2024-03-30T13:10:59.745911+0000","last_undegraded":"2024-03-30T13:10:59.745911+0000","last_fullsized":"2024-03-30T13:10:59.745911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:02:29.613267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681089+0000","last_change":"2024-03-30T13:10:57.692912+0000","last_active":"2024-03-30T13:10:58.681089+0000","last_peered":"2024-03-30T13:10:58.681089+0000","last_clean":"2024-03-30T13:10:58.681089+0000","last_became_active":"2024-03-30T13:10:57.692639+0000","last_became_peered":"2024-03-30T13:10:57.692639+0000","last_unstale":"2024-03-30T13:10:58.681089+0000","last_undegraded":"2024-03-30T13:10:58.681089+0000","last_fullsized":"2024-03-30T13:10:58.681089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:01:30.821857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672797+0000","last_change":"2024-03-30T13:10:57.692104+0000","last_active":"2024-03-30T13:10:58.672797+0000","last_peered":"2024-03-30T13:10:58.672797+0000","last_clean":"2024-03-30T13:10:58.672797+0000","last_became_active":"2024-03-30T13:10:57.691816+0000","last_became_peered":"2024-03-30T13:10:57.691816+0000","last_unstale":"2024-03-30T13:10:58.672797+0000","last_undegraded":"2024-03-30T13:10:58.672797+0000","last_fullsized":"2024-03-30T13:10:58.672797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032948+0000","last_change":"2024-03-30T13:10:57.691507+0000","last_active":"2024-03-30T13:10:59.032948+0000","last_peered":"2024-03-30T13:10:59.032948+0000","last_clean":"2024-03-30T13:10:59.032948+0000","last_became_active":"2024-03-30T13:10:57.691248+0000","last_became_peered":"2024-03-30T13:10:57.691248+0000","last_unstale":"2024-03-30T13:10:59.032948+0000","last_undegraded":"2024-03-30T13:10:59.032948+0000","last_fullsized":"2024-03-30T13:10:59.032948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678498+0000","last_change":"2024-03-30T13:10:57.685316+0000","last_active":"2024-03-30T13:10:58.678498+0000","last_peered":"2024-03-30T13:10:58.678498+0000","last_clean":"2024-03-30T13:10:58.678498+0000","last_became_active":"2024-03-30T13:10:57.685053+0000","last_became_peered":"2024-03-30T13:10:57.685053+0000","last_unstale":"2024-03-30T13:10:58.678498+0000","last_undegraded":"2024-03-30T13:10:58.678498+0000","last_fullsized":"2024-03-30T13:10:58.678498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:43:19.868190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746090+0000","last_change":"2024-03-30T13:10:57.684225+0000","last_active":"2024-03-30T13:10:59.746090+0000","last_peered":"2024-03-30T13:10:59.746090+0000","last_clean":"2024-03-30T13:10:59.746090+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:59.746090+0000","last_undegraded":"2024-03-30T13:10:59.746090+0000","last_fullsized":"2024-03-30T13:10:59.746090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:44:06.219419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676846+0000","last_change":"2024-03-30T13:10:57.687484+0000","last_active":"2024-03-30T13:10:58.676846+0000","last_peered":"2024-03-30T13:10:58.676846+0000","last_clean":"2024-03-30T13:10:58.676846+0000","last_became_active":"2024-03-30T13:10:57.687191+0000","last_became_peered":"2024-03-30T13:10:57.687191+0000","last_unstale":"2024-03-30T13:10:58.676846+0000","last_undegraded":"2024-03-30T13:10:58.676846+0000","last_fullsized":"2024-03-30T13:10:58.676846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:49:47.246424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746018+0000","last_change":"2024-03-30T13:10:57.687817+0000","last_active":"2024-03-30T13:10:59.746018+0000","last_peered":"2024-03-30T13:10:59.746018+0000","last_clean":"2024-03-30T13:10:59.746018+0000","last_became_active":"2024-03-30T13:10:57.687038+0000","last_became_peered":"2024-03-30T13:10:57.687038+0000","last_unstale":"2024-03-30T13:10:59.746018+0000","last_undegraded":"2024-03-30T13:10:59.746018+0000","last_fullsized":"2024-03-30T13:10:59.746018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:03:06.653896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681394+0000","last_change":"2024-03-30T13:10:57.681266+0000","last_active":"2024-03-30T13:10:58.681394+0000","last_peered":"2024-03-30T13:10:58.681394+0000","last_clean":"2024-03-30T13:10:58.681394+0000","last_became_active":"2024-03-30T13:10:57.680923+0000","last_became_peered":"2024-03-30T13:10:57.680923+0000","last_unstale":"2024-03-30T13:10:58.681394+0000","last_undegraded":"2024-03-30T13:10:58.681394+0000","last_fullsized":"2024-03-30T13:10:58.681394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:09:43.701244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745905+0000","last_change":"2024-03-30T13:10:57.690320+0000","last_active":"2024-03-30T13:10:59.745905+0000","last_peered":"2024-03-30T13:10:59.745905+0000","last_clean":"2024-03-30T13:10:59.745905+0000","last_became_active":"2024-03-30T13:10:57.689740+0000","last_became_peered":"2024-03-30T13:10:57.689740+0000","last_unstale":"2024-03-30T13:10:59.745905+0000","last_undegraded":"2024-03-30T13:10:59.745905+0000","last_fullsized":"2024-03-30T13:10:59.745905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:55.823728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745918+0000","last_change":"2024-03-30T13:10:57.683564+0000","last_active":"2024-03-30T13:10:59.745918+0000","last_peered":"2024-03-30T13:10:59.745918+0000","last_clean":"2024-03-30T13:10:59.745918+0000","last_became_active":"2024-03-30T13:10:57.683225+0000","last_became_peered":"2024-03-30T13:10:57.683225+0000","last_unstale":"2024-03-30T13:10:59.745918+0000","last_undegraded":"2024-03-30T13:10:59.745918+0000","last_fullsized":"2024-03-30T13:10:59.745918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:36.633906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032986+0000","last_change":"2024-03-30T13:10:57.692664+0000","last_active":"2024-03-30T13:10:59.032986+0000","last_peered":"2024-03-30T13:10:59.032986+0000","last_clean":"2024-03-30T13:10:59.032986+0000","last_became_active":"2024-03-30T13:10:57.692376+0000","last_became_peered":"2024-03-30T13:10:57.692376+0000","last_unstale":"2024-03-30T13:10:59.032986+0000","last_undegraded":"2024-03-30T13:10:59.032986+0000","last_fullsized":"2024-03-30T13:10:59.032986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676356+0000","last_change":"2024-03-30T13:10:57.688812+0000","last_active":"2024-03-30T13:10:58.676356+0000","last_peered":"2024-03-30T13:10:58.676356+0000","last_clean":"2024-03-30T13:10:58.676356+0000","last_became_active":"2024-03-30T13:10:57.688187+0000","last_became_peered":"2024-03-30T13:10:57.688187+0000","last_unstale":"2024-03-30T13:10:58.676356+0000","last_undegraded":"2024-03-30T13:10:58.676356+0000","last_fullsized":"2024-03-30T13:10:58.676356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:13:34.004100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033219+0000","last_change":"2024-03-30T13:10:57.682984+0000","last_active":"2024-03-30T13:10:59.033219+0000","last_peered":"2024-03-30T13:10:59.033219+0000","last_clean":"2024-03-30T13:10:59.033219+0000","last_became_active":"2024-03-30T13:10:57.682685+0000","last_became_peered":"2024-03-30T13:10:57.682685+0000","last_unstale":"2024-03-30T13:10:59.033219+0000","last_undegraded":"2024-03-30T13:10:59.033219+0000","last_fullsized":"2024-03-30T13:10:59.033219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:28:51.189027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.746047+0000","last_change":"2024-03-30T13:10:56.673822+0000","last_active":"2024-03-30T13:10:59.746047+0000","last_peered":"2024-03-30T13:10:59.746047+0000","last_clean":"2024-03-30T13:10:59.746047+0000","last_became_active":"2024-03-30T13:10:56.673533+0000","last_became_peered":"2024-03-30T13:10:56.673533+0000","last_unstale":"2024-03-30T13:10:59.746047+0000","last_undegraded":"2024-03-30T13:10:59.746047+0000","last_fullsized":"2024-03-30T13:10:59.746047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:51:29.821157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676839+0000","last_change":"2024-03-30T13:10:57.688749+0000","last_active":"2024-03-30T13:10:58.676839+0000","last_peered":"2024-03-30T13:10:58.676839+0000","last_clean":"2024-03-30T13:10:58.676839+0000","last_became_active":"2024-03-30T13:10:57.687457+0000","last_became_peered":"2024-03-30T13:10:57.687457+0000","last_unstale":"2024-03-30T13:10:58.676839+0000","last_undegraded":"2024-03-30T13:10:58.676839+0000","last_fullsized":"2024-03-30T13:10:58.676839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:42.131116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745498+0000","last_change":"2024-03-30T13:10:56.673044+0000","last_active":"2024-03-30T13:10:59.745498+0000","last_peered":"2024-03-30T13:10:59.745498+0000","last_clean":"2024-03-30T13:10:59.745498+0000","last_became_active":"2024-03-30T13:10:56.672750+0000","last_became_peered":"2024-03-30T13:10:56.672750+0000","last_unstale":"2024-03-30T13:10:59.745498+0000","last_undegraded":"2024-03-30T13:10:59.745498+0000","last_fullsized":"2024-03-30T13:10:59.745498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672861+0000","last_change":"2024-03-30T13:10:57.685414+0000","last_active":"2024-03-30T13:10:58.672861+0000","last_peered":"2024-03-30T13:10:58.672861+0000","last_clean":"2024-03-30T13:10:58.672861+0000","last_became_active":"2024-03-30T13:10:57.685131+0000","last_became_peered":"2024-03-30T13:10:57.685131+0000","last_unstale":"2024-03-30T13:10:58.672861+0000","last_undegraded":"2024-03-30T13:10:58.672861+0000","last_fullsized":"2024-03-30T13:10:58.672861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676898+0000","last_change":"2024-03-30T13:10:56.679798+0000","last_active":"2024-03-30T13:10:58.676898+0000","last_peered":"2024-03-30T13:10:58.676898+0000","last_clean":"2024-03-30T13:10:58.676898+0000","last_became_active":"2024-03-30T13:10:56.679500+0000","last_became_peered":"2024-03-30T13:10:56.679500+0000","last_unstale":"2024-03-30T13:10:58.676898+0000","last_undegraded":"2024-03-30T13:10:58.676898+0000","last_fullsized":"2024-03-30T13:10:58.676898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672316+0000","last_change":"2024-03-30T13:10:57.688829+0000","last_active":"2024-03-30T13:10:58.672316+0000","last_peered":"2024-03-30T13:10:58.672316+0000","last_clean":"2024-03-30T13:10:58.672316+0000","last_became_active":"2024-03-30T13:10:57.686054+0000","last_became_peered":"2024-03-30T13:10:57.686054+0000","last_unstale":"2024-03-30T13:10:58.672316+0000","last_undegraded":"2024-03-30T13:10:58.672316+0000","last_fullsized":"2024-03-30T13:10:58.672316+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:03:47.898459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672809+0000","last_change":"2024-03-30T13:10:56.674537+0000","last_active":"2024-03-30T13:10:58.672809+0000","last_peered":"2024-03-30T13:10:58.672809+0000","last_clean":"2024-03-30T13:10:58.672809+0000","last_became_active":"2024-03-30T13:10:56.674248+0000","last_became_peered":"2024-03-30T13:10:56.674248+0000","last_unstale":"2024-03-30T13:10:58.672809+0000","last_undegraded":"2024-03-30T13:10:58.672809+0000","last_fullsized":"2024-03-30T13:10:58.672809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745165+0000","last_change":"2024-03-30T13:10:57.693802+0000","last_active":"2024-03-30T13:10:59.745165+0000","last_peered":"2024-03-30T13:10:59.745165+0000","last_clean":"2024-03-30T13:10:59.745165+0000","last_became_active":"2024-03-30T13:10:57.693436+0000","last_became_peered":"2024-03-30T13:10:57.693436+0000","last_unstale":"2024-03-30T13:10:59.745165+0000","last_undegraded":"2024-03-30T13:10:59.745165+0000","last_fullsized":"2024-03-30T13:10:59.745165+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:27.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681179+0000","last_change":"2024-03-30T13:10:56.681576+0000","last_active":"2024-03-30T13:10:58.681179+0000","last_peered":"2024-03-30T13:10:58.681179+0000","last_clean":"2024-03-30T13:10:58.681179+0000","last_became_active":"2024-03-30T13:10:56.681081+0000","last_became_peered":"2024-03-30T13:10:56.681081+0000","last_unstale":"2024-03-30T13:10:58.681179+0000","last_undegraded":"2024-03-30T13:10:58.681179+0000","last_fullsized":"2024-03-30T13:10:58.681179+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681212+0000","last_change":"2024-03-30T13:10:57.691276+0000","last_active":"2024-03-30T13:10:58.681212+0000","last_peered":"2024-03-30T13:10:58.681212+0000","last_clean":"2024-03-30T13:10:58.681212+0000","last_became_active":"2024-03-30T13:10:57.690846+0000","last_became_peered":"2024-03-30T13:10:57.690846+0000","last_unstale":"2024-03-30T13:10:58.681212+0000","last_undegraded":"2024-03-30T13:10:58.681212+0000","last_fullsized":"2024-03-30T13:10:58.681212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:29.135500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032969+0000","last_change":"2024-03-30T13:10:56.677670+0000","last_active":"2024-03-30T13:10:59.032969+0000","last_peered":"2024-03-30T13:10:59.032969+0000","last_clean":"2024-03-30T13:10:59.032969+0000","last_became_active":"2024-03-30T13:10:56.677367+0000","last_became_peered":"2024-03-30T13:10:56.677367+0000","last_unstale":"2024-03-30T13:10:59.032969+0000","last_undegraded":"2024-03-30T13:10:59.032969+0000","last_fullsized":"2024-03-30T13:10:59.032969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:14:10.008446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672971+0000","last_change":"2024-03-30T13:10:57.681840+0000","last_active":"2024-03-30T13:10:58.672971+0000","last_peered":"2024-03-30T13:10:58.672971+0000","last_clean":"2024-03-30T13:10:58.672971+0000","last_became_active":"2024-03-30T13:10:57.681548+0000","last_became_peered":"2024-03-30T13:10:57.681548+0000","last_unstale":"2024-03-30T13:10:58.672971+0000","last_undegraded":"2024-03-30T13:10:58.672971+0000","last_fullsized":"2024-03-30T13:10:58.672971+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676449+0000","last_change":"2024-03-30T13:10:56.682810+0000","last_active":"2024-03-30T13:10:58.676449+0000","last_peered":"2024-03-30T13:10:58.676449+0000","last_clean":"2024-03-30T13:10:58.676449+0000","last_became_active":"2024-03-30T13:10:56.682437+0000","last_became_peered":"2024-03-30T13:10:56.682437+0000","last_unstale":"2024-03-30T13:10:58.676449+0000","last_undegraded":"2024-03-30T13:10:58.676449+0000","last_fullsized":"2024-03-30T13:10:58.676449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672038+0000","last_change":"2024-03-30T13:10:57.678420+0000","last_active":"2024-03-30T13:10:58.672038+0000","last_peered":"2024-03-30T13:10:58.672038+0000","last_clean":"2024-03-30T13:10:58.672038+0000","last_became_active":"2024-03-30T13:10:57.678115+0000","last_became_peered":"2024-03-30T13:10:57.678115+0000","last_unstale":"2024-03-30T13:10:58.672038+0000","last_undegraded":"2024-03-30T13:10:58.672038+0000","last_fullsized":"2024-03-30T13:10:58.672038+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:42:22.714841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672311+0000","last_change":"2024-03-30T13:10:56.678877+0000","last_active":"2024-03-30T13:10:58.672311+0000","last_peered":"2024-03-30T13:10:58.672311+0000","last_clean":"2024-03-30T13:10:58.672311+0000","last_became_active":"2024-03-30T13:10:56.678573+0000","last_became_peered":"2024-03-30T13:10:56.678573+0000","last_unstale":"2024-03-30T13:10:58.672311+0000","last_undegraded":"2024-03-30T13:10:58.672311+0000","last_fullsized":"2024-03-30T13:10:58.672311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678784+0000","last_change":"2024-03-30T13:10:57.685526+0000","last_active":"2024-03-30T13:10:58.678784+0000","last_peered":"2024-03-30T13:10:58.678784+0000","last_clean":"2024-03-30T13:10:58.678784+0000","last_became_active":"2024-03-30T13:10:57.681079+0000","last_became_peered":"2024-03-30T13:10:57.681079+0000","last_unstale":"2024-03-30T13:10:58.678784+0000","last_undegraded":"2024-03-30T13:10:58.678784+0000","last_fullsized":"2024-03-30T13:10:58.678784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:36:15.323960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672394+0000","last_change":"2024-03-30T13:10:56.680735+0000","last_active":"2024-03-30T13:10:58.672394+0000","last_peered":"2024-03-30T13:10:58.672394+0000","last_clean":"2024-03-30T13:10:58.672394+0000","last_became_active":"2024-03-30T13:10:56.680459+0000","last_became_peered":"2024-03-30T13:10:56.680459+0000","last_unstale":"2024-03-30T13:10:58.672394+0000","last_undegraded":"2024-03-30T13:10:58.672394+0000","last_fullsized":"2024-03-30T13:10:58.672394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:48:26.601466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033176+0000","last_change":"2024-03-30T13:10:57.687091+0000","last_active":"2024-03-30T13:10:59.033176+0000","last_peered":"2024-03-30T13:10:59.033176+0000","last_clean":"2024-03-30T13:10:59.033176+0000","last_became_active":"2024-03-30T13:10:57.686821+0000","last_became_peered":"2024-03-30T13:10:57.686821+0000","last_unstale":"2024-03-30T13:10:59.033176+0000","last_undegraded":"2024-03-30T13:10:59.033176+0000","last_fullsized":"2024-03-30T13:10:59.033176+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:16:42.538868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681476+0000","last_change":"2024-03-30T13:10:56.682722+0000","last_active":"2024-03-30T13:10:58.681476+0000","last_peered":"2024-03-30T13:10:58.681476+0000","last_clean":"2024-03-30T13:10:58.681476+0000","last_became_active":"2024-03-30T13:10:56.682272+0000","last_became_peered":"2024-03-30T13:10:56.682272+0000","last_unstale":"2024-03-30T13:10:58.681476+0000","last_undegraded":"2024-03-30T13:10:58.681476+0000","last_fullsized":"2024-03-30T13:10:58.681476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:45:14.325643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745459+0000","last_change":"2024-03-30T13:10:57.691155+0000","last_active":"2024-03-30T13:10:59.745459+0000","last_peered":"2024-03-30T13:10:59.745459+0000","last_clean":"2024-03-30T13:10:59.745459+0000","last_became_active":"2024-03-30T13:10:57.690702+0000","last_became_peered":"2024-03-30T13:10:57.690702+0000","last_unstale":"2024-03-30T13:10:59.745459+0000","last_undegraded":"2024-03-30T13:10:59.745459+0000","last_fullsized":"2024-03-30T13:10:59.745459+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681550+0000","last_change":"2024-03-30T13:10:56.683050+0000","last_active":"2024-03-30T13:10:58.681550+0000","last_peered":"2024-03-30T13:10:58.681550+0000","last_clean":"2024-03-30T13:10:58.681550+0000","last_became_active":"2024-03-30T13:10:56.682514+0000","last_became_peered":"2024-03-30T13:10:56.682514+0000","last_unstale":"2024-03-30T13:10:58.681550+0000","last_undegraded":"2024-03-30T13:10:58.681550+0000","last_fullsized":"2024-03-30T13:10:58.681550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:37:04.571137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672597+0000","last_change":"2024-03-30T13:10:57.692928+0000","last_active":"2024-03-30T13:10:58.672597+0000","last_peered":"2024-03-30T13:10:58.672597+0000","last_clean":"2024-03-30T13:10:58.672597+0000","last_became_active":"2024-03-30T13:10:57.692672+0000","last_became_peered":"2024-03-30T13:10:57.692672+0000","last_unstale":"2024-03-30T13:10:58.672597+0000","last_undegraded":"2024-03-30T13:10:58.672597+0000","last_fullsized":"2024-03-30T13:10:58.672597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672248+0000","last_change":"2024-03-30T13:10:56.676836+0000","last_active":"2024-03-30T13:10:58.672248+0000","last_peered":"2024-03-30T13:10:58.672248+0000","last_clean":"2024-03-30T13:10:58.672248+0000","last_became_active":"2024-03-30T13:10:56.676586+0000","last_became_peered":"2024-03-30T13:10:56.676586+0000","last_unstale":"2024-03-30T13:10:58.672248+0000","last_undegraded":"2024-03-30T13:10:58.672248+0000","last_fullsized":"2024-03-30T13:10:58.672248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:15.014930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745111+0000","last_change":"2024-03-30T13:10:57.690831+0000","last_active":"2024-03-30T13:10:59.745111+0000","last_peered":"2024-03-30T13:10:59.745111+0000","last_clean":"2024-03-30T13:10:59.745111+0000","last_became_active":"2024-03-30T13:10:57.690522+0000","last_became_peered":"2024-03-30T13:10:57.690522+0000","last_unstale":"2024-03-30T13:10:59.745111+0000","last_undegraded":"2024-03-30T13:10:59.745111+0000","last_fullsized":"2024-03-30T13:10:59.745111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:04:16.019984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672474+0000","last_change":"2024-03-30T13:10:56.676370+0000","last_active":"2024-03-30T13:10:58.672474+0000","last_peered":"2024-03-30T13:10:58.672474+0000","last_clean":"2024-03-30T13:10:58.672474+0000","last_became_active":"2024-03-30T13:10:56.675812+0000","last_became_peered":"2024-03-30T13:10:56.675812+0000","last_unstale":"2024-03-30T13:10:58.672474+0000","last_undegraded":"2024-03-30T13:10:58.672474+0000","last_fullsized":"2024-03-30T13:10:58.672474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:24.867217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671859+0000","last_change":"2024-03-30T13:10:57.684924+0000","last_active":"2024-03-30T13:10:58.671859+0000","last_peered":"2024-03-30T13:10:58.671859+0000","last_clean":"2024-03-30T13:10:58.671859+0000","last_became_active":"2024-03-30T13:10:57.684588+0000","last_became_peered":"2024-03-30T13:10:57.684588+0000","last_unstale":"2024-03-30T13:10:58.671859+0000","last_undegraded":"2024-03-30T13:10:58.671859+0000","last_fullsized":"2024-03-30T13:10:58.671859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:20:39.043512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676374+0000","last_change":"2024-03-30T13:10:56.683571+0000","last_active":"2024-03-30T13:10:58.676374+0000","last_peered":"2024-03-30T13:10:58.676374+0000","last_clean":"2024-03-30T13:10:58.676374+0000","last_became_active":"2024-03-30T13:10:56.683215+0000","last_became_peered":"2024-03-30T13:10:56.683215+0000","last_unstale":"2024-03-30T13:10:58.676374+0000","last_undegraded":"2024-03-30T13:10:58.676374+0000","last_fullsized":"2024-03-30T13:10:58.676374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033087+0000","last_change":"2024-03-30T13:10:57.690221+0000","last_active":"2024-03-30T13:10:59.033087+0000","last_peered":"2024-03-30T13:10:59.033087+0000","last_clean":"2024-03-30T13:10:59.033087+0000","last_became_active":"2024-03-30T13:10:57.689942+0000","last_became_peered":"2024-03-30T13:10:57.689942+0000","last_unstale":"2024-03-30T13:10:59.033087+0000","last_undegraded":"2024-03-30T13:10:59.033087+0000","last_fullsized":"2024-03-30T13:10:59.033087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:42:54.504494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745733+0000","last_change":"2024-03-30T13:10:56.674381+0000","last_active":"2024-03-30T13:10:59.745733+0000","last_peered":"2024-03-30T13:10:59.745733+0000","last_clean":"2024-03-30T13:10:59.745733+0000","last_became_active":"2024-03-30T13:10:56.674086+0000","last_became_peered":"2024-03-30T13:10:56.674086+0000","last_unstale":"2024-03-30T13:10:59.745733+0000","last_undegraded":"2024-03-30T13:10:59.745733+0000","last_fullsized":"2024-03-30T13:10:59.745733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:02.130520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672207+0000","last_change":"2024-03-30T13:10:57.689668+0000","last_active":"2024-03-30T13:10:58.672207+0000","last_peered":"2024-03-30T13:10:58.672207+0000","last_clean":"2024-03-30T13:10:58.672207+0000","last_became_active":"2024-03-30T13:10:57.689363+0000","last_became_peered":"2024-03-30T13:10:57.689363+0000","last_unstale":"2024-03-30T13:10:58.672207+0000","last_undegraded":"2024-03-30T13:10:58.672207+0000","last_fullsized":"2024-03-30T13:10:58.672207+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:25.435052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032876+0000","last_change":"2024-03-30T13:10:56.676925+0000","last_active":"2024-03-30T13:10:59.032876+0000","last_peered":"2024-03-30T13:10:59.032876+0000","last_clean":"2024-03-30T13:10:59.032876+0000","last_became_active":"2024-03-30T13:10:56.676627+0000","last_became_peered":"2024-03-30T13:10:56.676627+0000","last_unstale":"2024-03-30T13:10:59.032876+0000","last_undegraded":"2024-03-30T13:10:59.032876+0000","last_fullsized":"2024-03-30T13:10:59.032876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:26:49.695600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672078+0000","last_change":"2024-03-30T13:10:57.686177+0000","last_active":"2024-03-30T13:10:58.672078+0000","last_peered":"2024-03-30T13:10:58.672078+0000","last_clean":"2024-03-30T13:10:58.672078+0000","last_became_active":"2024-03-30T13:10:57.685857+0000","last_became_peered":"2024-03-30T13:10:57.685857+0000","last_unstale":"2024-03-30T13:10:58.672078+0000","last_undegraded":"2024-03-30T13:10:58.672078+0000","last_fullsized":"2024-03-30T13:10:58.672078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:28.503626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681448+0000","last_change":"2024-03-30T13:10:56.681832+0000","last_active":"2024-03-30T13:10:58.681448+0000","last_peered":"2024-03-30T13:10:58.681448+0000","last_clean":"2024-03-30T13:10:58.681448+0000","last_became_active":"2024-03-30T13:10:56.681522+0000","last_became_peered":"2024-03-30T13:10:56.681522+0000","last_unstale":"2024-03-30T13:10:58.681448+0000","last_undegraded":"2024-03-30T13:10:58.681448+0000","last_fullsized":"2024-03-30T13:10:58.681448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:30:46.595017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681440+0000","last_change":"2024-03-30T13:10:57.693544+0000","last_active":"2024-03-30T13:10:58.681440+0000","last_peered":"2024-03-30T13:10:58.681440+0000","last_clean":"2024-03-30T13:10:58.681440+0000","last_became_active":"2024-03-30T13:10:57.693276+0000","last_became_peered":"2024-03-30T13:10:57.693276+0000","last_unstale":"2024-03-30T13:10:58.681440+0000","last_undegraded":"2024-03-30T13:10:58.681440+0000","last_fullsized":"2024-03-30T13:10:58.681440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:31.858312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672976+0000","last_change":"2024-03-30T13:10:56.681831+0000","last_active":"2024-03-30T13:10:58.672976+0000","last_peered":"2024-03-30T13:10:58.672976+0000","last_clean":"2024-03-30T13:10:58.672976+0000","last_became_active":"2024-03-30T13:10:56.681581+0000","last_became_peered":"2024-03-30T13:10:56.681581+0000","last_unstale":"2024-03-30T13:10:58.672976+0000","last_undegraded":"2024-03-30T13:10:58.672976+0000","last_fullsized":"2024-03-30T13:10:58.672976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:54:54.818674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.673095+0000","last_change":"2024-03-30T13:10:57.689473+0000","last_active":"2024-03-30T13:10:58.673095+0000","last_peered":"2024-03-30T13:10:58.673095+0000","last_clean":"2024-03-30T13:10:58.673095+0000","last_became_active":"2024-03-30T13:10:57.689184+0000","last_became_peered":"2024-03-30T13:10:57.689184+0000","last_unstale":"2024-03-30T13:10:58.673095+0000","last_undegraded":"2024-03-30T13:10:58.673095+0000","last_fullsized":"2024-03-30T13:10:58.673095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678850+0000","last_change":"2024-03-30T13:10:56.671520+0000","last_active":"2024-03-30T13:10:58.678850+0000","last_peered":"2024-03-30T13:10:58.678850+0000","last_clean":"2024-03-30T13:10:58.678850+0000","last_became_active":"2024-03-30T13:10:56.671198+0000","last_became_peered":"2024-03-30T13:10:56.671198+0000","last_unstale":"2024-03-30T13:10:58.678850+0000","last_undegraded":"2024-03-30T13:10:58.678850+0000","last_fullsized":"2024-03-30T13:10:58.678850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:48:29.323071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.679016+0000","last_change":"2024-03-30T13:10:57.682760+0000","last_active":"2024-03-30T13:10:58.679016+0000","last_peered":"2024-03-30T13:10:58.679016+0000","last_clean":"2024-03-30T13:10:58.679016+0000","last_became_active":"2024-03-30T13:10:57.682495+0000","last_became_peered":"2024-03-30T13:10:57.682495+0000","last_unstale":"2024-03-30T13:10:58.679016+0000","last_undegraded":"2024-03-30T13:10:58.679016+0000","last_fullsized":"2024-03-30T13:10:58.679016+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:03:14.415400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676601+0000","last_change":"2024-03-30T13:10:56.679764+0000","last_active":"2024-03-30T13:10:58.676601+0000","last_peered":"2024-03-30T13:10:58.676601+0000","last_clean":"2024-03-30T13:10:58.676601+0000","last_became_active":"2024-03-30T13:10:56.679483+0000","last_became_peered":"2024-03-30T13:10:56.679483+0000","last_unstale":"2024-03-30T13:10:58.676601+0000","last_undegraded":"2024-03-30T13:10:58.676601+0000","last_fullsized":"2024-03-30T13:10:58.676601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:38.551550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033042+0000","last_change":"2024-03-30T13:10:57.693368+0000","last_active":"2024-03-30T13:10:59.033042+0000","last_peered":"2024-03-30T13:10:59.033042+0000","last_clean":"2024-03-30T13:10:59.033042+0000","last_became_active":"2024-03-30T13:10:57.693076+0000","last_became_peered":"2024-03-30T13:10:57.693076+0000","last_unstale":"2024-03-30T13:10:59.033042+0000","last_undegraded":"2024-03-30T13:10:59.033042+0000","last_fullsized":"2024-03-30T13:10:59.033042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T01:10:46.626224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678626+0000","last_change":"2024-03-30T13:10:56.677591+0000","last_active":"2024-03-30T13:10:58.678626+0000","last_peered":"2024-03-30T13:10:58.678626+0000","last_clean":"2024-03-30T13:10:58.678626+0000","last_became_active":"2024-03-30T13:10:56.677289+0000","last_became_peered":"2024-03-30T13:10:56.677289+0000","last_unstale":"2024-03-30T13:10:58.678626+0000","last_undegraded":"2024-03-30T13:10:58.678626+0000","last_fullsized":"2024-03-30T13:10:58.678626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:55:48.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745181+0000","last_change":"2024-03-30T13:10:57.686202+0000","last_active":"2024-03-30T13:10:59.745181+0000","last_peered":"2024-03-30T13:10:59.745181+0000","last_clean":"2024-03-30T13:10:59.745181+0000","last_became_active":"2024-03-30T13:10:57.685942+0000","last_became_peered":"2024-03-30T13:10:57.685942+0000","last_unstale":"2024-03-30T13:10:59.745181+0000","last_undegraded":"2024-03-30T13:10:59.745181+0000","last_fullsized":"2024-03-30T13:10:59.745181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:56:25.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678579+0000","last_change":"2024-03-30T13:10:56.677268+0000","last_active":"2024-03-30T13:10:58.678579+0000","last_peered":"2024-03-30T13:10:58.678579+0000","last_clean":"2024-03-30T13:10:58.678579+0000","last_became_active":"2024-03-30T13:10:56.676985+0000","last_became_peered":"2024-03-30T13:10:56.676985+0000","last_unstale":"2024-03-30T13:10:58.678579+0000","last_undegraded":"2024-03-30T13:10:58.678579+0000","last_fullsized":"2024-03-30T13:10:58.678579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:31:09.416890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681045+0000","last_change":"2024-03-30T13:10:57.686046+0000","last_active":"2024-03-30T13:10:58.681045+0000","last_peered":"2024-03-30T13:10:58.681045+0000","last_clean":"2024-03-30T13:10:58.681045+0000","last_became_active":"2024-03-30T13:10:57.685805+0000","last_became_peered":"2024-03-30T13:10:57.685805+0000","last_unstale":"2024-03-30T13:10:58.681045+0000","last_undegraded":"2024-03-30T13:10:58.681045+0000","last_fullsized":"2024-03-30T13:10:58.681045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:21:38.368777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033098+0000","last_change":"2024-03-30T13:10:56.680608+0000","last_active":"2024-03-30T13:10:59.033098+0000","last_peered":"2024-03-30T13:10:59.033098+0000","last_clean":"2024-03-30T13:10:59.033098+0000","last_became_active":"2024-03-30T13:10:56.680346+0000","last_became_peered":"2024-03-30T13:10:56.680346+0000","last_unstale":"2024-03-30T13:10:59.033098+0000","last_undegraded":"2024-03-30T13:10:59.033098+0000","last_fullsized":"2024-03-30T13:10:59.033098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672201+0000","last_change":"2024-03-30T13:10:57.678124+0000","last_active":"2024-03-30T13:10:58.672201+0000","last_peered":"2024-03-30T13:10:58.672201+0000","last_clean":"2024-03-30T13:10:58.672201+0000","last_became_active":"2024-03-30T13:10:57.677826+0000","last_became_peered":"2024-03-30T13:10:57.677826+0000","last_unstale":"2024-03-30T13:10:58.672201+0000","last_undegraded":"2024-03-30T13:10:58.672201+0000","last_fullsized":"2024-03-30T13:10:58.672201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:45:37.076824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.678461+0000","last_change":"2024-03-30T13:10:56.678610+0000","last_active":"2024-03-30T13:10:58.678461+0000","last_peered":"2024-03-30T13:10:58.678461+0000","last_clean":"2024-03-30T13:10:58.678461+0000","last_became_active":"2024-03-30T13:10:56.678357+0000","last_became_peered":"2024-03-30T13:10:56.678357+0000","last_unstale":"2024-03-30T13:10:58.678461+0000","last_undegraded":"2024-03-30T13:10:58.678461+0000","last_fullsized":"2024-03-30T13:10:58.678461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:40.674849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676600+0000","last_change":"2024-03-30T13:10:57.687196+0000","last_active":"2024-03-30T13:10:58.676600+0000","last_peered":"2024-03-30T13:10:58.676600+0000","last_clean":"2024-03-30T13:10:58.676600+0000","last_became_active":"2024-03-30T13:10:57.686683+0000","last_became_peered":"2024-03-30T13:10:57.686683+0000","last_unstale":"2024-03-30T13:10:58.676600+0000","last_undegraded":"2024-03-30T13:10:58.676600+0000","last_fullsized":"2024-03-30T13:10:58.676600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:44:15.354797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681577+0000","last_change":"2024-03-30T13:10:56.682787+0000","last_active":"2024-03-30T13:10:58.681577+0000","last_peered":"2024-03-30T13:10:58.681577+0000","last_clean":"2024-03-30T13:10:58.681577+0000","last_became_active":"2024-03-30T13:10:56.682536+0000","last_became_peered":"2024-03-30T13:10:56.682536+0000","last_unstale":"2024-03-30T13:10:58.681577+0000","last_undegraded":"2024-03-30T13:10:58.681577+0000","last_fullsized":"2024-03-30T13:10:58.681577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:28:07.167637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033243+0000","last_change":"2024-03-30T13:10:57.686653+0000","last_active":"2024-03-30T13:10:59.033243+0000","last_peered":"2024-03-30T13:10:59.033243+0000","last_clean":"2024-03-30T13:10:59.033243+0000","last_became_active":"2024-03-30T13:10:57.686380+0000","last_became_peered":"2024-03-30T13:10:57.686380+0000","last_unstale":"2024-03-30T13:10:59.033243+0000","last_undegraded":"2024-03-30T13:10:59.033243+0000","last_fullsized":"2024-03-30T13:10:59.033243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:13:33.282086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.680418+0000","last_change":"2024-03-30T13:10:56.682042+0000","last_active":"2024-03-30T13:10:58.680418+0000","last_peered":"2024-03-30T13:10:58.680418+0000","last_clean":"2024-03-30T13:10:58.680418+0000","last_became_active":"2024-03-30T13:10:56.681748+0000","last_became_peered":"2024-03-30T13:10:56.681748+0000","last_unstale":"2024-03-30T13:10:58.680418+0000","last_undegraded":"2024-03-30T13:10:58.680418+0000","last_fullsized":"2024-03-30T13:10:58.680418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:55:55.240936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672322+0000","last_change":"2024-03-30T13:10:57.676679+0000","last_active":"2024-03-30T13:10:58.672322+0000","last_peered":"2024-03-30T13:10:58.672322+0000","last_clean":"2024-03-30T13:10:58.672322+0000","last_became_active":"2024-03-30T13:10:57.675136+0000","last_became_peered":"2024-03-30T13:10:57.675136+0000","last_unstale":"2024-03-30T13:10:58.672322+0000","last_undegraded":"2024-03-30T13:10:58.672322+0000","last_fullsized":"2024-03-30T13:10:58.672322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:01:31.125961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745366+0000","last_change":"2024-03-30T13:10:56.682275+0000","last_active":"2024-03-30T13:10:59.745366+0000","last_peered":"2024-03-30T13:10:59.745366+0000","last_clean":"2024-03-30T13:10:59.745366+0000","last_became_active":"2024-03-30T13:10:56.681990+0000","last_became_peered":"2024-03-30T13:10:56.681990+0000","last_unstale":"2024-03-30T13:10:59.745366+0000","last_undegraded":"2024-03-30T13:10:59.745366+0000","last_fullsized":"2024-03-30T13:10:59.745366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:44:01.447947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745290+0000","last_change":"2024-03-30T13:10:57.692363+0000","last_active":"2024-03-30T13:10:59.745290+0000","last_peered":"2024-03-30T13:10:59.745290+0000","last_clean":"2024-03-30T13:10:59.745290+0000","last_became_active":"2024-03-30T13:10:57.692063+0000","last_became_peered":"2024-03-30T13:10:57.692063+0000","last_unstale":"2024-03-30T13:10:59.745290+0000","last_undegraded":"2024-03-30T13:10:59.745290+0000","last_fullsized":"2024-03-30T13:10:59.745290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:14:59.847747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745367+0000","last_change":"2024-03-30T13:10:56.680716+0000","last_active":"2024-03-30T13:10:59.745367+0000","last_peered":"2024-03-30T13:10:59.745367+0000","last_clean":"2024-03-30T13:10:59.745367+0000","last_became_active":"2024-03-30T13:10:56.680446+0000","last_became_peered":"2024-03-30T13:10:56.680446+0000","last_unstale":"2024-03-30T13:10:59.745367+0000","last_undegraded":"2024-03-30T13:10:59.745367+0000","last_fullsized":"2024-03-30T13:10:59.745367+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:05:37.608197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.676703+0000","last_change":"2024-03-30T13:10:57.684334+0000","last_active":"2024-03-30T13:10:58.676703+0000","last_peered":"2024-03-30T13:10:58.676703+0000","last_clean":"2024-03-30T13:10:58.676703+0000","last_became_active":"2024-03-30T13:10:57.684037+0000","last_became_peered":"2024-03-30T13:10:57.684037+0000","last_unstale":"2024-03-30T13:10:58.676703+0000","last_undegraded":"2024-03-30T13:10:58.676703+0000","last_fullsized":"2024-03-30T13:10:58.676703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:07:21.631648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.745812+0000","last_change":"2024-03-30T13:10:56.676202+0000","last_active":"2024-03-30T13:10:59.745812+0000","last_peered":"2024-03-30T13:10:59.745812+0000","last_clean":"2024-03-30T13:10:59.745812+0000","last_became_active":"2024-03-30T13:10:56.675907+0000","last_became_peered":"2024-03-30T13:10:56.675907+0000","last_unstale":"2024-03-30T13:10:59.745812+0000","last_undegraded":"2024-03-30T13:10:59.745812+0000","last_fullsized":"2024-03-30T13:10:59.745812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:58.975900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672618+0000","last_change":"2024-03-30T13:10:57.673049+0000","last_active":"2024-03-30T13:10:58.672618+0000","last_peered":"2024-03-30T13:10:58.672618+0000","last_clean":"2024-03-30T13:10:58.672618+0000","last_became_active":"2024-03-30T13:10:57.672738+0000","last_became_peered":"2024-03-30T13:10:57.672738+0000","last_unstale":"2024-03-30T13:10:58.672618+0000","last_undegraded":"2024-03-30T13:10:58.672618+0000","last_fullsized":"2024-03-30T13:10:58.672618+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:37:39.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.672426+0000","last_change":"2024-03-30T13:10:56.675234+0000","last_active":"2024-03-30T13:10:58.672426+0000","last_peered":"2024-03-30T13:10:58.672426+0000","last_clean":"2024-03-30T13:10:58.672426+0000","last_became_active":"2024-03-30T13:10:56.674965+0000","last_became_peered":"2024-03-30T13:10:56.674965+0000","last_unstale":"2024-03-30T13:10:58.672426+0000","last_undegraded":"2024-03-30T13:10:58.672426+0000","last_fullsized":"2024-03-30T13:10:58.672426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:18:35.628852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033158+0000","last_change":"2024-03-30T13:10:57.683624+0000","last_active":"2024-03-30T13:10:59.033158+0000","last_peered":"2024-03-30T13:10:59.033158+0000","last_clean":"2024-03-30T13:10:59.033158+0000","last_became_active":"2024-03-30T13:10:57.683346+0000","last_became_peered":"2024-03-30T13:10:57.683346+0000","last_unstale":"2024-03-30T13:10:59.033158+0000","last_undegraded":"2024-03-30T13:10:59.033158+0000","last_fullsized":"2024-03-30T13:10:59.033158+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:41:02.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.671837+0000","last_change":"2024-03-30T13:10:56.682711+0000","last_active":"2024-03-30T13:10:58.671837+0000","last_peered":"2024-03-30T13:10:58.671837+0000","last_clean":"2024-03-30T13:10:58.671837+0000","last_became_active":"2024-03-30T13:10:56.682415+0000","last_became_peered":"2024-03-30T13:10:56.682415+0000","last_unstale":"2024-03-30T13:10:58.671837+0000","last_undegraded":"2024-03-30T13:10:58.671837+0000","last_fullsized":"2024-03-30T13:10:58.671837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:10:34.096760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:58.681340+0000","last_change":"2024-03-30T13:10:57.684219+0000","last_active":"2024-03-30T13:10:58.681340+0000","last_peered":"2024-03-30T13:10:58.681340+0000","last_clean":"2024-03-30T13:10:58.681340+0000","last_became_active":"2024-03-30T13:10:57.683925+0000","last_became_peered":"2024-03-30T13:10:57.683925+0000","last_unstale":"2024-03-30T13:10:58.681340+0000","last_undegraded":"2024-03-30T13:10:58.681340+0000","last_fullsized":"2024-03-30T13:10:58.681340+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:56.651517+0000","last_clean_scrub_stamp":"2024-03-30T13:10:56.651517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:53:48.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.033155+0000","last_change":"2024-03-30T13:10:56.681620+0000","last_active":"2024-03-30T13:10:59.033155+0000","last_peered":"2024-03-30T13:10:59.033155+0000","last_clean":"2024-03-30T13:10:59.033155+0000","last_became_active":"2024-03-30T13:10:56.681341+0000","last_became_peered":"2024-03-30T13:10:56.681341+0000","last_unstale":"2024-03-30T13:10:59.033155+0000","last_undegraded":"2024-03-30T13:10:59.033155+0000","last_fullsized":"2024-03-30T13:10:59.033155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:55.646463+0000","last_clean_scrub_stamp":"2024-03-30T13:10:55.646463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:46:52.577909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-30T13:10:59.032961+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:10:59.032961+0000","last_peered":"2024-03-30T13:10:59.032961+0000","last_clean":"2024-03-30T13:10:59.032961+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:10:59.032961+0000","last_undegraded":"2024-03-30T13:10:59.032961+0000","last_fullsized":"2024-03-30T13:10:59.032961+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":199796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":199818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1040384,"data_stored":782924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":199284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":199304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":462848,"data_stored":199326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:06.606 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T13:11:06.606 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T13:11:06.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.752+0000 7f680cba8640 1 Processor -- start 2024-03-30T13:11:06.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.752+0000 7f680cba8640 1 -- start start 2024-03-30T13:11:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680cba8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f680815e4f0 0x7f68080b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680cba8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68080d2e40 con 0x7f68080b6520 2024-03-30T13:11:06.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680cba8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68080d3120 con 0x7f68080d3ee0 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680cba8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f68080d3a50 con 0x7f680815e4f0 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680659c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f680815e4f0 0x7f68080b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f6806d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f68080b6520 0x7f68080b74c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60706/0 (socket says 172.21.15.66:60706) 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f6806d9d640 1 -- 172.21.15.66:0/1733184034 learned_addr learned my addr 172.21.15.66:0/1733184034 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680559a640 1 -- 172.21.15.66:0/1733184034 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039982496 0 0) 0x7f68080d2e40 con 0x7f68080b6520 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680559a640 1 -- 172.21.15.66:0/1733184034 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67f4003520 con 0x7f68080b6520 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680659c640 1 -- 172.21.15.66:0/1733184034 >> v1:172.21.15.117:6789/0 conn(0x7f68080d3ee0 legacy=0x7f68080b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680659c640 1 -- 172.21.15.66:0/1733184034 >> v1:172.21.15.66:6789/0 conn(0x7f68080b6520 legacy=0x7f68080b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.753+0000 7f680659c640 1 -- 172.21.15.66:0/1733184034 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68080d3400 con 0x7f680815e4f0 2024-03-30T13:11:06.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680659c640 1 --2- 172.21.15.66:0/1733184034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f680815e4f0 0x7f68080b9f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67f80097a0 tx=0x7f67f802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a009890ba3e5312c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680559a640 1 -- 172.21.15.66:0/1733184034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f802f810 con 0x7f680815e4f0 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680559a640 1 -- 172.21.15.66:0/1733184034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f802f9b0 con 0x7f680815e4f0 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680559a640 1 -- 172.21.15.66:0/1733184034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f802fd80 con 0x7f680815e4f0 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680cba8640 1 -- 172.21.15.66:0/1733184034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f680815e4f0 msgr2=0x7f68080b9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680cba8640 1 --2- 172.21.15.66:0/1733184034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f680815e4f0 0x7f68080b9f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67f80097a0 tx=0x7f67f802b9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680cba8640 1 -- 172.21.15.66:0/1733184034 shutdown_connections 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680cba8640 1 --2- 172.21.15.66:0/1733184034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f680815e4f0 0x7f68080b9f20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.755+0000 7f680cba8640 1 -- 172.21.15.66:0/1733184034 >> 172.21.15.66:0/1733184034 conn(0x7f68080678f0 msgr2=0x7f68080d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.756+0000 7f680cba8640 1 -- 172.21.15.66:0/1733184034 shutdown_connections 2024-03-30T13:11:06.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.756+0000 7f680cba8640 1 -- 172.21.15.66:0/1733184034 wait complete. 2024-03-30T13:11:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.757+0000 7f680cba8640 1 Processor -- start 2024-03-30T13:11:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.757+0000 7f680cba8640 1 -- start start 2024-03-30T13:11:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 0x7f680814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080d3ee0 0x7f68080cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f680815e4f0 0x7f68080cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f68080d26f0 con 0x7f68080b6520 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f680815e8d0 con 0x7f680815e4f0 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f68080d2870 con 0x7f68080d3ee0 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 0x7f680814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 0x7f680814fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39604/0 (socket says 172.21.15.66:39604) 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 -- 172.21.15.66:0/1312087253 learned_addr learned my addr 172.21.15.66:0/1312087253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f6806d9d640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f680815e4f0 0x7f68080cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f6805d9b640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080d3ee0 0x7f68080cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 -- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080d3ee0 msgr2=0x7f68080cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080d3ee0 0x7f68080cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 -- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f680815e4f0 msgr2=0x7f68080cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f680815e4f0 0x7f68080cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 -- 172.21.15.66:0/1312087253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68081609f0 con 0x7f68080b6520 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f6806d9d640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f680815e4f0 0x7f68080cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680659c640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 0x7f680814fc00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098c0 tx=0x7f67f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8047070 con 0x7f68080b6520 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f80378d0 con 0x7f68080b6520 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6808160cd0 con 0x7f68080b6520 2024-03-30T13:11:06.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.758+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8037cc0 con 0x7f68080b6520 2024-03-30T13:11:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.759+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6808161290 con 0x7f68080b6520 2024-03-30T13:11:06.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.760+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6808075890 con 0x7f68080b6520 2024-03-30T13:11:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.761+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f67f804a070 con 0x7f68080b6520 2024-03-30T13:11:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.761+0000 7f67feffd640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f67e806c530 0x7f67e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.761+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f67f80ad510 con 0x7f68080b6520 2024-03-30T13:11:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.761+0000 7f6805d9b640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f67e806c530 0x7f67e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.762+0000 7f6805d9b640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f67e806c530 0x7f67e806e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f680815d1d0 tx=0x7f67f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:06.765+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f67f8033030 con 0x7f68080b6520 2024-03-30T13:11:07.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.021+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f680815ba30 con 0x7f68080b6520 2024-03-30T13:11:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.023+0000 7f67feffd640 1 -- 172.21.15.66:0/1312087253 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f67f807b910 con 0x7f68080b6520 2024-03-30T13:11:07.027 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-03-30T13:11:07.027 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQDcDghmupudNBAAFaQaghMGgLisibrdNbX0TQ== 2024-03-30T13:11:07.027 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-03-30T13:11:07.027 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-03-30T13:11:07.027 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-30T13:11:07.028 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-03-30T13:11:07.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.025+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f67e806c530 msgr2=0x7f67e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.025+0000 7f680cba8640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f67e806c530 0x7f67e806e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f680815d1d0 tx=0x7f67f0009040 comp rx=0 tx=0).stop 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.025+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 msgr2=0x7f680814fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.025+0000 7f680cba8640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 0x7f680814fc00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098c0 tx=0x7f67f8004730 comp rx=0 tx=0).stop 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 shutdown_connections 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f67e806c530 0x7f67e806e9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f680815e4f0 0x7f68080cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080d3ee0 0x7f68080cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 --2- 172.21.15.66:0/1312087253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68080b6520 0x7f680814fc00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 >> 172.21.15.66:0/1312087253 conn(0x7f68080678f0 msgr2=0x7f68080b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 shutdown_connections 2024-03-30T13:11:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.026+0000 7f680cba8640 1 -- 172.21.15.66:0/1312087253 wait complete. 2024-03-30T13:11:07.045 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.148+0000 7efd493f0640 1 Processor -- start 2024-03-30T13:11:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.148+0000 7efd493f0640 1 -- start start 2024-03-30T13:11:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd493f0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efd440b79c0 0x7efd44169110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd493f0640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd440b9d20 con 0x7efd4415e4f0 2024-03-30T13:11:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd493f0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd440ba000 con 0x7efd440b7530 2024-03-30T13:11:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd493f0640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efd440ba2e0 con 0x7efd440b79c0 2024-03-30T13:11:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd437fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efd440b79c0 0x7efd44169110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd42ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efd4415e4f0 0x7efd440cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60716/0 (socket says 172.21.15.66:60716) 2024-03-30T13:11:07.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd42ffd640 1 -- 172.21.15.66:0/3601191648 learned_addr learned my addr 172.21.15.66:0/3601191648 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:07.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd437fe640 1 -- 172.21.15.66:0/3601191648 >> v1:172.21.15.117:6789/0 conn(0x7efd440b7530 legacy=0x7efd440d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd437fe640 1 -- 172.21.15.66:0/3601191648 >> v1:172.21.15.66:6789/0 conn(0x7efd4415e4f0 legacy=0x7efd440cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.150+0000 7efd437fe640 1 -- 172.21.15.66:0/3601191648 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd440ceaf0 con 0x7efd440b79c0 2024-03-30T13:11:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.151+0000 7efd437fe640 1 --2- 172.21.15.66:0/3601191648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd440b79c0 0x7efd44169110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009d90 tx=0x7efd3c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64466dec069d3952 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.151+0000 7efd427fc640 1 -- 172.21.15.66:0/3601191648 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c039070 con 0x7efd440b79c0 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.151+0000 7efd427fc640 1 -- 172.21.15.66:0/3601191648 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c02cbd0 con 0x7efd440b79c0 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.151+0000 7efd427fc640 1 -- 172.21.15.66:0/3601191648 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c034440 con 0x7efd440b79c0 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 -- 172.21.15.66:0/3601191648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd440b79c0 msgr2=0x7efd44169110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 --2- 172.21.15.66:0/3601191648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd440b79c0 0x7efd44169110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009d90 tx=0x7efd3c02be20 comp rx=0 tx=0).stop 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 -- 172.21.15.66:0/3601191648 shutdown_connections 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 --2- 172.21.15.66:0/3601191648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd440b79c0 0x7efd44169110 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 -- 172.21.15.66:0/3601191648 >> 172.21.15.66:0/3601191648 conn(0x7efd440676e0 msgr2=0x7efd44067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 -- 172.21.15.66:0/3601191648 shutdown_connections 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.152+0000 7efd493f0640 1 -- 172.21.15.66:0/3601191648 wait complete. 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.153+0000 7efd493f0640 1 Processor -- start 2024-03-30T13:11:07.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.153+0000 7efd493f0640 1 -- start start 2024-03-30T13:11:07.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd493f0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd440b7530 0x7efd44153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd493f0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd440b79c0 0x7efd44155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd493f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 0x7efd44155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd493f0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd440d3240 con 0x7efd4415e4f0 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd493f0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd440d30c0 con 0x7efd440b79c0 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd493f0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efd440d33c0 con 0x7efd440b7530 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 0x7efd44155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 0x7efd44155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39616/0 (socket says 172.21.15.66:39616) 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 -- 172.21.15.66:0/442997359 learned_addr learned my addr 172.21.15.66:0/442997359 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd3bfff640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd440b79c0 0x7efd44155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd42ffd640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd440b7530 0x7efd44153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 -- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd440b7530 msgr2=0x7efd44153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd440b7530 0x7efd44153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 -- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd440b79c0 msgr2=0x7efd44155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd440b79c0 0x7efd44155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 -- 172.21.15.66:0/442997359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd440c8c40 con 0x7efd4415e4f0 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.154+0000 7efd437fe640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 0x7efd44155770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009eb0 tx=0x7efd3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.155+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c039050 con 0x7efd4415e4f0 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.155+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c045070 con 0x7efd4415e4f0 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.155+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c03e850 con 0x7efd4415e4f0 2024-03-30T13:11:07.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.155+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4415fe40 con 0x7efd4415e4f0 2024-03-30T13:11:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.155+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd44160400 con 0x7efd4415e4f0 2024-03-30T13:11:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.156+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd44075a10 con 0x7efd4415e4f0 2024-03-30T13:11:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.156+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd3c03e9f0 con 0x7efd4415e4f0 2024-03-30T13:11:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.157+0000 7efd3b7fe640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd2406c530 0x7efd2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.157+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7efd3c0ae520 con 0x7efd4415e4f0 2024-03-30T13:11:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.157+0000 7efd42ffd640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd2406c530 0x7efd2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.157+0000 7efd42ffd640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd2406c530 0x7efd2406e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd34007fd0 tx=0x7efd34007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.162+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd3c07c3f0 con 0x7efd4415e4f0 2024-03-30T13:11:07.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.386+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd440ce810 con 0x7efd4415e4f0 2024-03-30T13:11:07.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.386+0000 7efd3b7fe640 1 -- 172.21.15.66:0/442997359 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7efd3c0806f0 con 0x7efd4415e4f0 2024-03-30T13:11:07.390 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:57.041663+0000","modified":"2024-03-30T13:11:00.045554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1537031567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1537031567},{"type":"v1","addr":"172.21.15.66:6837","nonce":1537031567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 16 2024-03-30T13:11:07.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd2406c530 msgr2=0x7efd2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd2406c530 0x7efd2406e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd34007fd0 tx=0x7efd34007e80 comp rx=0 tx=0).stop 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 msgr2=0x7efd44155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 0x7efd44155770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009eb0 tx=0x7efd3c004a40 comp rx=0 tx=0).stop 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 shutdown_connections 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd2406c530 0x7efd2406e9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd4415e4f0 0x7efd44155770 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd440b79c0 0x7efd44155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 --2- 172.21.15.66:0/442997359 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd440b7530 0x7efd44153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 >> 172.21.15.66:0/442997359 conn(0x7efd440676e0 msgr2=0x7efd440d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 shutdown_connections 2024-03-30T13:11:07.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.388+0000 7efd493f0640 1 -- 172.21.15.66:0/442997359 wait complete. 2024-03-30T13:11:07.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T13:10:57.041663+0000', 'modified': '2024-03-30T13:11:00.045554+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5382': {'gid': 5382, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6837/1537031567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6835', 'nonce': 1537031567}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 1537031567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5382, 'qdb_cluster': [5382]} 2024-03-30T13:11:07.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T13:11:07.808 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T13:11:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.921+0000 7f6bab959640 1 Processor -- start 2024-03-30T13:11:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.921+0000 7f6bab959640 1 -- start start 2024-03-30T13:11:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6bab959640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ba40cec60 0x7f6ba40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6bab959640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba40d3120 con 0x7f6ba40d3ee0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6bab959640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba40d3400 con 0x7f6ba415e7a0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6bab959640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba40cfc40 con 0x7f6ba40cec60 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6ba9ef6640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6ba40d3ee0 0x7f6ba40b75a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60724/0 (socket says 172.21.15.66:60724) 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6ba9ef6640 1 -- 172.21.15.66:0/2666322339 learned_addr learned my addr 172.21.15.66:0/2666322339 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/2666322339 >> v2:172.21.15.117:3300/0 conn(0x7f6ba40cec60 0x7f6ba40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929092490 0 0) 0x7f6ba40d3120 con 0x7f6ba40d3ee0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.922+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b90003520 con 0x7f6ba40d3ee0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.923+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795294046 0 0) 0x7f6ba40d3400 con 0x7f6ba415e7a0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.923+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ba40d3120 con 0x7f6ba415e7a0 2024-03-30T13:11:07.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.923+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/2666322339 >> v1:172.21.15.117:6789/0 conn(0x7f6ba415e7a0 legacy=0x7f6ba40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.923+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/2666322339 >> v1:172.21.15.66:6789/0 conn(0x7f6ba40d3ee0 legacy=0x7f6ba40b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.923+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2482243459 0 0) 0x7f6b90003520 con 0x7f6ba40d3ee0 2024-03-30T13:11:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.923+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/2666322339 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba40d36e0 con 0x7f6ba40cec60 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/2666322339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba40cec60 0x7f6ba40d12b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00097a0 tx=0x7f6ba002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7451e68588a4b2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0037070 con 0x7f6ba40cec60 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba0030510 con 0x7f6ba40cec60 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6b97fff640 1 -- 172.21.15.66:0/2666322339 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0019f0 con 0x7f6ba40cec60 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 -- 172.21.15.66:0/2666322339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba40cec60 msgr2=0x7f6ba40d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 --2- 172.21.15.66:0/2666322339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba40cec60 0x7f6ba40d12b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00097a0 tx=0x7f6ba002b790 comp rx=0 tx=0).stop 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 -- 172.21.15.66:0/2666322339 shutdown_connections 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 --2- 172.21.15.66:0/2666322339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba40cec60 0x7f6ba40d12b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 -- 172.21.15.66:0/2666322339 >> 172.21.15.66:0/2666322339 conn(0x7f6ba40676b0 msgr2=0x7f6ba4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 -- 172.21.15.66:0/2666322339 shutdown_connections 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.924+0000 7f6bab959640 1 -- 172.21.15.66:0/2666322339 wait complete. 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 Processor -- start 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 -- start start 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 0x7f6ba40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba40d3ee0 0x7f6ba40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4155970 0x7f6ba414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40d1970 con 0x7f6ba40cec60 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40d17f0 con 0x7f6ba4155970 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6bab959640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba40d1af0 con 0x7f6ba40d3ee0 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 0x7f6ba40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.926+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 0x7f6ba40b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39628/0 (socket says 172.21.15.66:39628) 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 -- 172.21.15.66:0/2150129624 learned_addr learned my addr 172.21.15.66:0/2150129624 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba40d3ee0 0x7f6ba40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba9ef6640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4155970 0x7f6ba414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 -- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba40d3ee0 msgr2=0x7f6ba40ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba40d3ee0 0x7f6ba40ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 -- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4155970 msgr2=0x7f6ba414c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4155970 0x7f6ba414c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 -- 172.21.15.66:0/2150129624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4161ac0 con 0x7f6ba40cec60 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba40d3ee0 0x7f6ba40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba9ef6640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4155970 0x7f6ba414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 0x7f6ba40b4eb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c430 tx=0x7f6b9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6b95ffb640 1 -- 172.21.15.66:0/2150129624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98017070 con 0x7f6ba40cec60 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6b95ffb640 1 -- 172.21.15.66:0/2150129624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b980052f0 con 0x7f6ba40cec60 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba40b5600 con 0x7f6ba40cec60 2024-03-30T13:11:07.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6b95ffb640 1 -- 172.21.15.66:0/2150129624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98003830 con 0x7f6ba40cec60 2024-03-30T13:11:07.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.927+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba40b5bc0 con 0x7f6ba40cec60 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.929+0000 7f6b95ffb640 1 -- 172.21.15.66:0/2150129624 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6b9801a070 con 0x7f6ba40cec60 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.929+0000 7f6b95ffb640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b8806c530 0x7f6b8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.929+0000 7f6b95ffb640 1 -- 172.21.15.66:0/2150129624 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f6b9808ad40 con 0x7f6ba40cec60 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.929+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b8806c530 0x7f6b8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.930+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b8806c530 0x7f6b8806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0034000 tx=0x7f6ba002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.932+0000 7f6bae625740 1 Processor -- start 2024-03-30T13:11:07.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.932+0000 7f6bae625740 1 -- start start 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6860 0x55a4ec3c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6bae625740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec3c9d40 0x55a4ec3f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6bae625740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec333020 con 0x55a4ec3f6c40 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec2db5f0 con 0x55a4ec3f6860 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55a4ec32efd0 con 0x55a4ec3c9d40 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6ba8ef4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6ba8ef4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39642/0 (socket says 172.21.15.66:39642) 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/996334020 learned_addr learned my addr 172.21.15.66:0/996334020 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6860 0x55a4ec3c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.933+0000 7f6ba9ef6640 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec3c9d40 0x55a4ec3f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec3c9d40 msgr2=0x55a4ec3f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec3c9d40 0x55a4ec3f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6860 msgr2=0x55a4ec3c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6860 0x55a4ec3c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba8ef4640 1 -- 172.21.15.66:0/996334020 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ec0472d0 con 0x55a4ec3f6c40 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6860 0x55a4ec3c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f95f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009890 tx=0x7f6ba002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef40f5016656fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6b8effd640 1 -- 172.21.15.66:0/996334020 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba004f070 con 0x55a4ec3f6c40 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6b8effd640 1 -- 172.21.15.66:0/996334020 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba003f530 con 0x55a4ec3f6c40 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.934+0000 7f6b8effd640 1 -- 172.21.15.66:0/996334020 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0037040 con 0x55a4ec3f6c40 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 -- 172.21.15.66:0/996334020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 msgr2=0x55a4ec3f95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f95f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009890 tx=0x7f6ba002d000 comp rx=0 tx=0).stop 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 -- 172.21.15.66:0/996334020 shutdown_connections 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec3c9d40 0x55a4ec3f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f95f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 --2- 172.21.15.66:0/996334020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6860 0x55a4ec3c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 -- 172.21.15.66:0/996334020 >> 172.21.15.66:0/996334020 conn(0x55a4ec3cba70 msgr2=0x55a4ec4145f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 -- 172.21.15.66:0/996334020 shutdown_connections 2024-03-30T13:11:07.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 -- 172.21.15.66:0/996334020 wait complete. 2024-03-30T13:11:07.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 -- start start 2024-03-30T13:11:07.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.935+0000 7f6bae625740 1 Processor -- start 2024-03-30T13:11:07.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec3e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3fa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec424680 0x55a4ec3fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec3ce110 con 0x55a4ec3f6860 2024-03-30T13:11:07.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec2ee9c0 con 0x55a4ec3f6c40 2024-03-30T13:11:07.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55a4ec3d0950 con 0x55a4ec424680 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6bae625740 10 client.0 _ll_register_callbacks cb 0x55a4ec2d2fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec3e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec3e74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39648/0 (socket says 172.21.15.66:39648) 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6ba96f5640 1 -- 172.21.15.66:0/897532357 learned_addr learned my addr 172.21.15.66:0/897532357 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6ba9ef6640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec424680 0x55a4ec3fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.936+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba96f5640 1 -- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec424680 msgr2=0x55a4ec3fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec424680 0x55a4ec3fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba96f5640 1 -- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 msgr2=0x55a4ec3fa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3fa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba96f5640 1 -- 172.21.15.66:0/897532357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ec1116b0 con 0x55a4ec3f6860 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:07.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec3e74d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6b98058740 tx=0x7f6b980617f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a7a48f83d0d4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6b83fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6b83fff640 1 -- 172.21.15.66:0/897532357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98087060 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6b83fff640 1 -- 172.21.15.66:0/897532357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f6b98058090 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba003f530 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6b83fff640 1 -- 172.21.15.66:0/897532357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f6b98074070 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6bae625740 10 client.5580 fetch_fsmap learned FSMap version 16 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a4ec3fcc70 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6b83fff640 1 -- 172.21.15.66:0/897532357 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba003f530 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6b83fff640 1 -- 172.21.15.66:0/897532357 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f6b9807b080 con 0x55a4ec3f6860 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6bae625740 10 client.5580 fetch_fsmap finished waiting for FSMap version 16 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6bae625740 10 client.5580 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.937+0000 7f6bae625740 20 client.5580 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.938+0000 7f6bae625740 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec4129f0 0x55a4ec432780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.938+0000 7f6bae625740 4 client.5580 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:07.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.938+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4ec3074e0 con 0x55a4ec4129f0 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.938+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec4129f0 0x55a4ec432780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.938+0000 7f6b83fff640 10 client.5580 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.938+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec4129f0 0x55a4ec432780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde52e9fcb2047e server_cookie=36bcffc3098372d0 in_seq=0 out_seq=0 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6b83fff640 1 -- 172.21.15.66:0/897532357 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55a4ec3074e0 con 0x55a4ec4129f0 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6b83fff640 10 client.5580 handle_command_reply: tid=0 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 client.5580 shutdown 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 msgr2=0x55a4ec3e74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec3e74d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6b98058740 tx=0x7f6b980617f0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 shutdown_connections 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec4129f0 0x55a4ec432780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6b83fff640 0 client.5580 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec424680 0x55a4ec3fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 --2- 172.21.15.66:0/897532357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec3e74d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 >> 172.21.15.66:0/897532357 conn(0x55a4ec3c9d40 msgr2=0x55a4ec3cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 shutdown_connections 2024-03-30T13:11:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 1 -- 172.21.15.66:0/897532357 wait complete. 2024-03-30T13:11:07.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.939+0000 7f6bae625740 20 client.5580 trim_cache size 0 max 16384 2024-03-30T13:11:07.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 Processor -- start 2024-03-30T13:11:07.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 -- start start 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec411b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec3f9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4200090 con 0x55a4ec3f6860 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40cc2f0 con 0x55a4ec3f6c40 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba41ffae0 con 0x55a4ec4129f0 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec411b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec411b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39650/0 (socket says 172.21.15.66:39650) 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.958+0000 7f6ba96f5640 1 -- 172.21.15.66:0/4293357657 learned_addr learned my addr 172.21.15.66:0/4293357657 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba9ef6640 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec3f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba96f5640 1 -- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 msgr2=0x55a4ec3f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec3f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba96f5640 1 -- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 msgr2=0x55a4ec3f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba96f5640 1 -- 172.21.15.66:0/4293357657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ec3fe360 con 0x55a4ec3f6860 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec411b70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b98058330 tx=0x7f6b9805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adce0cd2ae335a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6b8e7fc640 1 -- 172.21.15.66:0/4293357657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9807e030 con 0x55a4ec3f6860 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6b8e7fc640 1 -- 172.21.15.66:0/4293357657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98054a80 con 0x55a4ec3f6860 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.959+0000 7f6b8e7fc640 1 -- 172.21.15.66:0/4293357657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98057e90 con 0x55a4ec3f6860 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 -- 172.21.15.66:0/4293357657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 msgr2=0x55a4ec411b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec411b70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b98058330 tx=0x7f6b9805c000 comp rx=0 tx=0).stop 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 -- 172.21.15.66:0/4293357657 shutdown_connections 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec3f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec3f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 --2- 172.21.15.66:0/4293357657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec411b70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 -- 172.21.15.66:0/4293357657 >> 172.21.15.66:0/4293357657 conn(0x55a4ec3c9d40 msgr2=0x55a4ec423f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 -- 172.21.15.66:0/4293357657 shutdown_connections 2024-03-30T13:11:07.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 -- 172.21.15.66:0/4293357657 wait complete. 2024-03-30T13:11:07.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 -- start start 2024-03-30T13:11:07.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.960+0000 7f6bae625740 1 Processor -- start 2024-03-30T13:11:07.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec486b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec423560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec437a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec412870 con 0x55a4ec3f6860 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec3fd6d0 con 0x55a4ec3f6c40 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55a4ec412e00 con 0x55a4ec4129f0 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6bae625740 10 client.0 _ll_register_callbacks cb 0x7f6b98059140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec486b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6ba96f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec486b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39662/0 (socket says 172.21.15.66:39662) 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.961+0000 7f6ba96f5640 1 -- 172.21.15.66:0/1327140162 learned_addr learned my addr 172.21.15.66:0/1327140162 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba9ef6640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec437a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec423560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba96f5640 1 -- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 msgr2=0x55a4ec437a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec437a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba96f5640 1 -- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 msgr2=0x55a4ec423560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec423560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba96f5640 1 -- 172.21.15.66:0/1327140162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ec423aa0 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec423560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6ba96f5640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec486b60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6b980586f0 tx=0x7f6b98088fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3e3b32ac4949ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6b83fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6b83fff640 1 -- 172.21.15.66:0/1327140162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9807e070 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6b83fff640 1 -- 172.21.15.66:0/1327140162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f6b98057b00 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b98054a80 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6b83fff640 1 -- 172.21.15.66:0/1327140162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f6b98056e00 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6bae625740 10 client.5586 fetch_fsmap learned FSMap version 16 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a4ec42bfd0 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.962+0000 7f6b83fff640 1 -- 172.21.15.66:0/1327140162 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98054a80 con 0x55a4ec3f6860 2024-03-30T13:11:07.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6b83fff640 1 -- 172.21.15.66:0/1327140162 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f6b9807b070 con 0x55a4ec3f6860 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 10 client.5586 fetch_fsmap finished waiting for FSMap version 16 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 10 client.5586 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 20 client.5586 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec424680 0x55a4ec42b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 4 client.5586 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "status"}) -- 0x55a4ec340560 con 0x55a4ec424680 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec424680 0x55a4ec42b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6b83fff640 10 client.5586 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6ba8ef4640 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec424680 0x55a4ec42b5d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e9582d5d9434c9 server_cookie=e084c910d94948b4 in_seq=0 out_seq=0 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6b83fff640 1 -- 172.21.15.66:0/1327140162 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a4ec340560 con 0x55a4ec424680 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6b83fff640 10 client.5586 handle_command_reply: tid=0 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 client.5586 shutdown 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 msgr2=0x55a4ec486b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec486b60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6b980586f0 tx=0x7f6b98088fb0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 shutdown_connections 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a4ec424680 0x55a4ec42b5d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a4ec4129f0 0x55a4ec437a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4ec3f6c40 0x55a4ec423560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6b83fff640 0 client.5586 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 --2- 172.21.15.66:0/1327140162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a4ec3f6860 0x55a4ec486b60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.963+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 >> 172.21.15.66:0/1327140162 conn(0x55a4ec3c9d40 msgr2=0x55a4ec3feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 shutdown_connections 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bae625740 1 -- 172.21.15.66:0/1327140162 wait complete. 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bae625740 20 client.5586 trim_cache size 0 max 16384 2024-03-30T13:11:07.973 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "03992909-781c-43b4-940b-d3d6d831de0b", 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5382, 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 10.294312937999999, 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 16, 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 40, 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 40, 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 17.375586419000001 2024-03-30T13:11:07.974 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b8806c530 msgr2=0x7f6b8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b8806c530 0x7f6b8806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0034000 tx=0x7f6ba002f000 comp rx=0 tx=0).stop 2024-03-30T13:11:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 msgr2=0x7f6ba40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 0x7f6ba40b4eb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c430 tx=0x7f6b9800c8f0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 shutdown_connections 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6b8806c530 0x7f6b8806e9f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4155970 0x7f6ba414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ba40d3ee0 0x7f6ba40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 --2- 172.21.15.66:0/2150129624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ba40cec60 0x7f6ba40b4eb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 >> 172.21.15.66:0/2150129624 conn(0x7f6ba40676b0 msgr2=0x7f6ba4067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 shutdown_connections 2024-03-30T13:11:07.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:07.964+0000 7f6bab959640 1 -- 172.21.15.66:0/2150129624 wait complete. 2024-03-30T13:11:07.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:08.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.082+0000 7fc36ab82640 1 Processor -- start 2024-03-30T13:11:08.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.082+0000 7fc36ab82640 1 -- start start 2024-03-30T13:11:08.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36ab82640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc36415e950 0x7fc3640d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36ab82640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3640d30d0 0x7fc3640b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36ab82640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3640d36f0 con 0x7fc3640d48c0 2024-03-30T13:11:08.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36ab82640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc3640d3cb0 con 0x7fc3640d30d0 2024-03-30T13:11:08.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36ab82640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc3640d4d00 con 0x7fc36415e950 2024-03-30T13:11:08.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36911f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3640d30d0 0x7fc3640b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc35bfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc3640d48c0 0x7fc3640d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60728/0 (socket says 172.21.15.66:60728) 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc35bfff640 1 -- 172.21.15.66:0/3365396626 learned_addr learned my addr 172.21.15.66:0/3365396626 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36891e640 1 --2- 172.21.15.66:0/3365396626 >> v2:172.21.15.117:3300/0 conn(0x7fc36415e950 0x7fc3640d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36911f640 1 -- 172.21.15.66:0/3365396626 >> v2:172.21.15.117:3300/0 conn(0x7fc36415e950 msgr2=0x7fc3640d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36911f640 1 --2- 172.21.15.66:0/3365396626 >> v2:172.21.15.117:3300/0 conn(0x7fc36415e950 0x7fc3640d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36911f640 1 -- 172.21.15.66:0/3365396626 >> v1:172.21.15.66:6789/0 conn(0x7fc3640d48c0 legacy=0x7fc3640d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc35b7fe640 1 -- 172.21.15.66:0/3365396626 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785041245 0 0) 0x7fc3640d36f0 con 0x7fc3640d48c0 2024-03-30T13:11:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.083+0000 7fc36911f640 1 -- 172.21.15.66:0/3365396626 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3640d39d0 con 0x7fc3640d30d0 2024-03-30T13:11:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.084+0000 7fc36911f640 1 --2- 172.21.15.66:0/3365396626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d30d0 0x7fc3640b82c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc3640d3e30 tx=0x7fc35c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66495f631d34509 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.084+0000 7fc35b7fe640 1 -- 172.21.15.66:0/3365396626 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c039070 con 0x7fc3640d30d0 2024-03-30T13:11:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.084+0000 7fc35b7fe640 1 -- 172.21.15.66:0/3365396626 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35c02cd00 con 0x7fc3640d30d0 2024-03-30T13:11:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.084+0000 7fc35b7fe640 1 -- 172.21.15.66:0/3365396626 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c034660 con 0x7fc3640d30d0 2024-03-30T13:11:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 -- 172.21.15.66:0/3365396626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d30d0 msgr2=0x7fc3640b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3365396626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d30d0 0x7fc3640b82c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc3640d3e30 tx=0x7fc35c02bf50 comp rx=0 tx=0).stop 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 -- 172.21.15.66:0/3365396626 shutdown_connections 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3365396626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d30d0 0x7fc3640b82c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3365396626 >> v2:172.21.15.117:3300/0 conn(0x7fc36415e950 0x7fc3640d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 -- 172.21.15.66:0/3365396626 >> 172.21.15.66:0/3365396626 conn(0x7fc3640676e0 msgr2=0x7fc364067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 -- 172.21.15.66:0/3365396626 shutdown_connections 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.085+0000 7fc36ab82640 1 -- 172.21.15.66:0/3365396626 wait complete. 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.087+0000 7fc36ab82640 1 Processor -- start 2024-03-30T13:11:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.087+0000 7fc36ab82640 1 -- start start 2024-03-30T13:11:08.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc36ab82640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3640d30d0 0x7fc3641e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc36ab82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 0x7fc3640ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc36ab82640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36415e950 0x7fc3641e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc36ab82640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc36415ed30 con 0x7fc3640d48c0 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc36ab82640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640b7da0 con 0x7fc36415e950 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc36ab82640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc36415eeb0 con 0x7fc3640d30d0 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.088+0000 7fc35bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 0x7fc3640ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 0x7fc3640ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39690/0 (socket says 172.21.15.66:39690) 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 -- 172.21.15.66:0/3504593573 learned_addr learned my addr 172.21.15.66:0/3504593573 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc36891e640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3640d30d0 0x7fc3641e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc36911f640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36415e950 0x7fc3641e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 -- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3640d30d0 msgr2=0x7fc3641e8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3640d30d0 0x7fc3641e8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 -- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36415e950 msgr2=0x7fc3641e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36415e950 0x7fc3641e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 -- 172.21.15.66:0/3504593573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3640c9940 con 0x7fc3640d48c0 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc35bfff640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 0x7fc3640ca700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c4d0 tx=0x7fc35400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354018070 con 0x7fc3640d48c0 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354007500 con 0x7fc3640d48c0 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3640c9c20 con 0x7fc3640d48c0 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354004a20 con 0x7fc3640d48c0 2024-03-30T13:11:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.089+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3640ca1e0 con 0x7fc3640d48c0 2024-03-30T13:11:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.091+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc35401b070 con 0x7fc3640d48c0 2024-03-30T13:11:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.091+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3640d1480 con 0x7fc3640d48c0 2024-03-30T13:11:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.091+0000 7fc3597fa640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc35006c560 0x7fc35006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.091+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc35408aa90 con 0x7fc3640d48c0 2024-03-30T13:11:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.091+0000 7fc36891e640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc35006c560 0x7fc35006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.092+0000 7fc36891e640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc35006c560 0x7fc35006ea20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc3640c9030 tx=0x7fc360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.096+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc3640d1480 con 0x7fc3640d48c0 2024-03-30T13:11:08.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.377+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc364075a10 con 0x7fc3640d48c0 2024-03-30T13:11:08.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.377+0000 7fc3597fa640 1 -- 172.21.15.66:0/3504593573 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fc354058fb0 con 0x7fc3640d48c0 2024-03-30T13:11:08.383 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T13:11:08.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc35006c560 msgr2=0x7fc35006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc35006c560 0x7fc35006ea20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc3640c9030 tx=0x7fc360008040 comp rx=0 tx=0).stop 2024-03-30T13:11:08.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 msgr2=0x7fc3640ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 0x7fc3640ca700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c4d0 tx=0x7fc35400c990 comp rx=0 tx=0).stop 2024-03-30T13:11:08.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 shutdown_connections 2024-03-30T13:11:08.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc35006c560 0x7fc35006ea20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc36415e950 0x7fc3641e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3640d48c0 0x7fc3640ca700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 --2- 172.21.15.66:0/3504593573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3640d30d0 0x7fc3641e8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 >> 172.21.15.66:0/3504593573 conn(0x7fc3640676e0 msgr2=0x7fc3640cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 shutdown_connections 2024-03-30T13:11:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08.380+0000 7fc36ab82640 1 -- 172.21.15.66:0/3504593573 wait complete. 2024-03-30T13:11:08.394 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:08.395 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:11:08.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.395 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:08.412 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42562sec preferred_lft 42562sec 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:20:f1:72:89:29 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:08.413 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::b002:76ff:fe8e:b0d7/64 scope link 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 62:09:dc:7a:25:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::6009:dcff:fe7a:2575/64 scope link 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 02:20:f1:72:89:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::20:f1ff:fe72:8929/64 scope link 2024-03-30T13:11:08.414 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:08.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.415 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:08.482 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-03-30T13:11:08.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.483 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-03-30T13:11:08.540 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-03-30T13:11:08.541 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:08.541 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:11:08.541 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:11:08.541 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:08.541 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:08.542 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-03-30T13:11:08.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:11:08.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:11:08.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:11:08.542 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-03-30T13:11:08.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:08.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:08.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.688 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:11:08.703 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T13:11:08.703 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T13:11:08.703 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:11:08.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.703 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-03-30T13:11:08.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:08.775 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:08.830 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:11:08.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:08.830 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:08.886 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-30T13:11:08.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:11:08.976 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:11:08.976 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:11:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:11:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:11:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-30T13:11:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 0@03992909-781c-43b4-940b-d3d6d831de0b.cephfs=/ 2024-03-30T13:11:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.66:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-30T13:11:08.984 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-30T13:11:08.984 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-03-30T13:11:08.984 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-30T13:11:08.984 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-03-30T13:11:08.984 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-30T13:11:08.985 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T13:11:08.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:08.986 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:09.022 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:11:09.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:11:09.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi066.stdout:client5120 2024-03-30T13:11:09.124 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:11:09.124 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:09.140 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:09.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.284+0000 7fbed25f0640 1 Processor -- start 2024-03-30T13:11:09.288 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.284+0000 7fbed25f0640 1 -- start start 2024-03-30T13:11:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed25f0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbecc0d4a90 0x7fbecc0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed25f0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbecc0d2f00 0x7fbecc0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed25f0640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbecc0d36f0 con 0x7fbecc15e950 2024-03-30T13:11:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed25f0640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbecc0d3cb0 con 0x7fbecc0d2f00 2024-03-30T13:11:09.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed25f0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fbecc0d39d0 con 0x7fbecc0d4a90 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed0b8d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbecc0d2f00 0x7fbecc0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbecbfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbecc15e950 0x7fbecc0d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60738/0 (socket says 172.21.15.66:60738) 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed0b8d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbecc0d2f00 0x7fbecc0b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39692/0 (socket says 172.21.15.66:39692) 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbecbfff640 1 -- 172.21.15.66:0/190659444 learned_addr learned my addr 172.21.15.66:0/190659444 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbecb7fe640 1 --2- 172.21.15.66:0/190659444 >> v2:172.21.15.117:3300/0 conn(0x7fbecc0d4a90 0x7fbecc0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed0b8d640 1 -- 172.21.15.66:0/190659444 >> v2:172.21.15.117:3300/0 conn(0x7fbecc0d4a90 msgr2=0x7fbecc0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed0b8d640 1 --2- 172.21.15.66:0/190659444 >> v2:172.21.15.117:3300/0 conn(0x7fbecc0d4a90 0x7fbecc0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed0b8d640 1 -- 172.21.15.66:0/190659444 >> v1:172.21.15.66:6789/0 conn(0x7fbecc15e950 legacy=0x7fbecc0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbed0b8d640 1 -- 172.21.15.66:0/190659444 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc0b7ac0 con 0x7fbecc0d2f00 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.285+0000 7fbecaffd640 1 -- 172.21.15.66:0/190659444 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389502604 0 0) 0x7fbecc0d36f0 con 0x7fbecc15e950 2024-03-30T13:11:09.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.286+0000 7fbed0b8d640 1 --2- 172.21.15.66:0/190659444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d2f00 0x7fbecc0b82c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0d3e30 tx=0x7fbec402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6a5ab3a129ccea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.286+0000 7fbecaffd640 1 -- 172.21.15.66:0/190659444 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4039070 con 0x7fbecc0d2f00 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.286+0000 7fbecaffd640 1 -- 172.21.15.66:0/190659444 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec402cd00 con 0x7fbecc0d2f00 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.286+0000 7fbecaffd640 1 -- 172.21.15.66:0/190659444 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4034660 con 0x7fbecc0d2f00 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 -- 172.21.15.66:0/190659444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d2f00 msgr2=0x7fbecc0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 --2- 172.21.15.66:0/190659444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d2f00 0x7fbecc0b82c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0d3e30 tx=0x7fbec402bf50 comp rx=0 tx=0).stop 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 -- 172.21.15.66:0/190659444 shutdown_connections 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 --2- 172.21.15.66:0/190659444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d2f00 0x7fbecc0b82c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 --2- 172.21.15.66:0/190659444 >> v2:172.21.15.117:3300/0 conn(0x7fbecc0d4a90 0x7fbecc0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 -- 172.21.15.66:0/190659444 >> 172.21.15.66:0/190659444 conn(0x7fbecc0676e0 msgr2=0x7fbecc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 -- 172.21.15.66:0/190659444 shutdown_connections 2024-03-30T13:11:09.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.287+0000 7fbed25f0640 1 -- 172.21.15.66:0/190659444 wait complete. 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.288+0000 7fbed25f0640 1 Processor -- start 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.288+0000 7fbed25f0640 1 -- start start 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed25f0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbecc0d2f00 0x7fbecc1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed25f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 0x7fbecc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed25f0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbecc15e950 0x7fbecc1f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed25f0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc15ed30 con 0x7fbecc0d4a90 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed25f0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc0d33b0 con 0x7fbecc0d2f00 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed25f0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc15eeb0 con 0x7fbecc15e950 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 0x7fbecc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 0x7fbecc0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39708/0 (socket says 172.21.15.66:39708) 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbecb7fe640 1 -- 172.21.15.66:0/2270089236 learned_addr learned my addr 172.21.15.66:0/2270089236 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:09.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbed0b8d640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbecc15e950 0x7fbecc1f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.289+0000 7fbecbfff640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbecc0d2f00 0x7fbecc1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbecb7fe640 1 -- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbecc15e950 msgr2=0x7fbecc1f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbecb7fe640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbecc15e950 0x7fbecc1f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbecb7fe640 1 -- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbecc0d2f00 msgr2=0x7fbecc1f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbecb7fe640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbecc0d2f00 0x7fbecc1f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbecb7fe640 1 -- 172.21.15.66:0/2270089236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc0c8bb0 con 0x7fbecc0d4a90 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbecb7fe640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 0x7fbecc0ca660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbec000e9b0 tx=0x7fbec000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0017070 con 0x7fbecc0d4a90 2024-03-30T13:11:09.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec000c9a0 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec000cd90 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc1f4de0 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.290+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc1f7b40 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.291+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbec001a070 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.291+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc075a10 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.292+0000 7fbec8ff9640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbeb806c530 0x7fbeb806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.292+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fbec008b7f0 con 0x7fbecc0d4a90 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.292+0000 7fbecbfff640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbeb806c530 0x7fbeb806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.292+0000 7fbecbfff640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbeb806c530 0x7fbeb806e9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbebc000f60 tx=0x7fbebc007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.296+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbec0059d10 con 0x7fbecc0d4a90 2024-03-30T13:11:09.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.520+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbecc0cf5f0 con 0x7fbecc0d4a90 2024-03-30T13:11:09.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.520+0000 7fbec8ff9640 1 -- 172.21.15.66:0/2270089236 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fbec0059610 con 0x7fbecc0d4a90 2024-03-30T13:11:09.524 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T13:11:09.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbeb806c530 msgr2=0x7fbeb806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbeb806c530 0x7fbeb806e9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbebc000f60 tx=0x7fbebc007c00 comp rx=0 tx=0).stop 2024-03-30T13:11:09.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 msgr2=0x7fbecc0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 0x7fbecc0ca660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbec000e9b0 tx=0x7fbec000ee70 comp rx=0 tx=0).stop 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 shutdown_connections 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fbeb806c530 0x7fbeb806e9f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbecc15e950 0x7fbecc1f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc0d4a90 0x7fbecc0ca660 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 --2- 172.21.15.66:0/2270089236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbecc0d2f00 0x7fbecc1f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 >> 172.21.15.66:0/2270089236 conn(0x7fbecc0676e0 msgr2=0x7fbecc0cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 shutdown_connections 2024-03-30T13:11:09.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:09.522+0000 7fbed25f0640 1 -- 172.21.15.66:0/2270089236 wait complete. 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:09.537 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:11:09.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:09.537 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-03-30T13:11:09.557 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:09.558 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:09.558 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:09.558 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.558 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:09.558 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.558 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:11:09.559 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:09.559 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-03-30T13:11:09.559 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:11:09.559 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:09.559 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-03-30T13:11:09.559 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T13:11:09.560 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:09.560 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:09.560 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:09.560 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T13:11:09.560 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42514sec preferred_lft 42514sec 2024-03-30T13:11:09.560 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-03-30T13:11:09.561 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.561 INFO:teuthology.orchestra.run.smithi117.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:09.561 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 6a:21:9d:8f:f6:b5 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:09.561 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:11:09.561 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.561 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::3c4b:adff:fe12:ffe9/64 scope link 2024-03-30T13:11:09.562 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.562 INFO:teuthology.orchestra.run.smithi117.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:11:09.562 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether e6:1e:e5:47:b7:28 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:09.562 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::e41e:e5ff:fe47:b728/64 scope link 2024-03-30T13:11:09.562 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.563 INFO:teuthology.orchestra.run.smithi117.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:11:09.563 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 6a:21:9d:8f:f6:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:09.563 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::6821:9dff:fe8f:f6b5/64 scope link 2024-03-30T13:11:09.563 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:09.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:09.564 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:09.628 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-03-30T13:11:09.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:09.630 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-03-30T13:11:09.691 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-03-30T13:11:09.691 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:09.691 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:11:09.691 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:11:09.691 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:09.692 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:09.692 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-03-30T13:11:09.692 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:11:09.692 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:11:09.692 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:11:09.692 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-03-30T13:11:09.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:11:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:09.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:09.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:09.834 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T13:11:09.849 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:09.850 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-03-30T13:11:09.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:09.918 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:09.972 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:11:09.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:09.973 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:10.027 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-30T13:11:10.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:10.028 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:11:10.112 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:11:10.113 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:11:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-30T13:11:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-30T13:11:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-03-30T13:11:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with new device syntax: 1@03992909-781c-43b4-940b-d3d6d831de0b.cephfs=/ 2024-03-30T13:11:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.66:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-03-30T13:11:10.120 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-30T13:11:10.121 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-03-30T13:11:10.121 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-30T13:11:10.121 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-03-30T13:11:10.121 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-30T13:11:10.122 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T13:11:10.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:10.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:10.150 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T13:11:10.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:11:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:11:10.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-30T13:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:client5607 2024-03-30T13:11:10.245 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:11:10.245 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:10.261 DEBUG:tasks.cephfs.test_uninlining:setting inline_data:1 2024-03-30T13:11:10.261 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs inline_data 1 --yes-i-really-really-mean-it 2024-03-30T13:11:10.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.369+0000 7ff7ed242640 1 Processor -- start 2024-03-30T13:11:10.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.369+0000 7ff7ed242640 1 -- start start 2024-03-30T13:11:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7ed242640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7ed242640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7e80d3ef0 0x7ff7e80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7ed242640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7e80d2910 con 0x7ff7e815e440 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7ed242640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e80d3520 con 0x7ff7e80d3ef0 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7ed242640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80d2bf0 con 0x7ff7e806d4a0 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7e659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.370+0000 7ff7e759e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7e80d3ef0 0x7ff7e80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e759e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7e80d3ef0 0x7ff7e80cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:37112/0 (socket says 172.21.15.66:37112) 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e6d9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff7e815e440 0x7ff7e80b9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:42890/0 (socket says 172.21.15.66:42890) 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:42352/0 (socket says 172.21.15.66:42352) 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e759e640 1 -- 172.21.15.66:0/64444250 learned_addr learned my addr 172.21.15.66:0/64444250 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e759e640 1 -- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 msgr2=0x7ff7e80d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e759e640 1 --2- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e759e640 1 -- 172.21.15.66:0/64444250 >> v1:172.21.15.117:6789/0 conn(0x7ff7e815e440 legacy=0x7ff7e80b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e759e640 1 -- 172.21.15.66:0/64444250 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e80d2ed0 con 0x7ff7e80d3ef0 2024-03-30T13:11:10.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e5d9b640 1 -- 172.21.15.66:0/64444250 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2107275325 0 0) 0x7ff7e80d2910 con 0x7ff7e815e440 2024-03-30T13:11:10.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.371+0000 7ff7e659c640 1 --2- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:10.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7e759e640 1 --2- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e80d3ef0 0x7ff7e80cf040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009e50 tx=0x7ff7e002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace6285f41330700 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7e5d9b640 1 -- 172.21.15.66:0/64444250 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e002cb90 con 0x7ff7e80d3ef0 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7e5d9b640 1 -- 172.21.15.66:0/64444250 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e002cd30 con 0x7ff7e80d3ef0 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7e5d9b640 1 -- 172.21.15.66:0/64444250 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0034610 con 0x7ff7e80d3ef0 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 -- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e80d3ef0 msgr2=0x7ff7e80cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 --2- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e80d3ef0 0x7ff7e80cf040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009e50 tx=0x7ff7e002bff0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 -- 172.21.15.66:0/64444250 shutdown_connections 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 --2- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e80d3ef0 0x7ff7e80cf040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 --2- 172.21.15.66:0/64444250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 -- 172.21.15.66:0/64444250 >> 172.21.15.66:0/64444250 conn(0x7ff7e80676b0 msgr2=0x7ff7e8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 -- 172.21.15.66:0/64444250 shutdown_connections 2024-03-30T13:11:10.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.372+0000 7ff7ed242640 1 -- 172.21.15.66:0/64444250 wait complete. 2024-03-30T13:11:10.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.374+0000 7ff7ed242640 1 Processor -- start 2024-03-30T13:11:10.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.374+0000 7ff7ed242640 1 -- start start 2024-03-30T13:11:10.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7ed242640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7ed242640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e815e440 0x7ff7e815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7ed242640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 0x7ff7e815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7ed242640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e815e9a0 con 0x7ff7e814f7d0 2024-03-30T13:11:10.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7ed242640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e815e820 con 0x7ff7e815e440 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7ed242640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80cf580 con 0x7ff7e806d4a0 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 0x7ff7e815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 0x7ff7e815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39712/0 (socket says 172.21.15.66:39712) 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 -- 172.21.15.66:0/525476473 learned_addr learned my addr 172.21.15.66:0/525476473 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e6d9d640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e659c640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e815e440 0x7ff7e815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 -- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 msgr2=0x7ff7e80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 -- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e815e440 msgr2=0x7ff7e815c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e815e440 0x7ff7e815c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 -- 172.21.15.66:0/525476473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e815ee70 con 0x7ff7e814f7d0 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e659c640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e815e440 0x7ff7e815c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.375+0000 7ff7e759e640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 0x7ff7e815c9a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009e50 tx=0x7ff7e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.376+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0043020 con 0x7ff7e814f7d0 2024-03-30T13:11:10.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.376+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e003e450 con 0x7ff7e814f7d0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.376+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e815f150 con 0x7ff7e814f7d0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.376+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0009510 con 0x7ff7e814f7d0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.376+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e815f710 con 0x7ff7e814f7d0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.377+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e80d2270 con 0x7ff7e814f7d0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.377+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff7e0046020 con 0x7ff7e814f7d0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.378+0000 7ff7df7fe640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7c806c530 0x7ff7c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.378+0000 7ff7e6d9d640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7c806c530 0x7ff7c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.378+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff7e00ade00 con 0x7ff7e814f7d0 2024-03-30T13:11:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.378+0000 7ff7e6d9d640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7c806c530 0x7ff7c806e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8007fd0 tx=0x7ff7d8007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.382+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff7e80d2270 con 0x7ff7e814f7d0 2024-03-30T13:11:10.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.606+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff7e815ae70 con 0x7ff7e814f7d0 2024-03-30T13:11:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.876+0000 7ff7df7fe640 1 -- 172.21.15.66:0/525476473 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true}]=0 inline data enabled v17) ==== 165+0+0 (secure 0 0 0) 0x7ff7e815ae70 con 0x7ff7e814f7d0 2024-03-30T13:11:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:inline data enabled 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7c806c530 msgr2=0x7ff7c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7c806c530 0x7ff7c806e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8007fd0 tx=0x7ff7d8007de0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 msgr2=0x7ff7e815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 0x7ff7e815c9a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009e50 tx=0x7ff7e0004a40 comp rx=0 tx=0).stop 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 shutdown_connections 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7c806c530 0x7ff7c806e9f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7e814f7d0 0x7ff7e815c9a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e815e440 0x7ff7e815c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 --2- 172.21.15.66:0/525476473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7e806d4a0 0x7ff7e80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 >> 172.21.15.66:0/525476473 conn(0x7ff7e80676b0 msgr2=0x7ff7e815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 shutdown_connections 2024-03-30T13:11:10.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:10.879+0000 7ff7ed242640 1 -- 172.21.15.66:0/525476473 wait complete. 2024-03-30T13:11:10.893 DEBUG:tasks.cephfs.test_uninlining:fetching tarball 2024-03-30T13:11:10.894 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && wget http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-30T13:11:10.928 INFO:teuthology.orchestra.run.smithi066.stderr:--2024-03-30 13:11:10-- http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-30T13:11:10.928 INFO:teuthology.orchestra.run.smithi066.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-30T13:11:10.965 INFO:teuthology.orchestra.run.smithi066.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-30T13:11:11.000 INFO:teuthology.orchestra.run.smithi066.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-30T13:11:11.001 INFO:teuthology.orchestra.run.smithi066.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-03-30T13:11:11.004 INFO:teuthology.orchestra.run.smithi066.stderr:Saving to: ‘linux-5.4.tar.gz’ 2024-03-30T13:11:11.006 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:11:11.081 INFO:teuthology.orchestra.run.smithi066.stderr: 0K .......... .......... .......... .......... .......... 0% 671K 4m11s 2024-03-30T13:11:11.116 INFO:teuthology.orchestra.run.smithi066.stderr: 50K .......... .......... .......... .......... .......... 0% 1.37M 3m6s 2024-03-30T13:11:11.153 INFO:teuthology.orchestra.run.smithi066.stderr: 100K .......... .......... .......... .......... .......... 0% 1.37M 2m44s 2024-03-30T13:11:11.199 INFO:teuthology.orchestra.run.smithi066.stderr: 150K .......... .......... .......... .......... .......... 0% 1.06M 2m41s 2024-03-30T13:11:11.200 INFO:teuthology.orchestra.run.smithi066.stderr: 200K .......... .......... .......... .......... .......... 0% 50.0M 2m10s 2024-03-30T13:11:11.234 INFO:teuthology.orchestra.run.smithi066.stderr: 250K .......... .......... .......... .......... .......... 0% 1.38M 2m8s 2024-03-30T13:11:11.274 INFO:teuthology.orchestra.run.smithi066.stderr: 300K .......... .......... .......... .......... .......... 0% 1.24M 2m9s 2024-03-30T13:11:11.310 INFO:teuthology.orchestra.run.smithi066.stderr: 350K .......... .......... .......... .......... .......... 0% 1.38M 2m7s 2024-03-30T13:11:11.358 INFO:teuthology.orchestra.run.smithi066.stderr: 400K .......... .......... .......... .......... .......... 0% 1.05M 2m11s 2024-03-30T13:11:11.358 INFO:teuthology.orchestra.run.smithi066.stderr: 450K .......... .......... .......... .......... .......... 0% 23.1M 1m58s 2024-03-30T13:11:11.393 INFO:teuthology.orchestra.run.smithi066.stderr: 500K .......... .......... .......... .......... .......... 0% 1.41M 1m58s 2024-03-30T13:11:11.429 INFO:teuthology.orchestra.run.smithi066.stderr: 550K .......... .......... .......... .......... .......... 0% 1.39M 1m58s 2024-03-30T13:11:11.429 INFO:teuthology.orchestra.run.smithi066.stderr: 600K .......... .......... .......... .......... .......... 0% 32.0M 1m49s 2024-03-30T13:11:11.466 INFO:teuthology.orchestra.run.smithi066.stderr: 650K .......... .......... .......... .......... .......... 0% 1.42M 1m50s 2024-03-30T13:11:11.502 INFO:teuthology.orchestra.run.smithi066.stderr: 700K .......... .......... .......... .......... .......... 0% 1.39M 1m50s 2024-03-30T13:11:11.503 INFO:teuthology.orchestra.run.smithi066.stderr: 750K .......... .......... .......... .......... .......... 0% 74.9M 1m43s 2024-03-30T13:11:11.538 INFO:teuthology.orchestra.run.smithi066.stderr: 800K .......... .......... .......... .......... .......... 0% 1.39M 1m44s 2024-03-30T13:11:11.539 INFO:teuthology.orchestra.run.smithi066.stderr: 850K .......... .......... .......... .......... .......... 0% 37.4M 99s 2024-03-30T13:11:11.572 INFO:teuthology.orchestra.run.smithi066.stderr: 900K .......... .......... .......... .......... .......... 0% 1.42M 99s 2024-03-30T13:11:11.572 INFO:teuthology.orchestra.run.smithi066.stderr: 950K .......... .......... .......... .......... .......... 0% 46.2M 95s 2024-03-30T13:11:11.611 INFO:teuthology.orchestra.run.smithi066.stderr: 1000K .......... .......... .......... .......... .......... 0% 1.26M 96s 2024-03-30T13:11:11.611 INFO:teuthology.orchestra.run.smithi066.stderr: 1050K .......... .......... .......... .......... .......... 0% 69.7M 92s 2024-03-30T13:11:11.647 INFO:teuthology.orchestra.run.smithi066.stderr: 1100K .......... .......... .......... .......... .......... 0% 1.40M 93s 2024-03-30T13:11:11.647 INFO:teuthology.orchestra.run.smithi066.stderr: 1150K .......... .......... .......... .......... .......... 0% 99.9M 89s 2024-03-30T13:11:11.687 INFO:teuthology.orchestra.run.smithi066.stderr: 1200K .......... .......... .......... .......... .......... 0% 1.32M 91s 2024-03-30T13:11:11.687 INFO:teuthology.orchestra.run.smithi066.stderr: 1250K .......... .......... .......... .......... .......... 0% 68.3M 87s 2024-03-30T13:11:11.723 INFO:teuthology.orchestra.run.smithi066.stderr: 1300K .......... .......... .......... .......... .......... 0% 1.37M 88s 2024-03-30T13:11:11.723 INFO:teuthology.orchestra.run.smithi066.stderr: 1350K .......... .......... .......... .......... .......... 0% 62.2M 85s 2024-03-30T13:11:11.772 INFO:teuthology.orchestra.run.smithi066.stderr: 1400K .......... .......... .......... .......... .......... 0% 996K 88s 2024-03-30T13:11:11.773 INFO:teuthology.orchestra.run.smithi066.stderr: 1450K .......... .......... .......... .......... .......... 0% 115M 85s 2024-03-30T13:11:11.813 INFO:teuthology.orchestra.run.smithi066.stderr: 1500K .......... .......... .......... .......... .......... 0% 65.5M 82s 2024-03-30T13:11:11.813 INFO:teuthology.orchestra.run.smithi066.stderr: 1550K .......... .......... .......... .......... .......... 0% 1.24M 84s 2024-03-30T13:11:11.813 INFO:teuthology.orchestra.run.smithi066.stderr: 1600K .......... .......... .......... .......... .......... 0% 71.3M 81s 2024-03-30T13:11:11.855 INFO:teuthology.orchestra.run.smithi066.stderr: 1650K .......... .......... .......... .......... .......... 1% 1.20M 83s 2024-03-30T13:11:11.855 INFO:teuthology.orchestra.run.smithi066.stderr: 1700K .......... .......... .......... .......... .......... 1% 59.0M 81s 2024-03-30T13:11:11.855 INFO:teuthology.orchestra.run.smithi066.stderr: 1750K .......... .......... .......... .......... .......... 1% 111M 78s 2024-03-30T13:11:11.890 INFO:teuthology.orchestra.run.smithi066.stderr: 1800K .......... .......... .......... .......... .......... 1% 1.42M 79s 2024-03-30T13:11:11.890 INFO:teuthology.orchestra.run.smithi066.stderr: 1850K .......... .......... .......... .......... .......... 1% 53.1M 77s 2024-03-30T13:11:11.929 INFO:teuthology.orchestra.run.smithi066.stderr: 1900K .......... .......... .......... .......... .......... 1% 1.42M 78s 2024-03-30T13:11:11.929 INFO:teuthology.orchestra.run.smithi066.stderr: 1950K .......... .......... .......... .......... .......... 1% 46.2M 76s 2024-03-30T13:11:11.929 INFO:teuthology.orchestra.run.smithi066.stderr: 2000K .......... .......... .......... .......... .......... 1% 79.5M 75s 2024-03-30T13:11:11.978 INFO:teuthology.orchestra.run.smithi066.stderr: 2050K .......... .......... .......... .......... .......... 1% 987K 77s 2024-03-30T13:11:11.979 INFO:teuthology.orchestra.run.smithi066.stderr: 2100K .......... .......... .......... .......... .......... 1% 98.1M 75s 2024-03-30T13:11:11.979 INFO:teuthology.orchestra.run.smithi066.stderr: 2150K .......... .......... .......... .......... .......... 1% 111M 73s 2024-03-30T13:11:12.024 INFO:teuthology.orchestra.run.smithi066.stderr: 2200K .......... .......... .......... .......... .......... 1% 1.09M 75s 2024-03-30T13:11:12.025 INFO:teuthology.orchestra.run.smithi066.stderr: 2250K .......... .......... .......... .......... .......... 1% 49.2M 73s 2024-03-30T13:11:12.025 INFO:teuthology.orchestra.run.smithi066.stderr: 2300K .......... .......... .......... .......... .......... 1% 115M 72s 2024-03-30T13:11:12.063 INFO:teuthology.orchestra.run.smithi066.stderr: 2350K .......... .......... .......... .......... .......... 1% 68.2M 70s 2024-03-30T13:11:12.063 INFO:teuthology.orchestra.run.smithi066.stderr: 2400K .......... .......... .......... .......... .......... 1% 1.37M 71s 2024-03-30T13:11:12.063 INFO:teuthology.orchestra.run.smithi066.stderr: 2450K .......... .......... .......... .......... .......... 1% 94.8M 70s 2024-03-30T13:11:12.063 INFO:teuthology.orchestra.run.smithi066.stderr: 2500K .......... .......... .......... .......... .......... 1% 52.2M 69s 2024-03-30T13:11:12.107 INFO:teuthology.orchestra.run.smithi066.stderr: 2550K .......... .......... .......... .......... .......... 1% 1.15M 70s 2024-03-30T13:11:12.107 INFO:teuthology.orchestra.run.smithi066.stderr: 2600K .......... .......... .......... .......... .......... 1% 47.8M 69s 2024-03-30T13:11:12.107 INFO:teuthology.orchestra.run.smithi066.stderr: 2650K .......... .......... .......... .......... .......... 1% 49.1M 67s 2024-03-30T13:11:12.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:12.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:12.153 INFO:teuthology.orchestra.run.smithi066.stderr: 2700K .......... .......... .......... .......... .......... 1% 1.11M 69s 2024-03-30T13:11:12.153 INFO:teuthology.orchestra.run.smithi066.stderr: 2750K .......... .......... .......... .......... .......... 1% 48.6M 68s 2024-03-30T13:11:12.153 INFO:teuthology.orchestra.run.smithi066.stderr: 2800K .......... .......... .......... .......... .......... 1% 43.7M 67s 2024-03-30T13:11:12.153 INFO:teuthology.orchestra.run.smithi066.stderr: 2850K .......... .......... .......... .......... .......... 1% 32.5M 65s 2024-03-30T13:11:12.203 INFO:teuthology.orchestra.run.smithi066.stderr: 2900K .......... .......... .......... .......... .......... 1% 1.03M 67s 2024-03-30T13:11:12.203 INFO:teuthology.orchestra.run.smithi066.stderr: 2950K .......... .......... .......... .......... .......... 1% 36.1M 66s 2024-03-30T13:11:12.203 INFO:teuthology.orchestra.run.smithi066.stderr: 3000K .......... .......... .......... .......... .......... 1% 28.9M 65s 2024-03-30T13:11:12.203 INFO:teuthology.orchestra.run.smithi066.stderr: 3050K .......... .......... .......... .......... .......... 1% 44.2M 64s 2024-03-30T13:11:12.255 INFO:teuthology.orchestra.run.smithi066.stderr: 3100K .......... .......... .......... .......... .......... 1% 1.03M 65s 2024-03-30T13:11:12.255 INFO:teuthology.orchestra.run.smithi066.stderr: 3150K .......... .......... .......... .......... .......... 1% 33.8M 64s 2024-03-30T13:11:12.255 INFO:teuthology.orchestra.run.smithi066.stderr: 3200K .......... .......... .......... .......... .......... 1% 31.0M 63s 2024-03-30T13:11:12.255 INFO:teuthology.orchestra.run.smithi066.stderr: 3250K .......... .......... .......... .......... .......... 1% 30.5M 63s 2024-03-30T13:11:12.295 INFO:teuthology.orchestra.run.smithi066.stderr: 3300K .......... .......... .......... .......... .......... 1% 1.33M 63s 2024-03-30T13:11:12.295 INFO:teuthology.orchestra.run.smithi066.stderr: 3350K .......... .......... .......... .......... .......... 2% 107M 62s 2024-03-30T13:11:12.295 INFO:teuthology.orchestra.run.smithi066.stderr: 3400K .......... .......... .......... .......... .......... 2% 132M 62s 2024-03-30T13:11:12.296 INFO:teuthology.orchestra.run.smithi066.stderr: 3450K .......... .......... .......... .......... .......... 2% 95.4M 61s 2024-03-30T13:11:12.331 INFO:teuthology.orchestra.run.smithi066.stderr: 3500K .......... .......... .......... .......... .......... 2% 1.43M 61s 2024-03-30T13:11:12.331 INFO:teuthology.orchestra.run.smithi066.stderr: 3550K .......... .......... .......... .......... .......... 2% 115M 61s 2024-03-30T13:11:12.332 INFO:teuthology.orchestra.run.smithi066.stderr: 3600K .......... .......... .......... .......... .......... 2% 112M 60s 2024-03-30T13:11:12.332 INFO:teuthology.orchestra.run.smithi066.stderr: 3650K .......... .......... .......... .......... .......... 2% 108M 59s 2024-03-30T13:11:12.332 INFO:teuthology.orchestra.run.smithi066.stderr: 3700K .......... .......... .......... .......... .......... 2% 110M 58s 2024-03-30T13:11:12.368 INFO:teuthology.orchestra.run.smithi066.stderr: 3750K .......... .......... .......... .......... .......... 2% 1.44M 59s 2024-03-30T13:11:12.368 INFO:teuthology.orchestra.run.smithi066.stderr: 3800K .......... .......... .......... .......... .......... 2% 86.3M 58s 2024-03-30T13:11:12.368 INFO:teuthology.orchestra.run.smithi066.stderr: 3850K .......... .......... .......... .......... .......... 2% 107M 57s 2024-03-30T13:11:12.368 INFO:teuthology.orchestra.run.smithi066.stderr: 3900K .......... .......... .......... .......... .......... 2% 81.6M 57s 2024-03-30T13:11:12.403 INFO:teuthology.orchestra.run.smithi066.stderr: 3950K .......... .......... .......... .......... .......... 2% 1.45M 57s 2024-03-30T13:11:12.403 INFO:teuthology.orchestra.run.smithi066.stderr: 4000K .......... .......... .......... .......... .......... 2% 135M 57s 2024-03-30T13:11:12.403 INFO:teuthology.orchestra.run.smithi066.stderr: 4050K .......... .......... .......... .......... .......... 2% 113M 56s 2024-03-30T13:11:12.403 INFO:teuthology.orchestra.run.smithi066.stderr: 4100K .......... .......... .......... .......... .......... 2% 107M 55s 2024-03-30T13:11:12.403 INFO:teuthology.orchestra.run.smithi066.stderr: 4150K .......... .......... .......... .......... .......... 2% 86.3M 55s 2024-03-30T13:11:12.451 INFO:teuthology.orchestra.run.smithi066.stderr: 4200K .......... .......... .......... .......... .......... 2% 1.08M 56s 2024-03-30T13:11:12.451 INFO:teuthology.orchestra.run.smithi066.stderr: 4250K .......... .......... .......... .......... .......... 2% 109M 55s 2024-03-30T13:11:12.451 INFO:teuthology.orchestra.run.smithi066.stderr: 4300K .......... .......... .......... .......... .......... 2% 107M 54s 2024-03-30T13:11:12.451 INFO:teuthology.orchestra.run.smithi066.stderr: 4350K .......... .......... .......... .......... .......... 2% 115M 54s 2024-03-30T13:11:12.451 INFO:teuthology.orchestra.run.smithi066.stderr: 4400K .......... .......... .......... .......... .......... 2% 104M 53s 2024-03-30T13:11:12.488 INFO:teuthology.orchestra.run.smithi066.stderr: 4450K .......... .......... .......... .......... .......... 2% 1.45M 54s 2024-03-30T13:11:12.488 INFO:teuthology.orchestra.run.smithi066.stderr: 4500K .......... .......... .......... .......... .......... 2% 105M 53s 2024-03-30T13:11:12.488 INFO:teuthology.orchestra.run.smithi066.stderr: 4550K .......... .......... .......... .......... .......... 2% 125M 53s 2024-03-30T13:11:12.488 INFO:teuthology.orchestra.run.smithi066.stderr: 4600K .......... .......... .......... .......... .......... 2% 106M 52s 2024-03-30T13:11:12.488 INFO:teuthology.orchestra.run.smithi066.stderr: 4650K .......... .......... .......... .......... .......... 2% 111M 51s 2024-03-30T13:11:12.488 INFO:teuthology.orchestra.run.smithi066.stderr: 4700K .......... .......... .......... .......... .......... 2% 109M 51s 2024-03-30T13:11:12.533 INFO:teuthology.orchestra.run.smithi066.stderr: 4750K .......... .......... .......... .......... .......... 2% 1.12M 52s 2024-03-30T13:11:12.533 INFO:teuthology.orchestra.run.smithi066.stderr: 4800K .......... .......... .......... .......... .......... 2% 101M 51s 2024-03-30T13:11:12.533 INFO:teuthology.orchestra.run.smithi066.stderr: 4850K .......... .......... .......... .......... .......... 2% 109M 51s 2024-03-30T13:11:12.533 INFO:teuthology.orchestra.run.smithi066.stderr: 4900K .......... .......... .......... .......... .......... 2% 114M 50s 2024-03-30T13:11:12.533 INFO:teuthology.orchestra.run.smithi066.stderr: 4950K .......... .......... .......... .......... .......... 2% 111M 50s 2024-03-30T13:11:12.534 INFO:teuthology.orchestra.run.smithi066.stderr: 5000K .......... .......... .......... .......... .......... 2% 116M 49s 2024-03-30T13:11:12.570 INFO:teuthology.orchestra.run.smithi066.stderr: 5050K .......... .......... .......... .......... .......... 3% 1.39M 50s 2024-03-30T13:11:12.570 INFO:teuthology.orchestra.run.smithi066.stderr: 5100K .......... .......... .......... .......... .......... 3% 172M 49s 2024-03-30T13:11:12.570 INFO:teuthology.orchestra.run.smithi066.stderr: 5150K .......... .......... .......... .......... .......... 3% 107M 49s 2024-03-30T13:11:12.570 INFO:teuthology.orchestra.run.smithi066.stderr: 5200K .......... .......... .......... .......... .......... 3% 118M 49s 2024-03-30T13:11:12.570 INFO:teuthology.orchestra.run.smithi066.stderr: 5250K .......... .......... .......... .......... .......... 3% 113M 48s 2024-03-30T13:11:12.571 INFO:teuthology.orchestra.run.smithi066.stderr: 5300K .......... .......... .......... .......... .......... 3% 111M 48s 2024-03-30T13:11:12.620 INFO:teuthology.orchestra.run.smithi066.stderr: 5350K .......... .......... .......... .......... .......... 3% 1.05M 49s 2024-03-30T13:11:12.620 INFO:teuthology.orchestra.run.smithi066.stderr: 5400K .......... .......... .......... .......... .......... 3% 135M 48s 2024-03-30T13:11:12.620 INFO:teuthology.orchestra.run.smithi066.stderr: 5450K .......... .......... .......... .......... .......... 3% 114M 48s 2024-03-30T13:11:12.620 INFO:teuthology.orchestra.run.smithi066.stderr: 5500K .......... .......... .......... .......... .......... 3% 109M 47s 2024-03-30T13:11:12.620 INFO:teuthology.orchestra.run.smithi066.stderr: 5550K .......... .......... .......... .......... .......... 3% 104M 47s 2024-03-30T13:11:12.620 INFO:teuthology.orchestra.run.smithi066.stderr: 5600K .......... .......... .......... .......... .......... 3% 104M 46s 2024-03-30T13:11:12.661 INFO:teuthology.orchestra.run.smithi066.stderr: 5650K .......... .......... .......... .......... .......... 3% 1.46M 47s 2024-03-30T13:11:12.661 INFO:teuthology.orchestra.run.smithi066.stderr: 5700K .......... .......... .......... .......... .......... 3% 111M 47s 2024-03-30T13:11:12.661 INFO:teuthology.orchestra.run.smithi066.stderr: 5750K .......... .......... .......... .......... .......... 3% 102M 46s 2024-03-30T13:11:12.661 INFO:teuthology.orchestra.run.smithi066.stderr: 5800K .......... .......... .......... .......... .......... 3% 93.8M 46s 2024-03-30T13:11:12.661 INFO:teuthology.orchestra.run.smithi066.stderr: 5850K .......... .......... .......... .......... .......... 3% 101M 45s 2024-03-30T13:11:12.662 INFO:teuthology.orchestra.run.smithi066.stderr: 5900K .......... .......... .......... .......... .......... 3% 91.8M 45s 2024-03-30T13:11:12.662 INFO:teuthology.orchestra.run.smithi066.stderr: 5950K .......... .......... .......... .......... .......... 3% 96.4M 45s 2024-03-30T13:11:12.695 INFO:teuthology.orchestra.run.smithi066.stderr: 6000K .......... .......... .......... .......... .......... 3% 1.49M 45s 2024-03-30T13:11:12.695 INFO:teuthology.orchestra.run.smithi066.stderr: 6050K .......... .......... .......... .......... .......... 3% 89.0M 45s 2024-03-30T13:11:12.695 INFO:teuthology.orchestra.run.smithi066.stderr: 6100K .......... .......... .......... .......... .......... 3% 82.4M 44s 2024-03-30T13:11:12.695 INFO:teuthology.orchestra.run.smithi066.stderr: 6150K .......... .......... .......... .......... .......... 3% 92.3M 44s 2024-03-30T13:11:12.695 INFO:teuthology.orchestra.run.smithi066.stderr: 6200K .......... .......... .......... .......... .......... 3% 102M 44s 2024-03-30T13:11:12.695 INFO:teuthology.orchestra.run.smithi066.stderr: 6250K .......... .......... .......... .......... .......... 3% 110M 43s 2024-03-30T13:11:12.734 INFO:teuthology.orchestra.run.smithi066.stderr: 6300K .......... .......... .......... .......... .......... 3% 93.8M 43s 2024-03-30T13:11:12.734 INFO:teuthology.orchestra.run.smithi066.stderr: 6350K .......... .......... .......... .......... .......... 3% 1.40M 43s 2024-03-30T13:11:12.734 INFO:teuthology.orchestra.run.smithi066.stderr: 6400K .......... .......... .......... .......... .......... 3% 126M 43s 2024-03-30T13:11:12.734 INFO:teuthology.orchestra.run.smithi066.stderr: 6450K .......... .......... .......... .......... .......... 3% 109M 43s 2024-03-30T13:11:12.735 INFO:teuthology.orchestra.run.smithi066.stderr: 6500K .......... .......... .......... .......... .......... 3% 113M 42s 2024-03-30T13:11:12.735 INFO:teuthology.orchestra.run.smithi066.stderr: 6550K .......... .......... .......... .......... .......... 3% 108M 42s 2024-03-30T13:11:12.735 INFO:teuthology.orchestra.run.smithi066.stderr: 6600K .......... .......... .......... .......... .......... 3% 110M 42s 2024-03-30T13:11:12.735 INFO:teuthology.orchestra.run.smithi066.stderr: 6650K .......... .......... .......... .......... .......... 3% 106M 42s 2024-03-30T13:11:12.772 INFO:teuthology.orchestra.run.smithi066.stderr: 6700K .......... .......... .......... .......... .......... 4% 1.41M 42s 2024-03-30T13:11:12.772 INFO:teuthology.orchestra.run.smithi066.stderr: 6750K .......... .......... .......... .......... .......... 4% 143M 42s 2024-03-30T13:11:12.772 INFO:teuthology.orchestra.run.smithi066.stderr: 6800K .......... .......... .......... .......... .......... 4% 117M 41s 2024-03-30T13:11:12.772 INFO:teuthology.orchestra.run.smithi066.stderr: 6850K .......... .......... .......... .......... .......... 4% 113M 41s 2024-03-30T13:11:12.773 INFO:teuthology.orchestra.run.smithi066.stderr: 6900K .......... .......... .......... .......... .......... 4% 107M 41s 2024-03-30T13:11:12.773 INFO:teuthology.orchestra.run.smithi066.stderr: 6950K .......... .......... .......... .......... .......... 4% 110M 41s 2024-03-30T13:11:12.773 INFO:teuthology.orchestra.run.smithi066.stderr: 7000K .......... .......... .......... .......... .......... 4% 110M 40s 2024-03-30T13:11:12.818 INFO:teuthology.orchestra.run.smithi066.stderr: 7050K .......... .......... .......... .......... .......... 4% 1.23M 41s 2024-03-30T13:11:12.818 INFO:teuthology.orchestra.run.smithi066.stderr: 7100K .......... .......... .......... .......... .......... 4% 114M 41s 2024-03-30T13:11:12.818 INFO:teuthology.orchestra.run.smithi066.stderr: 7150K .......... .......... .......... .......... .......... 4% 106M 40s 2024-03-30T13:11:12.819 INFO:teuthology.orchestra.run.smithi066.stderr: 7200K .......... .......... .......... .......... .......... 4% 102M 40s 2024-03-30T13:11:12.819 INFO:teuthology.orchestra.run.smithi066.stderr: 7250K .......... .......... .......... .......... .......... 4% 111M 40s 2024-03-30T13:11:12.819 INFO:teuthology.orchestra.run.smithi066.stderr: 7300K .......... .......... .......... .......... .......... 4% 107M 39s 2024-03-30T13:11:12.819 INFO:teuthology.orchestra.run.smithi066.stderr: 7350K .......... .......... .......... .......... .......... 4% 111M 39s 2024-03-30T13:11:12.819 INFO:teuthology.orchestra.run.smithi066.stderr: 7400K .......... .......... .......... .......... .......... 4% 103M 39s 2024-03-30T13:11:12.853 INFO:teuthology.orchestra.run.smithi066.stderr: 7450K .......... .......... .......... .......... .......... 4% 109M 39s 2024-03-30T13:11:12.853 INFO:teuthology.orchestra.run.smithi066.stderr: 7500K .......... .......... .......... .......... .......... 4% 1.21M 39s 2024-03-30T13:11:12.853 INFO:teuthology.orchestra.run.smithi066.stderr: 7550K .......... .......... .......... .......... .......... 4% 96.8M 39s 2024-03-30T13:11:12.853 INFO:teuthology.orchestra.run.smithi066.stderr: 7600K .......... .......... .......... .......... .......... 4% 182M 39s 2024-03-30T13:11:12.853 INFO:teuthology.orchestra.run.smithi066.stderr: 7650K .......... .......... .......... .......... .......... 4% 156M 38s 2024-03-30T13:11:12.853 INFO:teuthology.orchestra.run.smithi066.stderr: 7700K .......... .......... .......... .......... .......... 4% 108M 38s 2024-03-30T13:11:12.854 INFO:teuthology.orchestra.run.smithi066.stderr: 7750K .......... .......... .......... .......... .......... 4% 106M 38s 2024-03-30T13:11:12.854 INFO:teuthology.orchestra.run.smithi066.stderr: 7800K .......... .......... .......... .......... .......... 4% 116M 38s 2024-03-30T13:11:12.854 INFO:teuthology.orchestra.run.smithi066.stderr: 7850K .......... .......... .......... .......... .......... 4% 84.2M 37s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 7900K .......... .......... .......... .......... .......... 4% 1.12M 38s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 7950K .......... .......... .......... .......... .......... 4% 99.5M 38s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8000K .......... .......... .......... .......... .......... 4% 117M 38s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8050K .......... .......... .......... .......... .......... 4% 106M 37s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8100K .......... .......... .......... .......... .......... 4% 117M 37s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8150K .......... .......... .......... .......... .......... 4% 100M 37s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8200K .......... .......... .......... .......... .......... 4% 117M 37s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8250K .......... .......... .......... .......... .......... 4% 102M 36s 2024-03-30T13:11:12.903 INFO:teuthology.orchestra.run.smithi066.stderr: 8300K .......... .......... .......... .......... .......... 4% 87.6M 36s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8350K .......... .......... .......... .......... .......... 4% 1.53M 37s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8400K .......... .......... .......... .......... .......... 5% 92.7M 36s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8450K .......... .......... .......... .......... .......... 5% 105M 36s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8500K .......... .......... .......... .......... .......... 5% 90.2M 36s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8550K .......... .......... .......... .......... .......... 5% 112M 36s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8600K .......... .......... .......... .......... .......... 5% 89.2M 36s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8650K .......... .......... .......... .......... .......... 5% 114M 35s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8700K .......... .......... .......... .......... .......... 5% 99.7M 35s 2024-03-30T13:11:12.939 INFO:teuthology.orchestra.run.smithi066.stderr: 8750K .......... .......... .......... .......... .......... 5% 44.4M 35s 2024-03-30T13:11:12.982 INFO:teuthology.orchestra.run.smithi066.stderr: 8800K .......... .......... .......... .......... .......... 5% 1.58M 35s 2024-03-30T13:11:12.982 INFO:teuthology.orchestra.run.smithi066.stderr: 8850K .......... .......... .......... .......... .......... 5% 101M 35s 2024-03-30T13:11:12.982 INFO:teuthology.orchestra.run.smithi066.stderr: 8900K .......... .......... .......... .......... .......... 5% 76.5M 35s 2024-03-30T13:11:12.982 INFO:teuthology.orchestra.run.smithi066.stderr: 8950K .......... .......... .......... .......... .......... 5% 103M 35s 2024-03-30T13:11:12.983 INFO:teuthology.orchestra.run.smithi066.stderr: 9000K .......... .......... .......... .......... .......... 5% 71.5M 35s 2024-03-30T13:11:12.983 INFO:teuthology.orchestra.run.smithi066.stderr: 9050K .......... .......... .......... .......... .......... 5% 52.1M 34s 2024-03-30T13:11:12.983 INFO:teuthology.orchestra.run.smithi066.stderr: 9100K .......... .......... .......... .......... .......... 5% 27.8M 34s 2024-03-30T13:11:12.983 INFO:teuthology.orchestra.run.smithi066.stderr: 9150K .......... .......... .......... .......... .......... 5% 19.0M 34s 2024-03-30T13:11:12.983 INFO:teuthology.orchestra.run.smithi066.stderr: 9200K .......... .......... .......... .......... .......... 5% 9.25M 34s 2024-03-30T13:11:12.983 INFO:teuthology.orchestra.run.smithi066.stderr: 9250K .......... .......... .......... .......... .......... 5% 7.54M 34s 2024-03-30T13:11:13.013 INFO:teuthology.orchestra.run.smithi066.stderr: 9300K .......... .......... .......... .......... .......... 5% 2.99M 34s 2024-03-30T13:11:13.013 INFO:teuthology.orchestra.run.smithi066.stderr: 9350K .......... .......... .......... .......... .......... 5% 64.8M 34s 2024-03-30T13:11:13.013 INFO:teuthology.orchestra.run.smithi066.stderr: 9400K .......... .......... .......... .......... .......... 5% 73.0M 34s 2024-03-30T13:11:13.013 INFO:teuthology.orchestra.run.smithi066.stderr: 9450K .......... .......... .......... .......... .......... 5% 94.5M 33s 2024-03-30T13:11:13.013 INFO:teuthology.orchestra.run.smithi066.stderr: 9500K .......... .......... .......... .......... .......... 5% 104M 33s 2024-03-30T13:11:13.014 INFO:teuthology.orchestra.run.smithi066.stderr: 9550K .......... .......... .......... .......... .......... 5% 66.3M 33s 2024-03-30T13:11:13.014 INFO:teuthology.orchestra.run.smithi066.stderr: 9600K .......... .......... .......... .......... .......... 5% 28.9M 33s 2024-03-30T13:11:13.014 INFO:teuthology.orchestra.run.smithi066.stderr: 9650K .......... .......... .......... .......... .......... 5% 17.8M 33s 2024-03-30T13:11:13.014 INFO:teuthology.orchestra.run.smithi066.stderr: 9700K .......... .......... .......... .......... .......... 5% 19.2M 33s 2024-03-30T13:11:13.019 INFO:teuthology.orchestra.run.smithi066.stderr: 9750K .......... .......... .......... .......... .......... 5% 5.49M 33s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 9800K .......... .......... .......... .......... .......... 5% 2.19M 33s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 9850K .......... .......... .......... .......... .......... 5% 42.3M 33s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 9900K .......... .......... .......... .......... .......... 5% 98.1M 32s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 9950K .......... .......... .......... .......... .......... 5% 52.2M 32s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 10000K .......... .......... .......... .......... .......... 5% 49.7M 32s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 10050K .......... .......... .......... .......... .......... 5% 48.4M 32s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 10100K .......... .......... .......... .......... .......... 6% 28.3M 32s 2024-03-30T13:11:13.062 INFO:teuthology.orchestra.run.smithi066.stderr: 10150K .......... .......... .......... .......... .......... 6% 22.4M 32s 2024-03-30T13:11:13.063 INFO:teuthology.orchestra.run.smithi066.stderr: 10200K .......... .......... .......... .......... .......... 6% 23.4M 32s 2024-03-30T13:11:13.063 INFO:teuthology.orchestra.run.smithi066.stderr: 10250K .......... .......... .......... .......... .......... 6% 6.52M 32s 2024-03-30T13:11:13.063 INFO:teuthology.orchestra.run.smithi066.stderr: 10300K .......... .......... .......... .......... .......... 6% 58.4M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10350K .......... .......... .......... .......... .......... 6% 2.31M 32s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10400K .......... .......... .......... .......... .......... 6% 111M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10450K .......... .......... .......... .......... .......... 6% 31.9M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10500K .......... .......... .......... .......... .......... 6% 101M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10550K .......... .......... .......... .......... .......... 6% 47.6M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10600K .......... .......... .......... .......... .......... 6% 47.6M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10650K .......... .......... .......... .......... .......... 6% 42.0M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10700K .......... .......... .......... .......... .......... 6% 20.8M 31s 2024-03-30T13:11:13.099 INFO:teuthology.orchestra.run.smithi066.stderr: 10750K .......... .......... .......... .......... .......... 6% 47.9M 30s 2024-03-30T13:11:13.109 INFO:teuthology.orchestra.run.smithi066.stderr: 10800K .......... .......... .......... .......... .......... 6% 2.33M 31s 2024-03-30T13:11:13.114 INFO:teuthology.orchestra.run.smithi066.stderr: 10850K .......... .......... .......... .......... .......... 6% 18.4M 30s 2024-03-30T13:11:13.145 INFO:teuthology.orchestra.run.smithi066.stderr: 10900K .......... .......... .......... .......... .......... 6% 3.24M 31s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 10950K .......... .......... .......... .......... .......... 6% 112M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11000K .......... .......... .......... .......... .......... 6% 103M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11050K .......... .......... .......... .......... .......... 6% 89.9M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11100K .......... .......... .......... .......... .......... 6% 106M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11150K .......... .......... .......... .......... .......... 6% 28.6M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11200K .......... .......... .......... .......... .......... 6% 15.5M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11250K .......... .......... .......... .......... .......... 6% 31.9M 30s 2024-03-30T13:11:13.146 INFO:teuthology.orchestra.run.smithi066.stderr: 11300K .......... .......... .......... .......... .......... 6% 78.5M 30s 2024-03-30T13:11:13.159 INFO:teuthology.orchestra.run.smithi066.stderr: 11350K .......... .......... .......... .......... .......... 6% 2.08M 30s 2024-03-30T13:11:13.174 INFO:teuthology.orchestra.run.smithi066.stderr: 11400K .......... .......... .......... .......... .......... 6% 3.86M 30s 2024-03-30T13:11:13.175 INFO:teuthology.orchestra.run.smithi066.stderr: 11450K .......... .......... .......... .......... .......... 6% 45.0M 30s 2024-03-30T13:11:13.198 INFO:teuthology.orchestra.run.smithi066.stderr: 11500K .......... .......... .......... .......... .......... 6% 4.64M 30s 2024-03-30T13:11:13.198 INFO:teuthology.orchestra.run.smithi066.stderr: 11550K .......... .......... .......... .......... .......... 6% 76.7M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11600K .......... .......... .......... .......... .......... 6% 83.4M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11650K .......... .......... .......... .......... .......... 6% 105M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11700K .......... .......... .......... .......... .......... 6% 98.6M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11750K .......... .......... .......... .......... .......... 7% 55.9M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11800K .......... .......... .......... .......... .......... 7% 37.7M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11850K .......... .......... .......... .......... .......... 7% 95.3M 29s 2024-03-30T13:11:13.199 INFO:teuthology.orchestra.run.smithi066.stderr: 11900K .......... .......... .......... .......... .......... 7% 102M 29s 2024-03-30T13:11:13.210 INFO:teuthology.orchestra.run.smithi066.stderr: 11950K .......... .......... .......... .......... .......... 7% 2.26M 29s 2024-03-30T13:11:13.231 INFO:teuthology.orchestra.run.smithi066.stderr: 12000K .......... .......... .......... .......... .......... 7% 3.22M 29s 2024-03-30T13:11:13.231 INFO:teuthology.orchestra.run.smithi066.stderr: 12050K .......... .......... .......... .......... .......... 7% 91.7M 29s 2024-03-30T13:11:13.231 INFO:teuthology.orchestra.run.smithi066.stderr: 12100K .......... .......... .......... .......... .......... 7% 110M 29s 2024-03-30T13:11:13.254 INFO:teuthology.orchestra.run.smithi066.stderr: 12150K .......... .......... .......... .......... .......... 7% 3.93M 29s 2024-03-30T13:11:13.254 INFO:teuthology.orchestra.run.smithi066.stderr: 12200K .......... .......... .......... .......... .......... 7% 59.3M 29s 2024-03-30T13:11:13.254 INFO:teuthology.orchestra.run.smithi066.stderr: 12250K .......... .......... .......... .......... .......... 7% 48.4M 28s 2024-03-30T13:11:13.255 INFO:teuthology.orchestra.run.smithi066.stderr: 12300K .......... .......... .......... .......... .......... 7% 31.7M 28s 2024-03-30T13:11:13.255 INFO:teuthology.orchestra.run.smithi066.stderr: 12350K .......... .......... .......... .......... .......... 7% 44.9M 28s 2024-03-30T13:11:13.255 INFO:teuthology.orchestra.run.smithi066.stderr: 12400K .......... .......... .......... .......... .......... 7% 53.4M 28s 2024-03-30T13:11:13.255 INFO:teuthology.orchestra.run.smithi066.stderr: 12450K .......... .......... .......... .......... .......... 7% 46.0M 28s 2024-03-30T13:11:13.255 INFO:teuthology.orchestra.run.smithi066.stderr: 12500K .......... .......... .......... .......... .......... 7% 54.5M 28s 2024-03-30T13:11:13.255 INFO:teuthology.orchestra.run.smithi066.stderr: 12550K .......... .......... .......... .......... .......... 7% 42.5M 28s 2024-03-30T13:11:13.268 INFO:teuthology.orchestra.run.smithi066.stderr: 12600K .......... .......... .......... .......... .......... 7% 2.46M 28s 2024-03-30T13:11:13.284 INFO:teuthology.orchestra.run.smithi066.stderr: 12650K .......... .......... .......... .......... .......... 7% 3.58M 28s 2024-03-30T13:11:13.284 INFO:teuthology.orchestra.run.smithi066.stderr: 12700K .......... .......... .......... .......... .......... 7% 111M 28s 2024-03-30T13:11:13.284 INFO:teuthology.orchestra.run.smithi066.stderr: 12750K .......... .......... .......... .......... .......... 7% 49.5M 28s 2024-03-30T13:11:13.308 INFO:teuthology.orchestra.run.smithi066.stderr: 12800K .......... .......... .......... .......... .......... 7% 4.96M 28s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 12850K .......... .......... .......... .......... .......... 7% 96.8M 28s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 12900K .......... .......... .......... .......... .......... 7% 88.7M 28s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 12950K .......... .......... .......... .......... .......... 7% 47.3M 27s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 13000K .......... .......... .......... .......... .......... 7% 107M 27s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 13050K .......... .......... .......... .......... .......... 7% 75.7M 27s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 13100K .......... .......... .......... .......... .......... 7% 105M 27s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 13150K .......... .......... .......... .......... .......... 7% 107M 27s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 13200K .......... .......... .......... .......... .......... 7% 59.2M 27s 2024-03-30T13:11:13.309 INFO:teuthology.orchestra.run.smithi066.stderr: 13250K .......... .......... .......... .......... .......... 7% 114M 27s 2024-03-30T13:11:13.347 INFO:teuthology.orchestra.run.smithi066.stderr: 13300K .......... .......... .......... .......... .......... 7% 1013K 27s 2024-03-30T13:11:13.352 INFO:teuthology.orchestra.run.smithi066.stderr: 13350K .......... .......... .......... .......... .......... 7% 32.2M 27s 2024-03-30T13:11:13.352 INFO:teuthology.orchestra.run.smithi066.stderr: 13400K .......... .......... .......... .......... .......... 7% 102M 27s 2024-03-30T13:11:13.377 INFO:teuthology.orchestra.run.smithi066.stderr: 13450K .......... .......... .......... .......... .......... 8% 4.97M 27s 2024-03-30T13:11:13.377 INFO:teuthology.orchestra.run.smithi066.stderr: 13500K .......... .......... .......... .......... .......... 8% 119M 27s 2024-03-30T13:11:13.377 INFO:teuthology.orchestra.run.smithi066.stderr: 13550K .......... .......... .......... .......... .......... 8% 49.2M 27s 2024-03-30T13:11:13.377 INFO:teuthology.orchestra.run.smithi066.stderr: 13600K .......... .......... .......... .......... .......... 8% 112M 27s 2024-03-30T13:11:13.377 INFO:teuthology.orchestra.run.smithi066.stderr: 13650K .......... .......... .......... .......... .......... 8% 107M 27s 2024-03-30T13:11:13.378 INFO:teuthology.orchestra.run.smithi066.stderr: 13700K .......... .......... .......... .......... .......... 8% 114M 27s 2024-03-30T13:11:13.378 INFO:teuthology.orchestra.run.smithi066.stderr: 13750K .......... .......... .......... .......... .......... 8% 109M 26s 2024-03-30T13:11:13.378 INFO:teuthology.orchestra.run.smithi066.stderr: 13800K .......... .......... .......... .......... .......... 8% 103M 26s 2024-03-30T13:11:13.378 INFO:teuthology.orchestra.run.smithi066.stderr: 13850K .......... .......... .......... .......... .......... 8% 49.5M 26s 2024-03-30T13:11:13.378 INFO:teuthology.orchestra.run.smithi066.stderr: 13900K .......... .......... .......... .......... .......... 8% 50.1M 26s 2024-03-30T13:11:13.378 INFO:teuthology.orchestra.run.smithi066.stderr: 13950K .......... .......... .......... .......... .......... 8% 102M 26s 2024-03-30T13:11:13.413 INFO:teuthology.orchestra.run.smithi066.stderr: 14000K .......... .......... .......... .......... .......... 8% 1.12M 26s 2024-03-30T13:11:13.413 INFO:teuthology.orchestra.run.smithi066.stderr: 14050K .......... .......... .......... .......... .......... 8% 47.9M 26s 2024-03-30T13:11:13.413 INFO:teuthology.orchestra.run.smithi066.stderr: 14100K .......... .......... .......... .......... .......... 8% 113M 26s 2024-03-30T13:11:13.413 INFO:teuthology.orchestra.run.smithi066.stderr: 14150K .......... .......... .......... .......... .......... 8% 40.4M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14200K .......... .......... .......... .......... .......... 8% 7.12M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14250K .......... .......... .......... .......... .......... 8% 40.9M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14300K .......... .......... .......... .......... .......... 8% 50.7M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14350K .......... .......... .......... .......... .......... 8% 112M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14400K .......... .......... .......... .......... .......... 8% 61.5M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14450K .......... .......... .......... .......... .......... 8% 83.6M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14500K .......... .......... .......... .......... .......... 8% 93.6M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14550K .......... .......... .......... .......... .......... 8% 111M 26s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14600K .......... .......... .......... .......... .......... 8% 48.6M 25s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14650K .......... .......... .......... .......... .......... 8% 47.9M 25s 2024-03-30T13:11:13.439 INFO:teuthology.orchestra.run.smithi066.stderr: 14700K .......... .......... .......... .......... .......... 8% 85.4M 25s 2024-03-30T13:11:13.449 INFO:teuthology.orchestra.run.smithi066.stderr: 14750K .......... .......... .......... .......... .......... 8% 2.64M 25s 2024-03-30T13:11:13.449 INFO:teuthology.orchestra.run.smithi066.stderr: 14800K .......... .......... .......... .......... .......... 8% 54.3M 25s 2024-03-30T13:11:13.449 INFO:teuthology.orchestra.run.smithi066.stderr: 14850K .......... .......... .......... .......... .......... 8% 90.8M 25s 2024-03-30T13:11:13.450 INFO:teuthology.orchestra.run.smithi066.stderr: 14900K .......... .......... .......... .......... .......... 8% 43.5M 25s 2024-03-30T13:11:13.455 INFO:teuthology.orchestra.run.smithi066.stderr: 14950K .......... .......... .......... .......... .......... 8% 5.75M 25s 2024-03-30T13:11:13.460 INFO:teuthology.orchestra.run.smithi066.stderr: 15000K .......... .......... .......... .......... .......... 8% 8.68M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15050K .......... .......... .......... .......... .......... 8% 9.81M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15100K .......... .......... .......... .......... .......... 8% 108M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15150K .......... .......... .......... .......... .......... 9% 108M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15200K .......... .......... .......... .......... .......... 9% 97.6M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15250K .......... .......... .......... .......... .......... 9% 109M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15300K .......... .......... .......... .......... .......... 9% 108M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15350K .......... .......... .......... .......... .......... 9% 106M 25s 2024-03-30T13:11:13.498 INFO:teuthology.orchestra.run.smithi066.stderr: 15400K .......... .......... .......... .......... .......... 9% 103M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15450K .......... .......... .......... .......... .......... 9% 84.1M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15500K .......... .......... .......... .......... .......... 9% 104M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15550K .......... .......... .......... .......... .......... 9% 5.03M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15600K .......... .......... .......... .......... .......... 9% 54.0M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15650K .......... .......... .......... .......... .......... 9% 41.5M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15700K .......... .......... .......... .......... .......... 9% 6.25M 24s 2024-03-30T13:11:13.499 INFO:teuthology.orchestra.run.smithi066.stderr: 15750K .......... .......... .......... .......... .......... 9% 45.5M 24s 2024-03-30T13:11:13.506 INFO:teuthology.orchestra.run.smithi066.stderr: 15800K .......... .......... .......... .......... .......... 9% 3.17M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 15850K .......... .......... .......... .......... .......... 9% 17.1M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 15900K .......... .......... .......... .......... .......... 9% 93.6M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 15950K .......... .......... .......... .......... .......... 9% 119M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16000K .......... .......... .......... .......... .......... 9% 109M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16050K .......... .......... .......... .......... .......... 9% 84.7M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16100K .......... .......... .......... .......... .......... 9% 98.8M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16150K .......... .......... .......... .......... .......... 9% 105M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16200K .......... .......... .......... .......... .......... 9% 103M 24s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16250K .......... .......... .......... .......... .......... 9% 81.6M 23s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16300K .......... .......... .......... .......... .......... 9% 75.8M 23s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16350K .......... .......... .......... .......... .......... 9% 32.7M 23s 2024-03-30T13:11:13.548 INFO:teuthology.orchestra.run.smithi066.stderr: 16400K .......... .......... .......... .......... .......... 9% 49.0M 23s 2024-03-30T13:11:13.549 INFO:teuthology.orchestra.run.smithi066.stderr: 16450K .......... .......... .......... .......... .......... 9% 102M 23s 2024-03-30T13:11:13.549 INFO:teuthology.orchestra.run.smithi066.stderr: 16500K .......... .......... .......... .......... .......... 9% 38.9M 23s 2024-03-30T13:11:13.549 INFO:teuthology.orchestra.run.smithi066.stderr: 16550K .......... .......... .......... .......... .......... 9% 6.28M 23s 2024-03-30T13:11:13.549 INFO:teuthology.orchestra.run.smithi066.stderr: 16600K .......... .......... .......... .......... .......... 9% 41.6M 23s 2024-03-30T13:11:13.557 INFO:teuthology.orchestra.run.smithi066.stderr: 16650K .......... .......... .......... .......... .......... 9% 1.87M 23s 2024-03-30T13:11:13.557 INFO:teuthology.orchestra.run.smithi066.stderr: 16700K .......... .......... .......... .......... .......... 9% 21.5M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 16750K .......... .......... .......... .......... .......... 9% 11.5M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 16800K .......... .......... .......... .......... .......... 9% 24.1M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 16850K .......... .......... .......... .......... .......... 10% 20.4M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 16900K .......... .......... .......... .......... .......... 10% 101M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 16950K .......... .......... .......... .......... .......... 10% 114M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17000K .......... .......... .......... .......... .......... 10% 99.6M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17050K .......... .......... .......... .......... .......... 10% 101M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17100K .......... .......... .......... .......... .......... 10% 98.7M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17150K .......... .......... .......... .......... .......... 10% 90.7M 23s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17200K .......... .......... .......... .......... .......... 10% 72.2M 22s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17250K .......... .......... .......... .......... .......... 10% 57.6M 22s 2024-03-30T13:11:13.597 INFO:teuthology.orchestra.run.smithi066.stderr: 17300K .......... .......... .......... .......... .......... 10% 61.1M 22s 2024-03-30T13:11:13.598 INFO:teuthology.orchestra.run.smithi066.stderr: 17350K .......... .......... .......... .......... .......... 10% 80.8M 22s 2024-03-30T13:11:13.598 INFO:teuthology.orchestra.run.smithi066.stderr: 17400K .......... .......... .......... .......... .......... 10% 66.5M 22s 2024-03-30T13:11:13.598 INFO:teuthology.orchestra.run.smithi066.stderr: 17450K .......... .......... .......... .......... .......... 10% 118M 22s 2024-03-30T13:11:13.598 INFO:teuthology.orchestra.run.smithi066.stderr: 17500K .......... .......... .......... .......... .......... 10% 49.5M 22s 2024-03-30T13:11:19.559 INFO:teuthology.orchestra.run.smithi066.stderr: 17550K .......... .......... .......... .......... .......... 10% 1.87M 22s 2024-03-30T13:11:19.559 INFO:teuthology.orchestra.run.smithi066.stderr: 17600K .......... .......... .......... .......... .......... 10% 88.0M 22s 2024-03-30T13:11:19.559 INFO:teuthology.orchestra.run.smithi066.stderr: 17650K .......... .......... .......... .......... .......... 10% 101M 22s 2024-03-30T13:11:19.559 INFO:teuthology.orchestra.run.smithi066.stderr: 17700K .......... .......... .......... .......... .......... 10% 93.4M 22s 2024-03-30T13:11:19.559 INFO:teuthology.orchestra.run.smithi066.stderr: 17750K .......... .......... .......... .......... .......... 10% 119M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 17800K .......... .......... .......... .......... .......... 10% 108M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 17850K .......... .......... .......... .......... .......... 10% 111M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 17900K .......... .......... .......... .......... .......... 10% 105M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 17950K .......... .......... .......... .......... .......... 10% 115M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18000K .......... .......... .......... .......... .......... 10% 99.9M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18050K .......... .......... .......... .......... .......... 10% 109M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18100K .......... .......... .......... .......... .......... 10% 106M 22s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18150K .......... .......... .......... .......... .......... 10% 66.9M 21s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18200K .......... .......... .......... .......... .......... 10% 33.6M 21s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18250K .......... .......... .......... .......... .......... 10% 24.8M 21s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18300K .......... .......... .......... .......... .......... 10% 31.9M 21s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18350K .......... .......... .......... .......... .......... 10% 31.6M 21s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18400K .......... .......... .......... .......... .......... 10% 24.5M 21s 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi066.stderr: 18450K .......... .......... .......... .......... .......... 10% 2.35M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18500K .......... .......... .......... .......... .......... 11% 79.8M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18550K .......... .......... .......... .......... .......... 11% 90.4M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18600K .......... .......... .......... .......... .......... 11% 101M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18650K .......... .......... .......... .......... .......... 11% 84.9M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18700K .......... .......... .......... .......... .......... 11% 107M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18750K .......... .......... .......... .......... .......... 11% 117M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18800K .......... .......... .......... .......... .......... 11% 103M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18850K .......... .......... .......... .......... .......... 11% 67.5M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18900K .......... .......... .......... .......... .......... 11% 24.7M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 18950K .......... .......... .......... .......... .......... 11% 41.3M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 19000K .......... .......... .......... .......... .......... 11% 24.7M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 19050K .......... .......... .......... .......... .......... 11% 23.7M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 19100K .......... .......... .......... .......... .......... 11% 40.0M 21s 2024-03-30T13:11:19.561 INFO:teuthology.orchestra.run.smithi066.stderr: 19150K .......... .......... .......... .......... .......... 11% 25.2M 21s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19200K .......... .......... .......... .......... .......... 11% 41.6M 21s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19250K .......... .......... .......... .......... .......... 11% 25.1M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19300K .......... .......... .......... .......... .......... 11% 24.6M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19350K .......... .......... .......... .......... .......... 11% 42.1M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19400K .......... .......... .......... .......... .......... 11% 29.3M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19450K .......... .......... .......... .......... .......... 11% 2.77M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19500K .......... .......... .......... .......... .......... 11% 84.1M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19550K .......... .......... .......... .......... .......... 11% 86.1M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19600K .......... .......... .......... .......... .......... 11% 105M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19650K .......... .......... .......... .......... .......... 11% 91.2M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19700K .......... .......... .......... .......... .......... 11% 88.0M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19750K .......... .......... .......... .......... .......... 11% 110M 20s 2024-03-30T13:11:19.562 INFO:teuthology.orchestra.run.smithi066.stderr: 19800K .......... .......... .......... .......... .......... 11% 92.1M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 19850K .......... .......... .......... .......... .......... 11% 89.6M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 19900K .......... .......... .......... .......... .......... 11% 98.1M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 19950K .......... .......... .......... .......... .......... 11% 99.0M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20000K .......... .......... .......... .......... .......... 11% 108M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20050K .......... .......... .......... .......... .......... 11% 19.3M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20100K .......... .......... .......... .......... .......... 11% 4.69M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20150K .......... .......... .......... .......... .......... 11% 91.0M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20200K .......... .......... .......... .......... .......... 12% 74.5M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20250K .......... .......... .......... .......... .......... 12% 97.6M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20300K .......... .......... .......... .......... .......... 12% 69.4M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20350K .......... .......... .......... .......... .......... 12% 84.8M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20400K .......... .......... .......... .......... .......... 12% 73.0M 19s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20450K .......... .......... .......... .......... .......... 12% 3.68M 20s 2024-03-30T13:11:19.563 INFO:teuthology.orchestra.run.smithi066.stderr: 20500K .......... .......... .......... .......... .......... 12% 110M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20550K .......... .......... .......... .......... .......... 12% 112M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20600K .......... .......... .......... .......... .......... 12% 105M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20650K .......... .......... .......... .......... .......... 12% 93.5M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20700K .......... .......... .......... .......... .......... 12% 82.2M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20750K .......... .......... .......... .......... .......... 12% 107M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20800K .......... .......... .......... .......... .......... 12% 95.4M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20850K .......... .......... .......... .......... .......... 12% 39.9M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20900K .......... .......... .......... .......... .......... 12% 23.9M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 20950K .......... .......... .......... .......... .......... 12% 25.1M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 21000K .......... .......... .......... .......... .......... 12% 39.9M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 21050K .......... .......... .......... .......... .......... 12% 24.4M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 21100K .......... .......... .......... .......... .......... 12% 32.6M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 21150K .......... .......... .......... .......... .......... 12% 9.28M 19s 2024-03-30T13:11:19.564 INFO:teuthology.orchestra.run.smithi066.stderr: 21200K .......... .......... .......... .......... .......... 12% 74.8M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21250K .......... .......... .......... .......... .......... 12% 90.7M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21300K .......... .......... .......... .......... .......... 12% 107M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21350K .......... .......... .......... .......... .......... 12% 27.7M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21400K .......... .......... .......... .......... .......... 12% 36.6M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21450K .......... .......... .......... .......... .......... 12% 23.1M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21500K .......... .......... .......... .......... .......... 12% 2.72M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21550K .......... .......... .......... .......... .......... 12% 108M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21600K .......... .......... .......... .......... .......... 12% 112M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21650K .......... .......... .......... .......... .......... 12% 101M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21700K .......... .......... .......... .......... .......... 12% 108M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21750K .......... .......... .......... .......... .......... 12% 110M 19s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21800K .......... .......... .......... .......... .......... 12% 111M 18s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21850K .......... .......... .......... .......... .......... 12% 114M 18s 2024-03-30T13:11:19.565 INFO:teuthology.orchestra.run.smithi066.stderr: 21900K .......... .......... .......... .......... .......... 13% 75.6M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 21950K .......... .......... .......... .......... .......... 13% 98.9M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22000K .......... .......... .......... .......... .......... 13% 113M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22050K .......... .......... .......... .......... .......... 13% 96.3M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22100K .......... .......... .......... .......... .......... 13% 106M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22150K .......... .......... .......... .......... .......... 13% 81.3M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22200K .......... .......... .......... .......... .......... 13% 56.1M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22250K .......... .......... .......... .......... .......... 13% 12.9M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22300K .......... .......... .......... .......... .......... 13% 98.8M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22350K .......... .......... .......... .......... .......... 13% 81.0M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22400K .......... .......... .......... .......... .......... 13% 26.9M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22450K .......... .......... .......... .......... .......... 13% 20.5M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22500K .......... .......... .......... .......... .......... 13% 11.7M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22550K .......... .......... .......... .......... .......... 13% 6.17M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22600K .......... .......... .......... .......... .......... 13% 6.68M 18s 2024-03-30T13:11:19.566 INFO:teuthology.orchestra.run.smithi066.stderr: 22650K .......... .......... .......... .......... .......... 13% 108M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 22700K .......... .......... .......... .......... .......... 13% 101M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 22750K .......... .......... .......... .......... .......... 13% 119M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 22800K .......... .......... .......... .......... .......... 13% 107M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 22850K .......... .......... .......... .......... .......... 13% 113M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 22900K .......... .......... .......... .......... .......... 13% 101M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 22950K .......... .......... .......... .......... .......... 13% 109M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23000K .......... .......... .......... .......... .......... 13% 43.5M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23050K .......... .......... .......... .......... .......... 13% 24.0M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23100K .......... .......... .......... .......... .......... 13% 23.9M 18s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23150K .......... .......... .......... .......... .......... 13% 39.4M 17s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23200K .......... .......... .......... .......... .......... 13% 23.5M 17s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23250K .......... .......... .......... .......... .......... 13% 43.9M 17s 2024-03-30T13:11:19.567 INFO:teuthology.orchestra.run.smithi066.stderr: 23300K .......... .......... .......... .......... .......... 13% 24.7M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23350K .......... .......... .......... .......... .......... 13% 34.7M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23400K .......... .......... .......... .......... .......... 13% 20.3M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23450K .......... .......... .......... .......... .......... 13% 29.5M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23500K .......... .......... .......... .......... .......... 13% 52.9M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23550K .......... .......... .......... .......... .......... 14% 27.5M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23600K .......... .......... .......... .......... .......... 14% 29.6M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23650K .......... .......... .......... .......... .......... 14% 27.8M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23700K .......... .......... .......... .......... .......... 14% 24.9M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23750K .......... .......... .......... .......... .......... 14% 43.7M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23800K .......... .......... .......... .......... .......... 14% 3.77M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23850K .......... .......... .......... .......... .......... 14% 17.0M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23900K .......... .......... .......... .......... .......... 14% 106M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 23950K .......... .......... .......... .......... .......... 14% 104M 17s 2024-03-30T13:11:19.568 INFO:teuthology.orchestra.run.smithi066.stderr: 24000K .......... .......... .......... .......... .......... 14% 103M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24050K .......... .......... .......... .......... .......... 14% 78.1M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24100K .......... .......... .......... .......... .......... 14% 81.5M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24150K .......... .......... .......... .......... .......... 14% 81.3M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24200K .......... .......... .......... .......... .......... 14% 59.3M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24250K .......... .......... .......... .......... .......... 14% 86.6M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24300K .......... .......... .......... .......... .......... 14% 104M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24350K .......... .......... .......... .......... .......... 14% 80.0M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24400K .......... .......... .......... .......... .......... 14% 66.8M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24450K .......... .......... .......... .......... .......... 14% 25.5M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24500K .......... .......... .......... .......... .......... 14% 24.3M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24550K .......... .......... .......... .......... .......... 14% 43.1M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24600K .......... .......... .......... .......... .......... 14% 30.0M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24650K .......... .......... .......... .......... .......... 14% 30.0M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24700K .......... .......... .......... .......... .......... 14% 25.3M 17s 2024-03-30T13:11:19.569 INFO:teuthology.orchestra.run.smithi066.stderr: 24750K .......... .......... .......... .......... .......... 14% 11.3M 17s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 24800K .......... .......... .......... .......... .......... 14% 7.17M 17s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 24850K .......... .......... .......... .......... .......... 14% 5.98M 17s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 24900K .......... .......... .......... .......... .......... 14% 71.1M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 24950K .......... .......... .......... .......... .......... 14% 30.8M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25000K .......... .......... .......... .......... .......... 14% 39.3M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25050K .......... .......... .......... .......... .......... 14% 74.7M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25100K .......... .......... .......... .......... .......... 14% 117M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25150K .......... .......... .......... .......... .......... 14% 100M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25200K .......... .......... .......... .......... .......... 14% 93.8M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25250K .......... .......... .......... .......... .......... 15% 107M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25300K .......... .......... .......... .......... .......... 15% 110M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25350K .......... .......... .......... .......... .......... 15% 107M 16s 2024-03-30T13:11:19.570 INFO:teuthology.orchestra.run.smithi066.stderr: 25400K .......... .......... .......... .......... .......... 15% 113M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25450K .......... .......... .......... .......... .......... 15% 85.6M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25500K .......... .......... .......... .......... .......... 15% 98.5M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25550K .......... .......... .......... .......... .......... 15% 93.8M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25600K .......... .......... .......... .......... .......... 15% 24.8M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25650K .......... .......... .......... .......... .......... 15% 34.1M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25700K .......... .......... .......... .......... .......... 15% 28.3M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25750K .......... .......... .......... .......... .......... 15% 23.0M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25800K .......... .......... .......... .......... .......... 15% 43.1M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25850K .......... .......... .......... .......... .......... 15% 24.5M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25900K .......... .......... .......... .......... .......... 15% 34.8M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 25950K .......... .......... .......... .......... .......... 15% 23.7M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 26000K .......... .......... .......... .......... .......... 15% 32.4M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 26050K .......... .......... .......... .......... .......... 15% 36.1M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 26100K .......... .......... .......... .......... .......... 15% 6.00M 16s 2024-03-30T13:11:19.571 INFO:teuthology.orchestra.run.smithi066.stderr: 26150K .......... .......... .......... .......... .......... 15% 18.7M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26200K .......... .......... .......... .......... .......... 15% 17.1M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26250K .......... .......... .......... .......... .......... 15% 6.17M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26300K .......... .......... .......... .......... .......... 15% 14.2M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26350K .......... .......... .......... .......... .......... 15% 94.1M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26400K .......... .......... .......... .......... .......... 15% 113M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26450K .......... .......... .......... .......... .......... 15% 92.6M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26500K .......... .......... .......... .......... .......... 15% 96.7M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26550K .......... .......... .......... .......... .......... 15% 91.9M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26600K .......... .......... .......... .......... .......... 15% 90.4M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26650K .......... .......... .......... .......... .......... 15% 109M 16s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26700K .......... .......... .......... .......... .......... 15% 54.3M 15s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26750K .......... .......... .......... .......... .......... 15% 41.7M 15s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26800K .......... .......... .......... .......... .......... 15% 61.3M 15s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26850K .......... .......... .......... .......... .......... 15% 71.2M 15s 2024-03-30T13:11:19.572 INFO:teuthology.orchestra.run.smithi066.stderr: 26900K .......... .......... .......... .......... .......... 15% 109M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 26950K .......... .......... .......... .......... .......... 16% 104M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27000K .......... .......... .......... .......... .......... 16% 84.2M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27050K .......... .......... .......... .......... .......... 16% 88.0M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27100K .......... .......... .......... .......... .......... 16% 29.8M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27150K .......... .......... .......... .......... .......... 16% 37.3M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27200K .......... .......... .......... .......... .......... 16% 23.7M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27250K .......... .......... .......... .......... .......... 16% 24.1M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27300K .......... .......... .......... .......... .......... 16% 53.2M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27350K .......... .......... .......... .......... .......... 16% 24.0M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27400K .......... .......... .......... .......... .......... 16% 4.87M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27450K .......... .......... .......... .......... .......... 16% 13.4M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27500K .......... .......... .......... .......... .......... 16% 11.4M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27550K .......... .......... .......... .......... .......... 16% 51.7M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27600K .......... .......... .......... .......... .......... 16% 50.2M 15s 2024-03-30T13:11:19.573 INFO:teuthology.orchestra.run.smithi066.stderr: 27650K .......... .......... .......... .......... .......... 16% 77.2M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 27700K .......... .......... .......... .......... .......... 16% 110M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 27750K .......... .......... .......... .......... .......... 16% 100M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 27800K .......... .......... .......... .......... .......... 16% 115M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 27850K .......... .......... .......... .......... .......... 16% 86.1M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 27900K .......... .......... .......... .......... .......... 16% 113M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 27950K .......... .......... .......... .......... .......... 16% 108M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28000K .......... .......... .......... .......... .......... 16% 86.2M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28050K .......... .......... .......... .......... .......... 16% 79.6M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28100K .......... .......... .......... .......... .......... 16% 66.2M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28150K .......... .......... .......... .......... .......... 16% 22.4M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28200K .......... .......... .......... .......... .......... 16% 44.8M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28250K .......... .......... .......... .......... .......... 16% 24.0M 15s 2024-03-30T13:11:19.574 INFO:teuthology.orchestra.run.smithi066.stderr: 28300K .......... .......... .......... .......... .......... 16% 42.3M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28350K .......... .......... .......... .......... .......... 16% 24.6M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28400K .......... .......... .......... .......... .......... 16% 37.3M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28450K .......... .......... .......... .......... .......... 16% 24.1M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28500K .......... .......... .......... .......... .......... 16% 27.9M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28550K .......... .......... .......... .......... .......... 16% 40.1M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28600K .......... .......... .......... .......... .......... 16% 23.5M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28650K .......... .......... .......... .......... .......... 17% 30.6M 15s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28700K .......... .......... .......... .......... .......... 17% 26.0M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28750K .......... .......... .......... .......... .......... 17% 23.3M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28800K .......... .......... .......... .......... .......... 17% 67.0M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28850K .......... .......... .......... .......... .......... 17% 25.6M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28900K .......... .......... .......... .......... .......... 17% 4.14M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 28950K .......... .......... .......... .......... .......... 17% 14.0M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 29000K .......... .......... .......... .......... .......... 17% 41.4M 14s 2024-03-30T13:11:19.575 INFO:teuthology.orchestra.run.smithi066.stderr: 29050K .......... .......... .......... .......... .......... 17% 45.8M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29100K .......... .......... .......... .......... .......... 17% 29.7M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29150K .......... .......... .......... .......... .......... 17% 30.1M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29200K .......... .......... .......... .......... .......... 17% 45.6M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29250K .......... .......... .......... .......... .......... 17% 34.1M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29300K .......... .......... .......... .......... .......... 17% 89.5M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29350K .......... .......... .......... .......... .......... 17% 60.6M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29400K .......... .......... .......... .......... .......... 17% 79.8M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29450K .......... .......... .......... .......... .......... 17% 88.3M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29500K .......... .......... .......... .......... .......... 17% 61.5M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29550K .......... .......... .......... .......... .......... 17% 71.6M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29600K .......... .......... .......... .......... .......... 17% 25.0M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29650K .......... .......... .......... .......... .......... 17% 27.2M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29700K .......... .......... .......... .......... .......... 17% 21.8M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29750K .......... .......... .......... .......... .......... 17% 16.7M 14s 2024-03-30T13:11:19.576 INFO:teuthology.orchestra.run.smithi066.stderr: 29800K .......... .......... .......... .......... .......... 17% 26.3M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 29850K .......... .......... .......... .......... .......... 17% 31.3M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 29900K .......... .......... .......... .......... .......... 17% 31.2M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 29950K .......... .......... .......... .......... .......... 17% 19.1M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30000K .......... .......... .......... .......... .......... 17% 16.1M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30050K .......... .......... .......... .......... .......... 17% 3.61M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30100K .......... .......... .......... .......... .......... 17% 18.0M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30150K .......... .......... .......... .......... .......... 17% 30.0M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30200K .......... .......... .......... .......... .......... 17% 45.1M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30250K .......... .......... .......... .......... .......... 17% 46.8M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30300K .......... .......... .......... .......... .......... 18% 48.3M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30350K .......... .......... .......... .......... .......... 18% 48.4M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30400K .......... .......... .......... .......... .......... 18% 31.1M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30450K .......... .......... .......... .......... .......... 18% 28.2M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30500K .......... .......... .......... .......... .......... 18% 24.3M 14s 2024-03-30T13:11:19.577 INFO:teuthology.orchestra.run.smithi066.stderr: 30550K .......... .......... .......... .......... .......... 18% 35.7M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30600K .......... .......... .......... .......... .......... 18% 52.4M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30650K .......... .......... .......... .......... .......... 18% 47.6M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30700K .......... .......... .......... .......... .......... 18% 41.2M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30750K .......... .......... .......... .......... .......... 18% 46.6M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30800K .......... .......... .......... .......... .......... 18% 46.1M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30850K .......... .......... .......... .......... .......... 18% 108M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30900K .......... .......... .......... .......... .......... 18% 43.3M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 30950K .......... .......... .......... .......... .......... 18% 97.7M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 31000K .......... .......... .......... .......... .......... 18% 75.8M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 31050K .......... .......... .......... .......... .......... 18% 27.6M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 31100K .......... .......... .......... .......... .......... 18% 48.1M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 31150K .......... .......... .......... .......... .......... 18% 30.4M 14s 2024-03-30T13:11:19.578 INFO:teuthology.orchestra.run.smithi066.stderr: 31200K .......... .......... .......... .......... .......... 18% 24.9M 14s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31250K .......... .......... .......... .......... .......... 18% 24.2M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31300K .......... .......... .......... .......... .......... 18% 25.1M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31350K .......... .......... .......... .......... .......... 18% 36.9M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31400K .......... .......... .......... .......... .......... 18% 18.6M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31450K .......... .......... .......... .......... .......... 18% 24.0M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31500K .......... .......... .......... .......... .......... 18% 46.3M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31550K .......... .......... .......... .......... .......... 18% 37.8M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31600K .......... .......... .......... .......... .......... 18% 5.90M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31650K .......... .......... .......... .......... .......... 18% 3.96M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31700K .......... .......... .......... .......... .......... 18% 74.5M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31750K .......... .......... .......... .......... .......... 18% 34.6M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31800K .......... .......... .......... .......... .......... 18% 49.1M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31850K .......... .......... .......... .......... .......... 18% 47.0M 13s 2024-03-30T13:11:19.579 INFO:teuthology.orchestra.run.smithi066.stderr: 31900K .......... .......... .......... .......... .......... 18% 39.6M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 31950K .......... .......... .......... .......... .......... 18% 28.2M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32000K .......... .......... .......... .......... .......... 19% 27.8M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32050K .......... .......... .......... .......... .......... 19% 23.6M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32100K .......... .......... .......... .......... .......... 19% 101M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32150K .......... .......... .......... .......... .......... 19% 93.3M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32200K .......... .......... .......... .......... .......... 19% 112M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32250K .......... .......... .......... .......... .......... 19% 104M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32300K .......... .......... .......... .......... .......... 19% 99.8M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32350K .......... .......... .......... .......... .......... 19% 108M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32400K .......... .......... .......... .......... .......... 19% 102M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32450K .......... .......... .......... .......... .......... 19% 107M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32500K .......... .......... .......... .......... .......... 19% 106M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32550K .......... .......... .......... .......... .......... 19% 40.1M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32600K .......... .......... .......... .......... .......... 19% 31.4M 13s 2024-03-30T13:11:19.580 INFO:teuthology.orchestra.run.smithi066.stderr: 32650K .......... .......... .......... .......... .......... 19% 18.2M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 32700K .......... .......... .......... .......... .......... 19% 52.1M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 32750K .......... .......... .......... .......... .......... 19% 44.5M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 32800K .......... .......... .......... .......... .......... 19% 20.7M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 32850K .......... .......... .......... .......... .......... 19% 38.3M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 32900K .......... .......... .......... .......... .......... 19% 24.4M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 32950K .......... .......... .......... .......... .......... 19% 26.5M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33000K .......... .......... .......... .......... .......... 19% 26.3M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33050K .......... .......... .......... .......... .......... 19% 17.7M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33100K .......... .......... .......... .......... .......... 19% 42.7M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33150K .......... .......... .......... .......... .......... 19% 36.4M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33200K .......... .......... .......... .......... .......... 19% 78.2M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33250K .......... .......... .......... .......... .......... 19% 18.7M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33300K .......... .......... .......... .......... .......... 19% 63.3M 13s 2024-03-30T13:11:19.581 INFO:teuthology.orchestra.run.smithi066.stderr: 33350K .......... .......... .......... .......... .......... 19% 67.9M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33400K .......... .......... .......... .......... .......... 19% 70.2M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33450K .......... .......... .......... .......... .......... 19% 62.0M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33500K .......... .......... .......... .......... .......... 19% 105M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33550K .......... .......... .......... .......... .......... 19% 73.6M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33600K .......... .......... .......... .......... .......... 19% 75.3M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33650K .......... .......... .......... .......... .......... 19% 55.5M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33700K .......... .......... .......... .......... .......... 20% 64.0M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33750K .......... .......... .......... .......... .......... 20% 98.5M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33800K .......... .......... .......... .......... .......... 20% 81.9M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33850K .......... .......... .......... .......... .......... 20% 91.2M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33900K .......... .......... .......... .......... .......... 20% 100M 13s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 33950K .......... .......... .......... .......... .......... 20% 56.5M 12s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 34000K .......... .......... .......... .......... .......... 20% 46.5M 12s 2024-03-30T13:11:19.582 INFO:teuthology.orchestra.run.smithi066.stderr: 34050K .......... .......... .......... .......... .......... 20% 4.70M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34100K .......... .......... .......... .......... .......... 20% 7.38M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34150K .......... .......... .......... .......... .......... 20% 10.1M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34200K .......... .......... .......... .......... .......... 20% 28.3M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34250K .......... .......... .......... .......... .......... 20% 31.3M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34300K .......... .......... .......... .......... .......... 20% 24.4M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34350K .......... .......... .......... .......... .......... 20% 17.2M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34400K .......... .......... .......... .......... .......... 20% 8.88M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34450K .......... .......... .......... .......... .......... 20% 16.9M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34500K .......... .......... .......... .......... .......... 20% 59.5M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34550K .......... .......... .......... .......... .......... 20% 25.9M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34600K .......... .......... .......... .......... .......... 20% 25.9M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34650K .......... .......... .......... .......... .......... 20% 9.70M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34700K .......... .......... .......... .......... .......... 20% 19.3M 12s 2024-03-30T13:11:19.583 INFO:teuthology.orchestra.run.smithi066.stderr: 34750K .......... .......... .......... .......... .......... 20% 46.4M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 34800K .......... .......... .......... .......... .......... 20% 20.1M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 34850K .......... .......... .......... .......... .......... 20% 43.8M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 34900K .......... .......... .......... .......... .......... 20% 24.2M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 34950K .......... .......... .......... .......... .......... 20% 38.1M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35000K .......... .......... .......... .......... .......... 20% 26.9M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35050K .......... .......... .......... .......... .......... 20% 24.9M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35100K .......... .......... .......... .......... .......... 20% 38.9M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35150K .......... .......... .......... .......... .......... 20% 45.6M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35200K .......... .......... .......... .......... .......... 20% 49.8M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35250K .......... .......... .......... .......... .......... 20% 46.8M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35300K .......... .......... .......... .......... .......... 20% 44.6M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35350K .......... .......... .......... .......... .......... 21% 45.5M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35400K .......... .......... .......... .......... .......... 21% 125M 12s 2024-03-30T13:11:19.584 INFO:teuthology.orchestra.run.smithi066.stderr: 35450K .......... .......... .......... .......... .......... 21% 22.8M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35500K .......... .......... .......... .......... .......... 21% 47.3M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35550K .......... .......... .......... .......... .......... 21% 23.3M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35600K .......... .......... .......... .......... .......... 21% 47.1M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35650K .......... .......... .......... .......... .......... 21% 119M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35700K .......... .......... .......... .......... .......... 21% 49.5M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35750K .......... .......... .......... .......... .......... 21% 47.0M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35800K .......... .......... .......... .......... .......... 21% 22.6M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35850K .......... .......... .......... .......... .......... 21% 49.0M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35900K .......... .......... .......... .......... .......... 21% 45.9M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 35950K .......... .......... .......... .......... .......... 21% 31.5M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 36000K .......... .......... .......... .......... .......... 21% 29.5M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 36050K .......... .......... .......... .......... .......... 21% 47.9M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 36100K .......... .......... .......... .......... .......... 21% 28.7M 12s 2024-03-30T13:11:19.585 INFO:teuthology.orchestra.run.smithi066.stderr: 36150K .......... .......... .......... .......... .......... 21% 47.0M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36200K .......... .......... .......... .......... .......... 21% 22.7M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36250K .......... .......... .......... .......... .......... 21% 48.1M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36300K .......... .......... .......... .......... .......... 21% 30.4M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36350K .......... .......... .......... .......... .......... 21% 41.7M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36400K .......... .......... .......... .......... .......... 21% 28.8M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36450K .......... .......... .......... .......... .......... 21% 22.2M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36500K .......... .......... .......... .......... .......... 21% 44.7M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36550K .......... .......... .......... .......... .......... 21% 22.5M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36600K .......... .......... .......... .......... .......... 21% 30.8M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36650K .......... .......... .......... .......... .......... 21% 42.6M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36700K .......... .......... .......... .......... .......... 21% 21.4M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36750K .......... .......... .......... .......... .......... 21% 110M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36800K .......... .......... .......... .......... .......... 21% 31.3M 12s 2024-03-30T13:11:19.586 INFO:teuthology.orchestra.run.smithi066.stderr: 36850K .......... .......... .......... .......... .......... 21% 22.1M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 36900K .......... .......... .......... .......... .......... 21% 112M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 36950K .......... .......... .......... .......... .......... 21% 27.3M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37000K .......... .......... .......... .......... .......... 21% 23.9M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37050K .......... .......... .......... .......... .......... 22% 33.3M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37100K .......... .......... .......... .......... .......... 22% 26.6M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37150K .......... .......... .......... .......... .......... 22% 34.6M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37200K .......... .......... .......... .......... .......... 22% 4.87M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37250K .......... .......... .......... .......... .......... 22% 4.47M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37300K .......... .......... .......... .......... .......... 22% 48.8M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37350K .......... .......... .......... .......... .......... 22% 60.9M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37400K .......... .......... .......... .......... .......... 22% 73.9M 12s 2024-03-30T13:11:19.587 INFO:teuthology.orchestra.run.smithi066.stderr: 37450K .......... .......... .......... .......... .......... 22% 31.9M 12s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37500K .......... .......... .......... .......... .......... 22% 31.1M 12s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37550K .......... .......... .......... .......... .......... 22% 48.5M 12s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37600K .......... .......... .......... .......... .......... 22% 47.5M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37650K .......... .......... .......... .......... .......... 22% 47.9M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37700K .......... .......... .......... .......... .......... 22% 47.5M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37750K .......... .......... .......... .......... .......... 22% 42.6M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37800K .......... .......... .......... .......... .......... 22% 48.0M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37850K .......... .......... .......... .......... .......... 22% 48.4M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37900K .......... .......... .......... .......... .......... 22% 48.2M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 37950K .......... .......... .......... .......... .......... 22% 47.6M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 38000K .......... .......... .......... .......... .......... 22% 44.4M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 38050K .......... .......... .......... .......... .......... 22% 45.6M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 38100K .......... .......... .......... .......... .......... 22% 102M 11s 2024-03-30T13:11:19.588 INFO:teuthology.orchestra.run.smithi066.stderr: 38150K .......... .......... .......... .......... .......... 22% 111M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38200K .......... .......... .......... .......... .......... 22% 47.1M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38250K .......... .......... .......... .......... .......... 22% 30.7M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38300K .......... .......... .......... .......... .......... 22% 31.4M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38350K .......... .......... .......... .......... .......... 22% 25.7M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38400K .......... .......... .......... .......... .......... 22% 391M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38450K .......... .......... .......... .......... .......... 22% 68.4M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38500K .......... .......... .......... .......... .......... 22% 25.6M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38550K .......... .......... .......... .......... .......... 22% 28.3M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38600K .......... .......... .......... .......... .......... 22% 29.3M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38650K .......... .......... .......... .......... .......... 22% 48.9M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38700K .......... .......... .......... .......... .......... 22% 45.3M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38750K .......... .......... .......... .......... .......... 23% 30.8M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38800K .......... .......... .......... .......... .......... 23% 29.0M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38850K .......... .......... .......... .......... .......... 23% 59.1M 11s 2024-03-30T13:11:19.589 INFO:teuthology.orchestra.run.smithi066.stderr: 38900K .......... .......... .......... .......... .......... 23% 32.0M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 38950K .......... .......... .......... .......... .......... 23% 31.7M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39000K .......... .......... .......... .......... .......... 23% 37.9M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39050K .......... .......... .......... .......... .......... 23% 42.0M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39100K .......... .......... .......... .......... .......... 23% 20.0M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39150K .......... .......... .......... .......... .......... 23% 5.27M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39200K .......... .......... .......... .......... .......... 23% 11.6M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39250K .......... .......... .......... .......... .......... 23% 45.9M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39300K .......... .......... .......... .......... .......... 23% 108M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39350K .......... .......... .......... .......... .......... 23% 43.4M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39400K .......... .......... .......... .......... .......... 23% 101M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39450K .......... .......... .......... .......... .......... 23% 30.9M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39500K .......... .......... .......... .......... .......... 23% 30.1M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39550K .......... .......... .......... .......... .......... 23% 46.4M 11s 2024-03-30T13:11:19.590 INFO:teuthology.orchestra.run.smithi066.stderr: 39600K .......... .......... .......... .......... .......... 23% 39.6M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39650K .......... .......... .......... .......... .......... 23% 104M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39700K .......... .......... .......... .......... .......... 23% 30.5M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39750K .......... .......... .......... .......... .......... 23% 48.8M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39800K .......... .......... .......... .......... .......... 23% 48.1M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39850K .......... .......... .......... .......... .......... 23% 28.9M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39900K .......... .......... .......... .......... .......... 23% 114M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 39950K .......... .......... .......... .......... .......... 23% 31.4M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40000K .......... .......... .......... .......... .......... 23% 93.0M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40050K .......... .......... .......... .......... .......... 23% 95.5M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40100K .......... .......... .......... .......... .......... 23% 96.0M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40150K .......... .......... .......... .......... .......... 23% 106M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40200K .......... .......... .......... .......... .......... 23% 31.9M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40250K .......... .......... .......... .......... .......... 23% 27.4M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40300K .......... .......... .......... .......... .......... 23% 30.0M 11s 2024-03-30T13:11:19.591 INFO:teuthology.orchestra.run.smithi066.stderr: 40350K .......... .......... .......... .......... .......... 23% 30.8M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40400K .......... .......... .......... .......... .......... 23% 6.38M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40450K .......... .......... .......... .......... .......... 24% 8.22M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40500K .......... .......... .......... .......... .......... 24% 7.92M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40550K .......... .......... .......... .......... .......... 24% 30.0M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40600K .......... .......... .......... .......... .......... 24% 40.5M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40650K .......... .......... .......... .......... .......... 24% 38.1M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40700K .......... .......... .......... .......... .......... 24% 111M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40750K .......... .......... .......... .......... .......... 24% 24.1M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40800K .......... .......... .......... .......... .......... 24% 91.1M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40850K .......... .......... .......... .......... .......... 24% 19.0M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40900K .......... .......... .......... .......... .......... 24% 35.4M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 40950K .......... .......... .......... .......... .......... 24% 113M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 41000K .......... .......... .......... .......... .......... 24% 30.3M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 41050K .......... .......... .......... .......... .......... 24% 45.4M 11s 2024-03-30T13:11:19.592 INFO:teuthology.orchestra.run.smithi066.stderr: 41100K .......... .......... .......... .......... .......... 24% 94.7M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41150K .......... .......... .......... .......... .......... 24% 93.0M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41200K .......... .......... .......... .......... .......... 24% 111M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41250K .......... .......... .......... .......... .......... 24% 22.7M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41300K .......... .......... .......... .......... .......... 24% 44.0M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41350K .......... .......... .......... .......... .......... 24% 41.2M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41400K .......... .......... .......... .......... .......... 24% 22.9M 11s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41450K .......... .......... .......... .......... .......... 24% 49.1M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41500K .......... .......... .......... .......... .......... 24% 48.8M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41550K .......... .......... .......... .......... .......... 24% 30.6M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41600K .......... .......... .......... .......... .......... 24% 47.2M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41650K .......... .......... .......... .......... .......... 24% 33.3M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41700K .......... .......... .......... .......... .......... 24% 83.2M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41750K .......... .......... .......... .......... .......... 24% 29.8M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41800K .......... .......... .......... .......... .......... 24% 58.2M 10s 2024-03-30T13:11:19.593 INFO:teuthology.orchestra.run.smithi066.stderr: 41850K .......... .......... .......... .......... .......... 24% 41.6M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 41900K .......... .......... .......... .......... .......... 24% 42.6M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 41950K .......... .......... .......... .......... .......... 24% 104M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42000K .......... .......... .......... .......... .......... 24% 30.5M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42050K .......... .......... .......... .......... .......... 24% 44.8M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42100K .......... .......... .......... .......... .......... 25% 29.0M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42150K .......... .......... .......... .......... .......... 25% 29.8M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42200K .......... .......... .......... .......... .......... 25% 22.9M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42250K .......... .......... .......... .......... .......... 25% 3.65M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42300K .......... .......... .......... .......... .......... 25% 43.9M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42350K .......... .......... .......... .......... .......... 25% 46.9M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42400K .......... .......... .......... .......... .......... 25% 43.5M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42450K .......... .......... .......... .......... .......... 25% 96.2M 10s 2024-03-30T13:11:19.594 INFO:teuthology.orchestra.run.smithi066.stderr: 42500K .......... .......... .......... .......... .......... 25% 47.7M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42550K .......... .......... .......... .......... .......... 25% 45.0M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42600K .......... .......... .......... .......... .......... 25% 46.2M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42650K .......... .......... .......... .......... .......... 25% 46.8M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42700K .......... .......... .......... .......... .......... 25% 44.0M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42750K .......... .......... .......... .......... .......... 25% 22.9M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42800K .......... .......... .......... .......... .......... 25% 65.3M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42850K .......... .......... .......... .......... .......... 25% 65.0M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42900K .......... .......... .......... .......... .......... 25% 81.8M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 42950K .......... .......... .......... .......... .......... 25% 27.7M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 43000K .......... .......... .......... .......... .......... 25% 25.8M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 43050K .......... .......... .......... .......... .......... 25% 47.6M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 43100K .......... .......... .......... .......... .......... 25% 30.2M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 43150K .......... .......... .......... .......... .......... 25% 29.4M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 43200K .......... .......... .......... .......... .......... 25% 42.6M 10s 2024-03-30T13:11:19.595 INFO:teuthology.orchestra.run.smithi066.stderr: 43250K .......... .......... .......... .......... .......... 25% 40.1M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43300K .......... .......... .......... .......... .......... 25% 60.0M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43350K .......... .......... .......... .......... .......... 25% 21.8M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43400K .......... .......... .......... .......... .......... 25% 24.3M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43450K .......... .......... .......... .......... .......... 25% 16.4M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43500K .......... .......... .......... .......... .......... 25% 4.01M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43550K .......... .......... .......... .......... .......... 25% 70.8M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43600K .......... .......... .......... .......... .......... 25% 43.0M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43650K .......... .......... .......... .......... .......... 25% 22.7M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43700K .......... .......... .......... .......... .......... 25% 99.5M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43750K .......... .......... .......... .......... .......... 25% 30.1M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43800K .......... .......... .......... .......... .......... 26% 66.7M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43850K .......... .......... .......... .......... .......... 26% 44.5M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43900K .......... .......... .......... .......... .......... 26% 33.2M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 43950K .......... .......... .......... .......... .......... 26% 105M 10s 2024-03-30T13:11:19.596 INFO:teuthology.orchestra.run.smithi066.stderr: 44000K .......... .......... .......... .......... .......... 26% 29.8M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44050K .......... .......... .......... .......... .......... 26% 47.2M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44100K .......... .......... .......... .......... .......... 26% 49.0M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44150K .......... .......... .......... .......... .......... 26% 74.5M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44200K .......... .......... .......... .......... .......... 26% 86.4M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44250K .......... .......... .......... .......... .......... 26% 30.9M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44300K .......... .......... .......... .......... .......... 26% 103M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44350K .......... .......... .......... .......... .......... 26% 88.5M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44400K .......... .......... .......... .......... .......... 26% 102M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44450K .......... .......... .......... .......... .......... 26% 106M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44500K .......... .......... .......... .......... .......... 26% 23.1M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44550K .......... .......... .......... .......... .......... 26% 29.3M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44600K .......... .......... .......... .......... .......... 26% 45.5M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44650K .......... .......... .......... .......... .......... 26% 20.9M 10s 2024-03-30T13:11:19.597 INFO:teuthology.orchestra.run.smithi066.stderr: 44700K .......... .......... .......... .......... .......... 26% 31.7M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 44750K .......... .......... .......... .......... .......... 26% 20.3M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 44800K .......... .......... .......... .......... .......... 26% 37.4M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 44850K .......... .......... .......... .......... .......... 26% 29.4M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 44900K .......... .......... .......... .......... .......... 26% 25.4M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 44950K .......... .......... .......... .......... .......... 26% 42.6M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45000K .......... .......... .......... .......... .......... 26% 42.5M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45050K .......... .......... .......... .......... .......... 26% 44.8M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45100K .......... .......... .......... .......... .......... 26% 30.3M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45150K .......... .......... .......... .......... .......... 26% 13.5M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45200K .......... .......... .......... .......... .......... 26% 5.59M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45250K .......... .......... .......... .......... .......... 26% 10.1M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45300K .......... .......... .......... .......... .......... 26% 46.5M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45350K .......... .......... .......... .......... .......... 26% 48.7M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45400K .......... .......... .......... .......... .......... 26% 31.1M 10s 2024-03-30T13:11:19.598 INFO:teuthology.orchestra.run.smithi066.stderr: 45450K .......... .......... .......... .......... .......... 26% 100M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45500K .......... .......... .......... .......... .......... 27% 30.4M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45550K .......... .......... .......... .......... .......... 27% 28.2M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45600K .......... .......... .......... .......... .......... 27% 48.7M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45650K .......... .......... .......... .......... .......... 27% 42.7M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45700K .......... .......... .......... .......... .......... 27% 31.5M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45750K .......... .......... .......... .......... .......... 27% 47.6M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45800K .......... .......... .......... .......... .......... 27% 41.2M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45850K .......... .......... .......... .......... .......... 27% 111M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45900K .......... .......... .......... .......... .......... 27% 107M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 45950K .......... .......... .......... .......... .......... 27% 29.8M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 46000K .......... .......... .......... .......... .......... 27% 48.2M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 46050K .......... .......... .......... .......... .......... 27% 21.6M 10s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 46100K .......... .......... .......... .......... .......... 27% 117M 9s 2024-03-30T13:11:19.599 INFO:teuthology.orchestra.run.smithi066.stderr: 46150K .......... .......... .......... .......... .......... 27% 29.8M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46200K .......... .......... .......... .......... .......... 27% 46.5M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46250K .......... .......... .......... .......... .......... 27% 60.8M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46300K .......... .......... .......... .......... .......... 27% 26.4M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46350K .......... .......... .......... .......... .......... 27% 63.7M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46400K .......... .......... .......... .......... .......... 27% 23.0M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46450K .......... .......... .......... .......... .......... 27% 30.4M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46500K .......... .......... .......... .......... .......... 27% 14.3M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46550K .......... .......... .......... .......... .......... 27% 4.80M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46600K .......... .......... .......... .......... .......... 27% 46.9M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46650K .......... .......... .......... .......... .......... 27% 28.0M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46700K .......... .......... .......... .......... .......... 27% 29.0M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46750K .......... .......... .......... .......... .......... 27% 48.9M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46800K .......... .......... .......... .......... .......... 27% 43.1M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46850K .......... .......... .......... .......... .......... 27% 38.9M 9s 2024-03-30T13:11:19.600 INFO:teuthology.orchestra.run.smithi066.stderr: 46900K .......... .......... .......... .......... .......... 27% 29.8M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 46950K .......... .......... .......... .......... .......... 27% 45.4M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47000K .......... .......... .......... .......... .......... 27% 47.5M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47050K .......... .......... .......... .......... .......... 27% 46.8M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47100K .......... .......... .......... .......... .......... 27% 49.2M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47150K .......... .......... .......... .......... .......... 28% 48.1M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47200K .......... .......... .......... .......... .......... 28% 30.1M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47250K .......... .......... .......... .......... .......... 28% 49.0M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47300K .......... .......... .......... .......... .......... 28% 44.3M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47350K .......... .......... .......... .......... .......... 28% 106M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47400K .......... .......... .......... .......... .......... 28% 97.3M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47450K .......... .......... .......... .......... .......... 28% 23.0M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47500K .......... .......... .......... .......... .......... 28% 49.3M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47550K .......... .......... .......... .......... .......... 28% 26.9M 9s 2024-03-30T13:11:19.601 INFO:teuthology.orchestra.run.smithi066.stderr: 47600K .......... .......... .......... .......... .......... 28% 45.4M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47650K .......... .......... .......... .......... .......... 28% 27.0M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47700K .......... .......... .......... .......... .......... 28% 26.4M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47750K .......... .......... .......... .......... .......... 28% 81.9M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47800K .......... .......... .......... .......... .......... 28% 41.2M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47850K .......... .......... .......... .......... .......... 28% 48.3M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47900K .......... .......... .......... .......... .......... 28% 37.6M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 47950K .......... .......... .......... .......... .......... 28% 28.9M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48000K .......... .......... .......... .......... .......... 28% 43.7M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48050K .......... .......... .......... .......... .......... 28% 7.44M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48100K .......... .......... .......... .......... .......... 28% 6.25M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48150K .......... .......... .......... .......... .......... 28% 4.55M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48200K .......... .......... .......... .......... .......... 28% 30.1M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48250K .......... .......... .......... .......... .......... 28% 211M 9s 2024-03-30T13:11:19.602 INFO:teuthology.orchestra.run.smithi066.stderr: 48300K .......... .......... .......... .......... .......... 28% 35.7M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48350K .......... .......... .......... .......... .......... 28% 47.8M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48400K .......... .......... .......... .......... .......... 28% 36.5M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48450K .......... .......... .......... .......... .......... 28% 49.6M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48500K .......... .......... .......... .......... .......... 28% 46.3M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48550K .......... .......... .......... .......... .......... 28% 42.2M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48600K .......... .......... .......... .......... .......... 28% 49.2M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48650K .......... .......... .......... .......... .......... 28% 68.9M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48700K .......... .......... .......... .......... .......... 28% 49.4M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48750K .......... .......... .......... .......... .......... 28% 40.6M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48800K .......... .......... .......... .......... .......... 28% 34.5M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48850K .......... .......... .......... .......... .......... 29% 57.9M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48900K .......... .......... .......... .......... .......... 29% 46.5M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 48950K .......... .......... .......... .......... .......... 29% 32.0M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 49000K .......... .......... .......... .......... .......... 29% 55.5M 9s 2024-03-30T13:11:19.603 INFO:teuthology.orchestra.run.smithi066.stderr: 49050K .......... .......... .......... .......... .......... 29% 30.6M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49100K .......... .......... .......... .......... .......... 29% 41.5M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49150K .......... .......... .......... .......... .......... 29% 45.0M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49200K .......... .......... .......... .......... .......... 29% 43.9M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49250K .......... .......... .......... .......... .......... 29% 45.4M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49300K .......... .......... .......... .......... .......... 29% 40.9M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49350K .......... .......... .......... .......... .......... 29% 26.7M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49400K .......... .......... .......... .......... .......... 29% 46.7M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49450K .......... .......... .......... .......... .......... 29% 30.4M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49500K .......... .......... .......... .......... .......... 29% 44.5M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49550K .......... .......... .......... .......... .......... 29% 30.6M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49600K .......... .......... .......... .......... .......... 29% 42.1M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49650K .......... .......... .......... .......... .......... 29% 24.0M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49700K .......... .......... .......... .......... .......... 29% 4.00M 9s 2024-03-30T13:11:19.604 INFO:teuthology.orchestra.run.smithi066.stderr: 49750K .......... .......... .......... .......... .......... 29% 21.9M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 49800K .......... .......... .......... .......... .......... 29% 25.9M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 49850K .......... .......... .......... .......... .......... 29% 26.8M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 49900K .......... .......... .......... .......... .......... 29% 40.8M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 49950K .......... .......... .......... .......... .......... 29% 26.9M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50000K .......... .......... .......... .......... .......... 29% 94.2M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50050K .......... .......... .......... .......... .......... 29% 30.0M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50100K .......... .......... .......... .......... .......... 29% 50.6M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50150K .......... .......... .......... .......... .......... 29% 38.9M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50200K .......... .......... .......... .......... .......... 29% 36.6M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50250K .......... .......... .......... .......... .......... 29% 49.8M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50300K .......... .......... .......... .......... .......... 29% 30.3M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50350K .......... .......... .......... .......... .......... 29% 40.9M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50400K .......... .......... .......... .......... .......... 29% 29.6M 9s 2024-03-30T13:11:19.605 INFO:teuthology.orchestra.run.smithi066.stderr: 50450K .......... .......... .......... .......... .......... 29% 92.5M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50500K .......... .......... .......... .......... .......... 29% 50.1M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50550K .......... .......... .......... .......... .......... 30% 29.8M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50600K .......... .......... .......... .......... .......... 30% 85.4M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50650K .......... .......... .......... .......... .......... 30% 96.9M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50700K .......... .......... .......... .......... .......... 30% 37.4M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50750K .......... .......... .......... .......... .......... 30% 29.3M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50800K .......... .......... .......... .......... .......... 30% 105M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50850K .......... .......... .......... .......... .......... 30% 22.2M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50900K .......... .......... .......... .......... .......... 30% 105M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 50950K .......... .......... .......... .......... .......... 30% 22.5M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 51000K .......... .......... .......... .......... .......... 30% 45.7M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 51050K .......... .......... .......... .......... .......... 30% 43.6M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 51100K .......... .......... .......... .......... .......... 30% 35.9M 9s 2024-03-30T13:11:19.606 INFO:teuthology.orchestra.run.smithi066.stderr: 51150K .......... .......... .......... .......... .......... 30% 45.9M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51200K .......... .......... .......... .......... .......... 30% 29.0M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51250K .......... .......... .......... .......... .......... 30% 7.45M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51300K .......... .......... .......... .......... .......... 30% 7.76M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51350K .......... .......... .......... .......... .......... 30% 7.07M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51400K .......... .......... .......... .......... .......... 30% 40.5M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51450K .......... .......... .......... .......... .......... 30% 26.1M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51500K .......... .......... .......... .......... .......... 30% 36.7M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51550K .......... .......... .......... .......... .......... 30% 45.9M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51600K .......... .......... .......... .......... .......... 30% 27.8M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51650K .......... .......... .......... .......... .......... 30% 42.5M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51700K .......... .......... .......... .......... .......... 30% 41.8M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51750K .......... .......... .......... .......... .......... 30% 29.0M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51800K .......... .......... .......... .......... .......... 30% 88.4M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51850K .......... .......... .......... .......... .......... 30% 28.9M 9s 2024-03-30T13:11:19.607 INFO:teuthology.orchestra.run.smithi066.stderr: 51900K .......... .......... .......... .......... .......... 30% 64.8M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 51950K .......... .......... .......... .......... .......... 30% 38.4M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52000K .......... .......... .......... .......... .......... 30% 42.1M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52050K .......... .......... .......... .......... .......... 30% 41.6M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52100K .......... .......... .......... .......... .......... 30% 29.4M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52150K .......... .......... .......... .......... .......... 30% 46.6M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52200K .......... .......... .......... .......... .......... 30% 59.0M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52250K .......... .......... .......... .......... .......... 31% 95.8M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52300K .......... .......... .......... .......... .......... 31% 30.1M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52350K .......... .......... .......... .......... .......... 31% 73.4M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52400K .......... .......... .......... .......... .......... 31% 41.4M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52450K .......... .......... .......... .......... .......... 31% 48.1M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52500K .......... .......... .......... .......... .......... 31% 27.2M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52550K .......... .......... .......... .......... .......... 31% 64.2M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52600K .......... .......... .......... .......... .......... 31% 30.0M 8s 2024-03-30T13:11:19.608 INFO:teuthology.orchestra.run.smithi066.stderr: 52650K .......... .......... .......... .......... .......... 31% 103M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 52700K .......... .......... .......... .......... .......... 31% 22.3M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 52750K .......... .......... .......... .......... .......... 31% 28.7M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 52800K .......... .......... .......... .......... .......... 31% 43.8M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 52850K .......... .......... .......... .......... .......... 31% 18.9M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 52900K .......... .......... .......... .......... .......... 31% 10.2M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 52950K .......... .......... .......... .......... .......... 31% 4.51M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53000K .......... .......... .......... .......... .......... 31% 48.9M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53050K .......... .......... .......... .......... .......... 31% 47.5M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53100K .......... .......... .......... .......... .......... 31% 32.8M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53150K .......... .......... .......... .......... .......... 31% 64.6M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53200K .......... .......... .......... .......... .......... 31% 43.7M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53250K .......... .......... .......... .......... .......... 31% 28.9M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53300K .......... .......... .......... .......... .......... 31% 57.6M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53350K .......... .......... .......... .......... .......... 31% 36.4M 8s 2024-03-30T13:11:19.609 INFO:teuthology.orchestra.run.smithi066.stderr: 53400K .......... .......... .......... .......... .......... 31% 46.0M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53450K .......... .......... .......... .......... .......... 31% 33.2M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53500K .......... .......... .......... .......... .......... 31% 47.2M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53550K .......... .......... .......... .......... .......... 31% 41.2M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53600K .......... .......... .......... .......... .......... 31% 25.3M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53650K .......... .......... .......... .......... .......... 31% 29.9M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53700K .......... .......... .......... .......... .......... 31% 77.8M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53750K .......... .......... .......... .......... .......... 31% 89.2M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53800K .......... .......... .......... .......... .......... 31% 101M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53850K .......... .......... .......... .......... .......... 31% 30.0M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53900K .......... .......... .......... .......... .......... 32% 76.2M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 53950K .......... .......... .......... .......... .......... 32% 65.8M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 54000K .......... .......... .......... .......... .......... 32% 38.7M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 54050K .......... .......... .......... .......... .......... 32% 46.4M 8s 2024-03-30T13:11:19.610 INFO:teuthology.orchestra.run.smithi066.stderr: 54100K .......... .......... .......... .......... .......... 32% 28.8M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54150K .......... .......... .......... .......... .......... 32% 26.7M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54200K .......... .......... .......... .......... .......... 32% 41.7M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54250K .......... .......... .......... .......... .......... 32% 39.4M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54300K .......... .......... .......... .......... .......... 32% 78.0M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54350K .......... .......... .......... .......... .......... 32% 21.8M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54400K .......... .......... .......... .......... .......... 32% 12.7M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54450K .......... .......... .......... .......... .......... 32% 7.47M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54500K .......... .......... .......... .......... .......... 32% 8.83M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54550K .......... .......... .......... .......... .......... 32% 89.7M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54600K .......... .......... .......... .......... .......... 32% 45.6M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54650K .......... .......... .......... .......... .......... 32% 27.2M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54700K .......... .......... .......... .......... .......... 32% 47.9M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54750K .......... .......... .......... .......... .......... 32% 45.2M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54800K .......... .......... .......... .......... .......... 32% 39.0M 8s 2024-03-30T13:11:19.611 INFO:teuthology.orchestra.run.smithi066.stderr: 54850K .......... .......... .......... .......... .......... 32% 47.7M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 54900K .......... .......... .......... .......... .......... 32% 31.5M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 54950K .......... .......... .......... .......... .......... 32% 39.9M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55000K .......... .......... .......... .......... .......... 32% 32.8M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55050K .......... .......... .......... .......... .......... 32% 29.6M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55100K .......... .......... .......... .......... .......... 32% 100M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55150K .......... .......... .......... .......... .......... 32% 28.0M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55200K .......... .......... .......... .......... .......... 32% 41.6M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55250K .......... .......... .......... .......... .......... 32% 45.7M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55300K .......... .......... .......... .......... .......... 32% 41.9M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55350K .......... .......... .......... .......... .......... 32% 99.0M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55400K .......... .......... .......... .......... .......... 32% 92.3M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55450K .......... .......... .......... .......... .......... 32% 80.9M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55500K .......... .......... .......... .......... .......... 32% 49.8M 8s 2024-03-30T13:11:19.612 INFO:teuthology.orchestra.run.smithi066.stderr: 55550K .......... .......... .......... .......... .......... 32% 67.1M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55600K .......... .......... .......... .......... .......... 33% 45.0M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55650K .......... .......... .......... .......... .......... 33% 48.7M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55700K .......... .......... .......... .......... .......... 33% 35.8M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55750K .......... .......... .......... .......... .......... 33% 37.9M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55800K .......... .......... .......... .......... .......... 33% 30.0M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55850K .......... .......... .......... .......... .......... 33% 33.4M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55900K .......... .......... .......... .......... .......... 33% 35.8M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 55950K .......... .......... .......... .......... .......... 33% 7.62M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 56000K .......... .......... .......... .......... .......... 33% 4.78M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 56050K .......... .......... .......... .......... .......... 33% 41.7M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 56100K .......... .......... .......... .......... .......... 33% 26.2M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 56150K .......... .......... .......... .......... .......... 33% 31.0M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 56200K .......... .......... .......... .......... .......... 33% 49.2M 8s 2024-03-30T13:11:19.613 INFO:teuthology.orchestra.run.smithi066.stderr: 56250K .......... .......... .......... .......... .......... 33% 27.3M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56300K .......... .......... .......... .......... .......... 33% 28.1M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56350K .......... .......... .......... .......... .......... 33% 47.9M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56400K .......... .......... .......... .......... .......... 33% 38.9M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56450K .......... .......... .......... .......... .......... 33% 52.6M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56500K .......... .......... .......... .......... .......... 33% 29.3M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56550K .......... .......... .......... .......... .......... 33% 44.9M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56600K .......... .......... .......... .......... .......... 33% 29.9M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56650K .......... .......... .......... .......... .......... 33% 25.5M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56700K .......... .......... .......... .......... .......... 33% 101M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56750K .......... .......... .......... .......... .......... 33% 68.3M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56800K .......... .......... .......... .......... .......... 33% 96.3M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56850K .......... .......... .......... .......... .......... 33% 76.0M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56900K .......... .......... .......... .......... .......... 33% 61.8M 8s 2024-03-30T13:11:19.614 INFO:teuthology.orchestra.run.smithi066.stderr: 56950K .......... .......... .......... .......... .......... 33% 68.8M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57000K .......... .......... .......... .......... .......... 33% 29.7M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57050K .......... .......... .......... .......... .......... 33% 34.7M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57100K .......... .......... .......... .......... .......... 33% 32.7M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57150K .......... .......... .......... .......... .......... 33% 40.9M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57200K .......... .......... .......... .......... .......... 33% 44.4M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57250K .......... .......... .......... .......... .......... 33% 29.2M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57300K .......... .......... .......... .......... .......... 34% 46.6M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57350K .......... .......... .......... .......... .......... 34% 26.3M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57400K .......... .......... .......... .......... .......... 34% 26.1M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57450K .......... .......... .......... .......... .......... 34% 75.4M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57500K .......... .......... .......... .......... .......... 34% 5.00M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57550K .......... .......... .......... .......... .......... 34% 6.46M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57600K .......... .......... .......... .......... .......... 34% 28.0M 8s 2024-03-30T13:11:19.615 INFO:teuthology.orchestra.run.smithi066.stderr: 57650K .......... .......... .......... .......... .......... 34% 29.9M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 57700K .......... .......... .......... .......... .......... 34% 40.6M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 57750K .......... .......... .......... .......... .......... 34% 42.5M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 57800K .......... .......... .......... .......... .......... 34% 60.5M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 57850K .......... .......... .......... .......... .......... 34% 48.3M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 57900K .......... .......... .......... .......... .......... 34% 43.9M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 57950K .......... .......... .......... .......... .......... 34% 42.8M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58000K .......... .......... .......... .......... .......... 34% 45.3M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58050K .......... .......... .......... .......... .......... 34% 44.7M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58100K .......... .......... .......... .......... .......... 34% 32.2M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58150K .......... .......... .......... .......... .......... 34% 24.5M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58200K .......... .......... .......... .......... .......... 34% 59.5M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58250K .......... .......... .......... .......... .......... 34% 25.9M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58300K .......... .......... .......... .......... .......... 34% 45.0M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58350K .......... .......... .......... .......... .......... 34% 39.0M 8s 2024-03-30T13:11:19.616 INFO:teuthology.orchestra.run.smithi066.stderr: 58400K .......... .......... .......... .......... .......... 34% 33.1M 8s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58450K .......... .......... .......... .......... .......... 34% 103M 8s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58500K .......... .......... .......... .......... .......... 34% 88.6M 8s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58550K .......... .......... .......... .......... .......... 34% 32.9M 8s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58600K .......... .......... .......... .......... .......... 34% 27.7M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58650K .......... .......... .......... .......... .......... 34% 42.3M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58700K .......... .......... .......... .......... .......... 34% 46.4M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58750K .......... .......... .......... .......... .......... 34% 45.2M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58800K .......... .......... .......... .......... .......... 34% 28.2M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58850K .......... .......... .......... .......... .......... 34% 73.2M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58900K .......... .......... .......... .......... .......... 34% 32.3M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 58950K .......... .......... .......... .......... .......... 35% 13.5M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 59000K .......... .......... .......... .......... .......... 35% 7.28M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 59050K .......... .......... .......... .......... .......... 35% 7.10M 7s 2024-03-30T13:11:19.617 INFO:teuthology.orchestra.run.smithi066.stderr: 59100K .......... .......... .......... .......... .......... 35% 42.3M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59150K .......... .......... .......... .......... .......... 35% 27.8M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59200K .......... .......... .......... .......... .......... 35% 25.9M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59250K .......... .......... .......... .......... .......... 35% 44.6M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59300K .......... .......... .......... .......... .......... 35% 42.2M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59350K .......... .......... .......... .......... .......... 35% 44.0M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59400K .......... .......... .......... .......... .......... 35% 28.8M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59450K .......... .......... .......... .......... .......... 35% 34.9M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59500K .......... .......... .......... .......... .......... 35% 53.3M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59550K .......... .......... .......... .......... .......... 35% 38.1M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59600K .......... .......... .......... .......... .......... 35% 42.6M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59650K .......... .......... .......... .......... .......... 35% 32.0M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59700K .......... .......... .......... .......... .......... 35% 31.4M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59750K .......... .......... .......... .......... .......... 35% 55.3M 7s 2024-03-30T13:11:19.618 INFO:teuthology.orchestra.run.smithi066.stderr: 59800K .......... .......... .......... .......... .......... 35% 34.9M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 59850K .......... .......... .......... .......... .......... 35% 43.1M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 59900K .......... .......... .......... .......... .......... 35% 26.7M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 59950K .......... .......... .......... .......... .......... 35% 39.2M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60000K .......... .......... .......... .......... .......... 35% 85.5M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60050K .......... .......... .......... .......... .......... 35% 34.4M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60100K .......... .......... .......... .......... .......... 35% 84.5M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60150K .......... .......... .......... .......... .......... 35% 33.4M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60200K .......... .......... .......... .......... .......... 35% 30.4M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60250K .......... .......... .......... .......... .......... 35% 64.6M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60300K .......... .......... .......... .......... .......... 35% 45.1M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60350K .......... .......... .......... .......... .......... 35% 39.1M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60400K .......... .......... .......... .......... .......... 35% 41.5M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60450K .......... .......... .......... .......... .......... 35% 51.2M 7s 2024-03-30T13:11:19.619 INFO:teuthology.orchestra.run.smithi066.stderr: 60500K .......... .......... .......... .......... .......... 35% 37.6M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60550K .......... .......... .......... .......... .......... 35% 24.4M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60600K .......... .......... .......... .......... .......... 35% 6.60M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60650K .......... .......... .......... .......... .......... 36% 9.30M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60700K .......... .......... .......... .......... .......... 36% 32.8M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60750K .......... .......... .......... .......... .......... 36% 78.6M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60800K .......... .......... .......... .......... .......... 36% 30.8M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60850K .......... .......... .......... .......... .......... 36% 47.8M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60900K .......... .......... .......... .......... .......... 36% 37.5M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 60950K .......... .......... .......... .......... .......... 36% 32.4M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 61000K .......... .......... .......... .......... .......... 36% 46.7M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 61050K .......... .......... .......... .......... .......... 36% 32.8M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 61100K .......... .......... .......... .......... .......... 36% 29.6M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 61150K .......... .......... .......... .......... .......... 36% 38.5M 7s 2024-03-30T13:11:19.620 INFO:teuthology.orchestra.run.smithi066.stderr: 61200K .......... .......... .......... .......... .......... 36% 29.3M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61250K .......... .......... .......... .......... .......... 36% 53.5M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61300K .......... .......... .......... .......... .......... 36% 21.9M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61350K .......... .......... .......... .......... .......... 36% 41.8M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61400K .......... .......... .......... .......... .......... 36% 45.1M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61450K .......... .......... .......... .......... .......... 36% 27.9M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61500K .......... .......... .......... .......... .......... 36% 46.4M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61550K .......... .......... .......... .......... .......... 36% 70.7M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61600K .......... .......... .......... .......... .......... 36% 30.9M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61650K .......... .......... .......... .......... .......... 36% 38.6M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61700K .......... .......... .......... .......... .......... 36% 31.6M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61750K .......... .......... .......... .......... .......... 36% 38.6M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61800K .......... .......... .......... .......... .......... 36% 56.8M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61850K .......... .......... .......... .......... .......... 36% 45.6M 7s 2024-03-30T13:11:19.621 INFO:teuthology.orchestra.run.smithi066.stderr: 61900K .......... .......... .......... .......... .......... 36% 25.4M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 61950K .......... .......... .......... .......... .......... 36% 9.49M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62000K .......... .......... .......... .......... .......... 36% 7.63M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62050K .......... .......... .......... .......... .......... 36% 5.45M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62100K .......... .......... .......... .......... .......... 36% 31.0M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62150K .......... .......... .......... .......... .......... 36% 40.8M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62200K .......... .......... .......... .......... .......... 36% 29.3M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62250K .......... .......... .......... .......... .......... 36% 30.4M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62300K .......... .......... .......... .......... .......... 36% 41.7M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62350K .......... .......... .......... .......... .......... 37% 30.1M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62400K .......... .......... .......... .......... .......... 37% 59.1M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62450K .......... .......... .......... .......... .......... 37% 44.3M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62500K .......... .......... .......... .......... .......... 37% 35.0M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62550K .......... .......... .......... .......... .......... 37% 63.0M 7s 2024-03-30T13:11:19.622 INFO:teuthology.orchestra.run.smithi066.stderr: 62600K .......... .......... .......... .......... .......... 37% 46.4M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62650K .......... .......... .......... .......... .......... 37% 66.5M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62700K .......... .......... .......... .......... .......... 37% 35.9M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62750K .......... .......... .......... .......... .......... 37% 40.4M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62800K .......... .......... .......... .......... .......... 37% 48.5M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62850K .......... .......... .......... .......... .......... 37% 39.9M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62900K .......... .......... .......... .......... .......... 37% 47.0M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 62950K .......... .......... .......... .......... .......... 37% 46.8M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63000K .......... .......... .......... .......... .......... 37% 54.1M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63050K .......... .......... .......... .......... .......... 37% 54.9M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63100K .......... .......... .......... .......... .......... 37% 72.9M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63150K .......... .......... .......... .......... .......... 37% 54.9M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63200K .......... .......... .......... .......... .......... 37% 27.1M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63250K .......... .......... .......... .......... .......... 37% 55.5M 7s 2024-03-30T13:11:19.623 INFO:teuthology.orchestra.run.smithi066.stderr: 63300K .......... .......... .......... .......... .......... 37% 37.6M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63350K .......... .......... .......... .......... .......... 37% 46.2M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63400K .......... .......... .......... .......... .......... 37% 9.00M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63450K .......... .......... .......... .......... .......... 37% 27.0M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63500K .......... .......... .......... .......... .......... 37% 66.2M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63550K .......... .......... .......... .......... .......... 37% 104M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63600K .......... .......... .......... .......... .......... 37% 121M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63650K .......... .......... .......... .......... .......... 37% 112M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63700K .......... .......... .......... .......... .......... 37% 23.1M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63750K .......... .......... .......... .......... .......... 37% 10.5M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63800K .......... .......... .......... .......... .......... 37% 4.14M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63850K .......... .......... .......... .......... .......... 37% 48.6M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63900K .......... .......... .......... .......... .......... 37% 29.3M 7s 2024-03-30T13:11:19.624 INFO:teuthology.orchestra.run.smithi066.stderr: 63950K .......... .......... .......... .......... .......... 37% 35.4M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64000K .......... .......... .......... .......... .......... 37% 52.4M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64050K .......... .......... .......... .......... .......... 38% 29.7M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64100K .......... .......... .......... .......... .......... 38% 46.1M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64150K .......... .......... .......... .......... .......... 38% 29.8M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64200K .......... .......... .......... .......... .......... 38% 44.1M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64250K .......... .......... .......... .......... .......... 38% 39.8M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64300K .......... .......... .......... .......... .......... 38% 35.8M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64350K .......... .......... .......... .......... .......... 38% 42.8M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64400K .......... .......... .......... .......... .......... 38% 29.6M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64450K .......... .......... .......... .......... .......... 38% 29.4M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64500K .......... .......... .......... .......... .......... 38% 46.8M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64550K .......... .......... .......... .......... .......... 38% 48.2M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64600K .......... .......... .......... .......... .......... 38% 29.3M 7s 2024-03-30T13:11:19.625 INFO:teuthology.orchestra.run.smithi066.stderr: 64650K .......... .......... .......... .......... .......... 38% 75.1M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 64700K .......... .......... .......... .......... .......... 38% 83.8M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 64750K .......... .......... .......... .......... .......... 38% 61.4M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 64800K .......... .......... .......... .......... .......... 38% 80.7M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 64850K .......... .......... .......... .......... .......... 38% 97.8M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 64900K .......... .......... .......... .......... .......... 38% 39.1M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 64950K .......... .......... .......... .......... .......... 38% 28.0M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65000K .......... .......... .......... .......... .......... 38% 47.5M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65050K .......... .......... .......... .......... .......... 38% 30.9M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65100K .......... .......... .......... .......... .......... 38% 30.0M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65150K .......... .......... .......... .......... .......... 38% 47.7M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65200K .......... .......... .......... .......... .......... 38% 41.5M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65250K .......... .......... .......... .......... .......... 38% 12.5M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65300K .......... .......... .......... .......... .......... 38% 4.76M 7s 2024-03-30T13:11:19.626 INFO:teuthology.orchestra.run.smithi066.stderr: 65350K .......... .......... .......... .......... .......... 38% 44.7M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65400K .......... .......... .......... .......... .......... 38% 24.2M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65450K .......... .......... .......... .......... .......... 38% 36.9M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65500K .......... .......... .......... .......... .......... 38% 49.0M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65550K .......... .......... .......... .......... .......... 38% 22.2M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65600K .......... .......... .......... .......... .......... 38% 45.7M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65650K .......... .......... .......... .......... .......... 38% 43.4M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65700K .......... .......... .......... .......... .......... 39% 47.0M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65750K .......... .......... .......... .......... .......... 39% 28.8M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65800K .......... .......... .......... .......... .......... 39% 46.2M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65850K .......... .......... .......... .......... .......... 39% 40.3M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65900K .......... .......... .......... .......... .......... 39% 33.5M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 65950K .......... .......... .......... .......... .......... 39% 80.3M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 66000K .......... .......... .......... .......... .......... 39% 32.0M 7s 2024-03-30T13:11:19.627 INFO:teuthology.orchestra.run.smithi066.stderr: 66050K .......... .......... .......... .......... .......... 39% 46.8M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66100K .......... .......... .......... .......... .......... 39% 29.5M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66150K .......... .......... .......... .......... .......... 39% 42.8M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66200K .......... .......... .......... .......... .......... 39% 80.0M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66250K .......... .......... .......... .......... .......... 39% 44.6M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66300K .......... .......... .......... .......... .......... 39% 107M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66350K .......... .......... .......... .......... .......... 39% 29.7M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66400K .......... .......... .......... .......... .......... 39% 38.9M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66450K .......... .......... .......... .......... .......... 39% 45.8M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66500K .......... .......... .......... .......... .......... 39% 45.7M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66550K .......... .......... .......... .......... .......... 39% 46.8M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66600K .......... .......... .......... .......... .......... 39% 33.6M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66650K .......... .......... .......... .......... .......... 39% 37.0M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66700K .......... .......... .......... .......... .......... 39% 56.1M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66750K .......... .......... .......... .......... .......... 39% 25.4M 7s 2024-03-30T13:11:19.628 INFO:teuthology.orchestra.run.smithi066.stderr: 66800K .......... .......... .......... .......... .......... 39% 35.6M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 66850K .......... .......... .......... .......... .......... 39% 20.4M 6s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 66900K .......... .......... .......... .......... .......... 39% 522K 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 66950K .......... .......... .......... .......... .......... 39% 362M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67000K .......... .......... .......... .......... .......... 39% 497M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67050K .......... .......... .......... .......... .......... 39% 398M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67100K .......... .......... .......... .......... .......... 39% 482M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67150K .......... .......... .......... .......... .......... 39% 472M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67200K .......... .......... .......... .......... .......... 39% 424M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67250K .......... .......... .......... .......... .......... 39% 479M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67300K .......... .......... .......... .......... .......... 39% 462M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67350K .......... .......... .......... .......... .......... 39% 370M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67400K .......... .......... .......... .......... .......... 40% 451M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67450K .......... .......... .......... .......... .......... 40% 423M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67500K .......... .......... .......... .......... .......... 40% 468M 7s 2024-03-30T13:11:19.629 INFO:teuthology.orchestra.run.smithi066.stderr: 67550K .......... .......... .......... .......... .......... 40% 399M 7s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67600K .......... .......... .......... .......... .......... 40% 424M 7s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67650K .......... .......... .......... .......... .......... 40% 470M 7s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67700K .......... .......... .......... .......... .......... 40% 467M 7s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67750K .......... .......... .......... .......... .......... 40% 411M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67800K .......... .......... .......... .......... .......... 40% 409M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67850K .......... .......... .......... .......... .......... 40% 462M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67900K .......... .......... .......... .......... .......... 40% 506M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 67950K .......... .......... .......... .......... .......... 40% 386M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 68000K .......... .......... .......... .......... .......... 40% 468M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 68050K .......... .......... .......... .......... .......... 40% 447M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 68100K .......... .......... .......... .......... .......... 40% 482M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 68150K .......... .......... .......... .......... .......... 40% 408M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 68200K .......... .......... .......... .......... .......... 40% 454M 6s 2024-03-30T13:11:19.630 INFO:teuthology.orchestra.run.smithi066.stderr: 68250K .......... .......... .......... .......... .......... 40% 117M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68300K .......... .......... .......... .......... .......... 40% 475M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68350K .......... .......... .......... .......... .......... 40% 497M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68400K .......... .......... .......... .......... .......... 40% 367M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68450K .......... .......... .......... .......... .......... 40% 489M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68500K .......... .......... .......... .......... .......... 40% 426M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68550K .......... .......... .......... .......... .......... 40% 456M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68600K .......... .......... .......... .......... .......... 40% 364M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68650K .......... .......... .......... .......... .......... 40% 492M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68700K .......... .......... .......... .......... .......... 40% 430M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68750K .......... .......... .......... .......... .......... 40% 454M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68800K .......... .......... .......... .......... .......... 40% 355M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68850K .......... .......... .......... .......... .......... 40% 483M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68900K .......... .......... .......... .......... .......... 40% 479M 6s 2024-03-30T13:11:19.631 INFO:teuthology.orchestra.run.smithi066.stderr: 68950K .......... .......... .......... .......... .......... 40% 437M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69000K .......... .......... .......... .......... .......... 40% 434M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69050K .......... .......... .......... .......... .......... 40% 391M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69100K .......... .......... .......... .......... .......... 41% 481M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69150K .......... .......... .......... .......... .......... 41% 413M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69200K .......... .......... .......... .......... .......... 41% 397M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69250K .......... .......... .......... .......... .......... 41% 496M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69300K .......... .......... .......... .......... .......... 41% 409M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69350K .......... .......... .......... .......... .......... 41% 478M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69400K .......... .......... .......... .......... .......... 41% 409M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69450K .......... .......... .......... .......... .......... 41% 407M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69500K .......... .......... .......... .......... .......... 41% 451M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69550K .......... .......... .......... .......... .......... 41% 483M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69600K .......... .......... .......... .......... .......... 41% 374M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69650K .......... .......... .......... .......... .......... 41% 1.34M 6s 2024-03-30T13:11:19.632 INFO:teuthology.orchestra.run.smithi066.stderr: 69700K .......... .......... .......... .......... .......... 41% 226M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 69750K .......... .......... .......... .......... .......... 41% 173M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 69800K .......... .......... .......... .......... .......... 41% 113M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 69850K .......... .......... .......... .......... .......... 41% 113M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 69900K .......... .......... .......... .......... .......... 41% 114M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 69950K .......... .......... .......... .......... .......... 41% 111M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70000K .......... .......... .......... .......... .......... 41% 44.7M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70050K .......... .......... .......... .......... .......... 41% 32.6M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70100K .......... .......... .......... .......... .......... 41% 47.6M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70150K .......... .......... .......... .......... .......... 41% 34.2M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70200K .......... .......... .......... .......... .......... 41% 40.7M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70250K .......... .......... .......... .......... .......... 41% 23.2M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70300K .......... .......... .......... .......... .......... 41% 28.6M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70350K .......... .......... .......... .......... .......... 41% 12.6M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70400K .......... .......... .......... .......... .......... 41% 3.83M 6s 2024-03-30T13:11:19.633 INFO:teuthology.orchestra.run.smithi066.stderr: 70450K .......... .......... .......... .......... .......... 41% 32.2M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70500K .......... .......... .......... .......... .......... 41% 36.3M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70550K .......... .......... .......... .......... .......... 41% 45.5M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70600K .......... .......... .......... .......... .......... 41% 55.3M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70650K .......... .......... .......... .......... .......... 41% 44.3M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70700K .......... .......... .......... .......... .......... 41% 35.5M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70750K .......... .......... .......... .......... .......... 42% 32.8M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70800K .......... .......... .......... .......... .......... 42% 40.2M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70850K .......... .......... .......... .......... .......... 42% 55.2M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70900K .......... .......... .......... .......... .......... 42% 44.6M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 70950K .......... .......... .......... .......... .......... 42% 30.1M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 71000K .......... .......... .......... .......... .......... 42% 39.1M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 71050K .......... .......... .......... .......... .......... 42% 27.8M 6s 2024-03-30T13:11:19.634 INFO:teuthology.orchestra.run.smithi066.stderr: 71100K .......... .......... .......... .......... .......... 42% 111M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71150K .......... .......... .......... .......... .......... 42% 35.3M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71200K .......... .......... .......... .......... .......... 42% 27.5M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71250K .......... .......... .......... .......... .......... 42% 27.7M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71300K .......... .......... .......... .......... .......... 42% 39.4M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71350K .......... .......... .......... .......... .......... 42% 61.8M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71400K .......... .......... .......... .......... .......... 42% 25.8M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71450K .......... .......... .......... .......... .......... 42% 37.2M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71500K .......... .......... .......... .......... .......... 42% 29.8M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71550K .......... .......... .......... .......... .......... 42% 26.8M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71600K .......... .......... .......... .......... .......... 42% 103M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71650K .......... .......... .......... .......... .......... 42% 40.5M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71700K .......... .......... .......... .......... .......... 42% 51.5M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71750K .......... .......... .......... .......... .......... 42% 40.7M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71800K .......... .......... .......... .......... .......... 42% 4.77M 6s 2024-03-30T13:11:19.635 INFO:teuthology.orchestra.run.smithi066.stderr: 71850K .......... .......... .......... .......... .......... 42% 19.7M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 71900K .......... .......... .......... .......... .......... 42% 865K 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 71950K .......... .......... .......... .......... .......... 42% 29.9M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72000K .......... .......... .......... .......... .......... 42% 24.7M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72050K .......... .......... .......... .......... .......... 42% 202M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72100K .......... .......... .......... .......... .......... 42% 608M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72150K .......... .......... .......... .......... .......... 42% 527M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72200K .......... .......... .......... .......... .......... 42% 537M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72250K .......... .......... .......... .......... .......... 42% 595M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72300K .......... .......... .......... .......... .......... 42% 602M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72350K .......... .......... .......... .......... .......... 42% 527M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72400K .......... .......... .......... .......... .......... 42% 587M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72450K .......... .......... .......... .......... .......... 43% 610M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72500K .......... .......... .......... .......... .......... 43% 600M 6s 2024-03-30T13:11:19.636 INFO:teuthology.orchestra.run.smithi066.stderr: 72550K .......... .......... .......... .......... .......... 43% 539M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72600K .......... .......... .......... .......... .......... 43% 613M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72650K .......... .......... .......... .......... .......... 43% 613M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72700K .......... .......... .......... .......... .......... 43% 625M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72750K .......... .......... .......... .......... .......... 43% 516M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72800K .......... .......... .......... .......... .......... 43% 620M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72850K .......... .......... .......... .......... .......... 43% 611M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72900K .......... .......... .......... .......... .......... 43% 571M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 72950K .......... .......... .......... .......... .......... 43% 498M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 73000K .......... .......... .......... .......... .......... 43% 549M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 73050K .......... .......... .......... .......... .......... 43% 613M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 73100K .......... .......... .......... .......... .......... 43% 626M 6s 2024-03-30T13:11:19.637 INFO:teuthology.orchestra.run.smithi066.stderr: 73150K .......... .......... .......... .......... .......... 43% 552M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73200K .......... .......... .......... .......... .......... 43% 622M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73250K .......... .......... .......... .......... .......... 43% 608M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73300K .......... .......... .......... .......... .......... 43% 632M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73350K .......... .......... .......... .......... .......... 43% 544M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73400K .......... .......... .......... .......... .......... 43% 623M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73450K .......... .......... .......... .......... .......... 43% 581M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73500K .......... .......... .......... .......... .......... 43% 618M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73550K .......... .......... .......... .......... .......... 43% 537M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73600K .......... .......... .......... .......... .......... 43% 615M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73650K .......... .......... .......... .......... .......... 43% 624M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73700K .......... .......... .......... .......... .......... 43% 631M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73750K .......... .......... .......... .......... .......... 43% 531M 6s 2024-03-30T13:11:19.638 INFO:teuthology.orchestra.run.smithi066.stderr: 73800K .......... .......... .......... .......... .......... 43% 606M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 73850K .......... .......... .......... .......... .......... 43% 605M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 73900K .......... .......... .......... .......... .......... 43% 609M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 73950K .......... .......... .......... .......... .......... 43% 536M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74000K .......... .......... .......... .......... .......... 43% 605M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74050K .......... .......... .......... .......... .......... 43% 624M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74100K .......... .......... .......... .......... .......... 43% 624M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74150K .......... .......... .......... .......... .......... 44% 543M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74200K .......... .......... .......... .......... .......... 44% 590M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74250K .......... .......... .......... .......... .......... 44% 605M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74300K .......... .......... .......... .......... .......... 44% 10.6M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74350K .......... .......... .......... .......... .......... 44% 5.55M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74400K .......... .......... .......... .......... .......... 44% 85.1M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74450K .......... .......... .......... .......... .......... 44% 62.4M 6s 2024-03-30T13:11:19.639 INFO:teuthology.orchestra.run.smithi066.stderr: 74500K .......... .......... .......... .......... .......... 44% 69.8M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74550K .......... .......... .......... .......... .......... 44% 92.2M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74600K .......... .......... .......... .......... .......... 44% 70.3M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74650K .......... .......... .......... .......... .......... 44% 101M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74700K .......... .......... .......... .......... .......... 44% 96.5M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74750K .......... .......... .......... .......... .......... 44% 97.9M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74800K .......... .......... .......... .......... .......... 44% 94.8M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74850K .......... .......... .......... .......... .......... 44% 95.9M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74900K .......... .......... .......... .......... .......... 44% 81.9M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 74950K .......... .......... .......... .......... .......... 44% 64.5M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 75000K .......... .......... .......... .......... .......... 44% 34.6M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 75050K .......... .......... .......... .......... .......... 44% 4.90M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 75100K .......... .......... .......... .......... .......... 44% 110M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 75150K .......... .......... .......... .......... .......... 44% 114M 6s 2024-03-30T13:11:19.640 INFO:teuthology.orchestra.run.smithi066.stderr: 75200K .......... .......... .......... .......... .......... 44% 108M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75250K .......... .......... .......... .......... .......... 44% 113M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75300K .......... .......... .......... .......... .......... 44% 114M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75350K .......... .......... .......... .......... .......... 44% 95.2M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75400K .......... .......... .......... .......... .......... 44% 58.7M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75450K .......... .......... .......... .......... .......... 44% 24.6M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75500K .......... .......... .......... .......... .......... 44% 36.0M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75550K .......... .......... .......... .......... .......... 44% 24.6M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75600K .......... .......... .......... .......... .......... 44% 24.6M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75650K .......... .......... .......... .......... .......... 44% 45.5M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75700K .......... .......... .......... .......... .......... 44% 25.1M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75750K .......... .......... .......... .......... .......... 44% 36.3M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75800K .......... .......... .......... .......... .......... 44% 6.16M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75850K .......... .......... .......... .......... .......... 45% 12.0M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75900K .......... .......... .......... .......... .......... 45% 85.5M 6s 2024-03-30T13:11:19.641 INFO:teuthology.orchestra.run.smithi066.stderr: 75950K .......... .......... .......... .......... .......... 45% 85.7M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76000K .......... .......... .......... .......... .......... 45% 81.6M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76050K .......... .......... .......... .......... .......... 45% 85.9M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76100K .......... .......... .......... .......... .......... 45% 76.8M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76150K .......... .......... .......... .......... .......... 45% 91.0M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76200K .......... .......... .......... .......... .......... 45% 45.8M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76250K .......... .......... .......... .......... .......... 45% 82.9M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76300K .......... .......... .......... .......... .......... 45% 27.8M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76350K .......... .......... .......... .......... .......... 45% 25.0M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76400K .......... .......... .......... .......... .......... 45% 42.9M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76450K .......... .......... .......... .......... .......... 45% 24.5M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76500K .......... .......... .......... .......... .......... 45% 38.1M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76550K .......... .......... .......... .......... .......... 45% 23.9M 6s 2024-03-30T13:11:19.642 INFO:teuthology.orchestra.run.smithi066.stderr: 76600K .......... .......... .......... .......... .......... 45% 23.9M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76650K .......... .......... .......... .......... .......... 45% 40.7M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76700K .......... .......... .......... .......... .......... 45% 30.5M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76750K .......... .......... .......... .......... .......... 45% 14.4M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76800K .......... .......... .......... .......... .......... 45% 4.22M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76850K .......... .......... .......... .......... .......... 45% 14.0M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76900K .......... .......... .......... .......... .......... 45% 120M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 76950K .......... .......... .......... .......... .......... 45% 69.2M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77000K .......... .......... .......... .......... .......... 45% 70.4M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77050K .......... .......... .......... .......... .......... 45% 61.0M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77100K .......... .......... .......... .......... .......... 45% 90.0M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77150K .......... .......... .......... .......... .......... 45% 103M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77200K .......... .......... .......... .......... .......... 45% 85.0M 6s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77250K .......... .......... .......... .......... .......... 45% 96.5M 5s 2024-03-30T13:11:19.643 INFO:teuthology.orchestra.run.smithi066.stderr: 77300K .......... .......... .......... .......... .......... 45% 52.4M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77350K .......... .......... .......... .......... .......... 45% 224M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77400K .......... .......... .......... .......... .......... 45% 106M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77450K .......... .......... .......... .......... .......... 45% 64.3M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77500K .......... .......... .......... .......... .......... 46% 34.9M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77550K .......... .......... .......... .......... .......... 46% 20.9M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77600K .......... .......... .......... .......... .......... 46% 31.0M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77650K .......... .......... .......... .......... .......... 46% 39.4M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77700K .......... .......... .......... .......... .......... 46% 25.9M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77750K .......... .......... .......... .......... .......... 46% 28.9M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77800K .......... .......... .......... .......... .......... 46% 25.8M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77850K .......... .......... .......... .......... .......... 46% 24.5M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77900K .......... .......... .......... .......... .......... 46% 62.4M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 77950K .......... .......... .......... .......... .......... 46% 25.5M 5s 2024-03-30T13:11:19.644 INFO:teuthology.orchestra.run.smithi066.stderr: 78000K .......... .......... .......... .......... .......... 46% 30.2M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78050K .......... .......... .......... .......... .......... 46% 24.6M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78100K .......... .......... .......... .......... .......... 46% 23.1M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78150K .......... .......... .......... .......... .......... 46% 55.7M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78200K .......... .......... .......... .......... .......... 46% 17.2M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78250K .......... .......... .......... .......... .......... 46% 5.79M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78300K .......... .......... .......... .......... .......... 46% 108M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78350K .......... .......... .......... .......... .......... 46% 72.2M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78400K .......... .......... .......... .......... .......... 46% 83.3M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78450K .......... .......... .......... .......... .......... 46% 76.4M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78500K .......... .......... .......... .......... .......... 46% 91.8M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78550K .......... .......... .......... .......... .......... 46% 83.7M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78600K .......... .......... .......... .......... .......... 46% 54.0M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78650K .......... .......... .......... .......... .......... 46% 42.4M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78700K .......... .......... .......... .......... .......... 46% 24.5M 5s 2024-03-30T13:11:19.645 INFO:teuthology.orchestra.run.smithi066.stderr: 78750K .......... .......... .......... .......... .......... 46% 24.3M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 78800K .......... .......... .......... .......... .......... 46% 40.4M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 78850K .......... .......... .......... .......... .......... 46% 24.0M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 78900K .......... .......... .......... .......... .......... 46% 40.4M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 78950K .......... .......... .......... .......... .......... 46% 24.4M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79000K .......... .......... .......... .......... .......... 46% 25.0M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79050K .......... .......... .......... .......... .......... 46% 39.1M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79100K .......... .......... .......... .......... .......... 46% 23.5M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79150K .......... .......... .......... .......... .......... 46% 47.7M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79200K .......... .......... .......... .......... .......... 47% 24.5M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79250K .......... .......... .......... .......... .......... 47% 32.2M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79300K .......... .......... .......... .......... .......... 47% 13.3M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79350K .......... .......... .......... .......... .......... 47% 3.89M 5s 2024-03-30T13:11:19.646 INFO:teuthology.orchestra.run.smithi066.stderr: 79400K .......... .......... .......... .......... .......... 47% 15.8M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79450K .......... .......... .......... .......... .......... 47% 16.5M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79500K .......... .......... .......... .......... .......... 47% 89.6M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79550K .......... .......... .......... .......... .......... 47% 90.2M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79600K .......... .......... .......... .......... .......... 47% 74.8M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79650K .......... .......... .......... .......... .......... 47% 104M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79700K .......... .......... .......... .......... .......... 47% 91.5M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79750K .......... .......... .......... .......... .......... 47% 67.3M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79800K .......... .......... .......... .......... .......... 47% 89.5M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79850K .......... .......... .......... .......... .......... 47% 102M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79900K .......... .......... .......... .......... .......... 47% 108M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 79950K .......... .......... .......... .......... .......... 47% 85.8M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 80000K .......... .......... .......... .......... .......... 47% 76.4M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 80050K .......... .......... .......... .......... .......... 47% 103M 5s 2024-03-30T13:11:19.647 INFO:teuthology.orchestra.run.smithi066.stderr: 80100K .......... .......... .......... .......... .......... 47% 75.8M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80150K .......... .......... .......... .......... .......... 47% 88.5M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80200K .......... .......... .......... .......... .......... 47% 25.5M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80250K .......... .......... .......... .......... .......... 47% 28.7M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80300K .......... .......... .......... .......... .......... 47% 30.2M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80350K .......... .......... .......... .......... .......... 47% 26.9M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80400K .......... .......... .......... .......... .......... 47% 43.4M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80450K .......... .......... .......... .......... .......... 47% 24.1M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80500K .......... .......... .......... .......... .......... 47% 24.3M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80550K .......... .......... .......... .......... .......... 47% 26.3M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80600K .......... .......... .......... .......... .......... 47% 49.9M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80650K .......... .......... .......... .......... .......... 47% 30.9M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80700K .......... .......... .......... .......... .......... 47% 22.6M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80750K .......... .......... .......... .......... .......... 47% 21.2M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80800K .......... .......... .......... .......... .......... 47% 7.05M 5s 2024-03-30T13:11:19.648 INFO:teuthology.orchestra.run.smithi066.stderr: 80850K .......... .......... .......... .......... .......... 47% 12.1M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 80900K .......... .......... .......... .......... .......... 48% 78.0M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 80950K .......... .......... .......... .......... .......... 48% 85.9M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81000K .......... .......... .......... .......... .......... 48% 96.1M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81050K .......... .......... .......... .......... .......... 48% 84.6M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81100K .......... .......... .......... .......... .......... 48% 98.4M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81150K .......... .......... .......... .......... .......... 48% 66.2M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81200K .......... .......... .......... .......... .......... 48% 100M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81250K .......... .......... .......... .......... .......... 48% 40.1M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81300K .......... .......... .......... .......... .......... 48% 36.4M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81350K .......... .......... .......... .......... .......... 48% 24.5M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81400K .......... .......... .......... .......... .......... 48% 23.9M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81450K .......... .......... .......... .......... .......... 48% 41.8M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81500K .......... .......... .......... .......... .......... 48% 24.8M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81550K .......... .......... .......... .......... .......... 48% 37.5M 5s 2024-03-30T13:11:19.649 INFO:teuthology.orchestra.run.smithi066.stderr: 81600K .......... .......... .......... .......... .......... 48% 23.7M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81650K .......... .......... .......... .......... .......... 48% 23.1M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81700K .......... .......... .......... .......... .......... 48% 49.9M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81750K .......... .......... .......... .......... .......... 48% 24.5M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81800K .......... .......... .......... .......... .......... 48% 34.6M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81850K .......... .......... .......... .......... .......... 48% 8.97M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81900K .......... .......... .......... .......... .......... 48% 9.46M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 81950K .......... .......... .......... .......... .......... 48% 7.74M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 82000K .......... .......... .......... .......... .......... 48% 83.9M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 82050K .......... .......... .......... .......... .......... 48% 81.7M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 82100K .......... .......... .......... .......... .......... 48% 84.4M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 82150K .......... .......... .......... .......... .......... 48% 97.9M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 82200K .......... .......... .......... .......... .......... 48% 95.9M 5s 2024-03-30T13:11:19.650 INFO:teuthology.orchestra.run.smithi066.stderr: 82250K .......... .......... .......... .......... .......... 48% 85.0M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82300K .......... .......... .......... .......... .......... 48% 96.1M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82350K .......... .......... .......... .......... .......... 48% 74.9M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82400K .......... .......... .......... .......... .......... 48% 95.2M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82450K .......... .......... .......... .......... .......... 48% 108M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82500K .......... .......... .......... .......... .......... 48% 33.2M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82550K .......... .......... .......... .......... .......... 49% 40.1M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82600K .......... .......... .......... .......... .......... 49% 24.7M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82650K .......... .......... .......... .......... .......... 49% 23.9M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82700K .......... .......... .......... .......... .......... 49% 42.4M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82750K .......... .......... .......... .......... .......... 49% 25.0M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82800K .......... .......... .......... .......... .......... 49% 28.7M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82850K .......... .......... .......... .......... .......... 49% 23.9M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82900K .......... .......... .......... .......... .......... 49% 25.2M 5s 2024-03-30T13:11:19.651 INFO:teuthology.orchestra.run.smithi066.stderr: 82950K .......... .......... .......... .......... .......... 49% 73.4M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83000K .......... .......... .......... .......... .......... 49% 23.4M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83050K .......... .......... .......... .......... .......... 49% 32.8M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83100K .......... .......... .......... .......... .......... 49% 21.8M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83150K .......... .......... .......... .......... .......... 49% 36.5M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83200K .......... .......... .......... .......... .......... 49% 36.4M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83250K .......... .......... .......... .......... .......... 49% 25.6M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83300K .......... .......... .......... .......... .......... 49% 20.1M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83350K .......... .......... .......... .......... .......... 49% 12.6M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83400K .......... .......... .......... .......... .......... 49% 9.16M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83450K .......... .......... .......... .......... .......... 49% 75.0M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83500K .......... .......... .......... .......... .......... 49% 73.1M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83550K .......... .......... .......... .......... .......... 49% 81.0M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83600K .......... .......... .......... .......... .......... 49% 78.9M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83650K .......... .......... .......... .......... .......... 49% 65.3M 5s 2024-03-30T13:11:19.652 INFO:teuthology.orchestra.run.smithi066.stderr: 83700K .......... .......... .......... .......... .......... 49% 87.8M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 83750K .......... .......... .......... .......... .......... 49% 31.4M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 83800K .......... .......... .......... .......... .......... 49% 24.5M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 83850K .......... .......... .......... .......... .......... 49% 38.7M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 83900K .......... .......... .......... .......... .......... 49% 24.7M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 83950K .......... .......... .......... .......... .......... 49% 39.4M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84000K .......... .......... .......... .......... .......... 49% 24.8M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84050K .......... .......... .......... .......... .......... 49% 24.2M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84100K .......... .......... .......... .......... .......... 49% 33.7M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84150K .......... .......... .......... .......... .......... 49% 25.4M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84200K .......... .......... .......... .......... .......... 49% 59.4M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84250K .......... .......... .......... .......... .......... 50% 22.3M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84300K .......... .......... .......... .......... .......... 50% 26.7M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84350K .......... .......... .......... .......... .......... 50% 31.7M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84400K .......... .......... .......... .......... .......... 50% 8.04M 5s 2024-03-30T13:11:19.653 INFO:teuthology.orchestra.run.smithi066.stderr: 84450K .......... .......... .......... .......... .......... 50% 7.31M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84500K .......... .......... .......... .......... .......... 50% 7.49M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84550K .......... .......... .......... .......... .......... 50% 61.0M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84600K .......... .......... .......... .......... .......... 50% 98.0M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84650K .......... .......... .......... .......... .......... 50% 91.0M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84700K .......... .......... .......... .......... .......... 50% 96.3M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84750K .......... .......... .......... .......... .......... 50% 63.7M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84800K .......... .......... .......... .......... .......... 50% 71.2M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84850K .......... .......... .......... .......... .......... 50% 92.4M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84900K .......... .......... .......... .......... .......... 50% 92.9M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 84950K .......... .......... .......... .......... .......... 50% 93.1M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 85000K .......... .......... .......... .......... .......... 50% 60.1M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 85050K .......... .......... .......... .......... .......... 50% 75.0M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 85100K .......... .......... .......... .......... .......... 50% 84.9M 5s 2024-03-30T13:11:19.654 INFO:teuthology.orchestra.run.smithi066.stderr: 85150K .......... .......... .......... .......... .......... 50% 98.1M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85200K .......... .......... .......... .......... .......... 50% 49.8M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85250K .......... .......... .......... .......... .......... 50% 24.8M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85300K .......... .......... .......... .......... .......... 50% 24.2M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85350K .......... .......... .......... .......... .......... 50% 31.7M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85400K .......... .......... .......... .......... .......... 50% 24.6M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85450K .......... .......... .......... .......... .......... 50% 63.1M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85500K .......... .......... .......... .......... .......... 50% 25.3M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85550K .......... .......... .......... .......... .......... 50% 29.8M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85600K .......... .......... .......... .......... .......... 50% 29.0M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85650K .......... .......... .......... .......... .......... 50% 26.0M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85700K .......... .......... .......... .......... .......... 50% 35.4M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85750K .......... .......... .......... .......... .......... 50% 24.8M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85800K .......... .......... .......... .......... .......... 50% 23.9M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85850K .......... .......... .......... .......... .......... 50% 18.7M 5s 2024-03-30T13:11:19.655 INFO:teuthology.orchestra.run.smithi066.stderr: 85900K .......... .......... .......... .......... .......... 50% 4.25M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 85950K .......... .......... .......... .......... .......... 51% 13.5M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86000K .......... .......... .......... .......... .......... 51% 85.2M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86050K .......... .......... .......... .......... .......... 51% 76.1M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86100K .......... .......... .......... .......... .......... 51% 98.5M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86150K .......... .......... .......... .......... .......... 51% 81.7M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86200K .......... .......... .......... .......... .......... 51% 78.7M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86250K .......... .......... .......... .......... .......... 51% 97.5M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86300K .......... .......... .......... .......... .......... 51% 73.9M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86350K .......... .......... .......... .......... .......... 51% 84.7M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86400K .......... .......... .......... .......... .......... 51% 88.1M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86450K .......... .......... .......... .......... .......... 51% 68.7M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86500K .......... .......... .......... .......... .......... 51% 58.4M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86550K .......... .......... .......... .......... .......... 51% 69.7M 5s 2024-03-30T13:11:19.656 INFO:teuthology.orchestra.run.smithi066.stderr: 86600K .......... .......... .......... .......... .......... 51% 46.2M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86650K .......... .......... .......... .......... .......... 51% 24.2M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86700K .......... .......... .......... .......... .......... 51% 24.3M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86750K .......... .......... .......... .......... .......... 51% 43.1M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86800K .......... .......... .......... .......... .......... 51% 25.1M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86850K .......... .......... .......... .......... .......... 51% 32.4M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86900K .......... .......... .......... .......... .......... 51% 22.6M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 86950K .......... .......... .......... .......... .......... 51% 7.07M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87000K .......... .......... .......... .......... .......... 51% 7.12M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87050K .......... .......... .......... .......... .......... 51% 9.46M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87100K .......... .......... .......... .......... .......... 51% 83.6M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87150K .......... .......... .......... .......... .......... 51% 87.9M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87200K .......... .......... .......... .......... .......... 51% 89.2M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87250K .......... .......... .......... .......... .......... 51% 103M 5s 2024-03-30T13:11:19.657 INFO:teuthology.orchestra.run.smithi066.stderr: 87300K .......... .......... .......... .......... .......... 51% 74.5M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87350K .......... .......... .......... .......... .......... 51% 89.3M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87400K .......... .......... .......... .......... .......... 51% 72.7M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87450K .......... .......... .......... .......... .......... 51% 78.9M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87500K .......... .......... .......... .......... .......... 51% 83.3M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87550K .......... .......... .......... .......... .......... 51% 91.7M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87600K .......... .......... .......... .......... .......... 51% 104M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87650K .......... .......... .......... .......... .......... 52% 98.5M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87700K .......... .......... .......... .......... .......... 52% 48.1M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87750K .......... .......... .......... .......... .......... 52% 43.2M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87800K .......... .......... .......... .......... .......... 52% 23.0M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87850K .......... .......... .......... .......... .......... 52% 36.1M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87900K .......... .......... .......... .......... .......... 52% 22.5M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 87950K .......... .......... .......... .......... .......... 52% 24.8M 5s 2024-03-30T13:11:19.658 INFO:teuthology.orchestra.run.smithi066.stderr: 88000K .......... .......... .......... .......... .......... 52% 54.3M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88050K .......... .......... .......... .......... .......... 52% 24.6M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88100K .......... .......... .......... .......... .......... 52% 31.5M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88150K .......... .......... .......... .......... .......... 52% 24.5M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88200K .......... .......... .......... .......... .......... 52% 25.1M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88250K .......... .......... .......... .......... .......... 52% 55.7M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88300K .......... .......... .......... .......... .......... 52% 24.1M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88350K .......... .......... .......... .......... .......... 52% 30.0M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88400K .......... .......... .......... .......... .......... 52% 13.0M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88450K .......... .......... .......... .......... .......... 52% 4.58M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88500K .......... .......... .......... .......... .......... 52% 80.3M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88550K .......... .......... .......... .......... .......... 52% 75.5M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88600K .......... .......... .......... .......... .......... 52% 104M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88650K .......... .......... .......... .......... .......... 52% 65.5M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88700K .......... .......... .......... .......... .......... 52% 76.8M 5s 2024-03-30T13:11:19.659 INFO:teuthology.orchestra.run.smithi066.stderr: 88750K .......... .......... .......... .......... .......... 52% 69.8M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 88800K .......... .......... .......... .......... .......... 52% 66.6M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 88850K .......... .......... .......... .......... .......... 52% 59.0M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 88900K .......... .......... .......... .......... .......... 52% 94.4M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 88950K .......... .......... .......... .......... .......... 52% 80.7M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89000K .......... .......... .......... .......... .......... 52% 75.7M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89050K .......... .......... .......... .......... .......... 52% 25.1M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89100K .......... .......... .......... .......... .......... 52% 23.3M 5s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89150K .......... .......... .......... .......... .......... 52% 41.4M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89200K .......... .......... .......... .......... .......... 52% 24.0M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89250K .......... .......... .......... .......... .......... 52% 42.1M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89300K .......... .......... .......... .......... .......... 53% 24.9M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89350K .......... .......... .......... .......... .......... 53% 30.5M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89400K .......... .......... .......... .......... .......... 53% 24.2M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89450K .......... .......... .......... .......... .......... 53% 25.8M 4s 2024-03-30T13:11:19.660 INFO:teuthology.orchestra.run.smithi066.stderr: 89500K .......... .......... .......... .......... .......... 53% 22.2M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89550K .......... .......... .......... .......... .......... 53% 4.51M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89600K .......... .......... .......... .......... .......... 53% 15.3M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89650K .......... .......... .......... .......... .......... 53% 81.0M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89700K .......... .......... .......... .......... .......... 53% 69.6M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89750K .......... .......... .......... .......... .......... 53% 89.5M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89800K .......... .......... .......... .......... .......... 53% 89.8M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89850K .......... .......... .......... .......... .......... 53% 94.4M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89900K .......... .......... .......... .......... .......... 53% 74.2M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 89950K .......... .......... .......... .......... .......... 53% 59.9M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 90000K .......... .......... .......... .......... .......... 53% 79.7M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 90050K .......... .......... .......... .......... .......... 53% 82.4M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 90100K .......... .......... .......... .......... .......... 53% 93.8M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 90150K .......... .......... .......... .......... .......... 53% 57.6M 4s 2024-03-30T13:11:19.661 INFO:teuthology.orchestra.run.smithi066.stderr: 90200K .......... .......... .......... .......... .......... 53% 30.2M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90250K .......... .......... .......... .......... .......... 53% 42.4M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90300K .......... .......... .......... .......... .......... 53% 24.8M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90350K .......... .......... .......... .......... .......... 53% 24.0M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90400K .......... .......... .......... .......... .......... 53% 36.7M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90450K .......... .......... .......... .......... .......... 53% 21.9M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90500K .......... .......... .......... .......... .......... 53% 62.9M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90550K .......... .......... .......... .......... .......... 53% 24.7M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90600K .......... .......... .......... .......... .......... 53% 29.2M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90650K .......... .......... .......... .......... .......... 53% 24.0M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90700K .......... .......... .......... .......... .......... 53% 25.5M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90750K .......... .......... .......... .......... .......... 53% 74.6M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90800K .......... .......... .......... .......... .......... 53% 24.2M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90850K .......... .......... .......... .......... .......... 53% 23.7M 4s 2024-03-30T13:11:19.662 INFO:teuthology.orchestra.run.smithi066.stderr: 90900K .......... .......... .......... .......... .......... 53% 29.4M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 90950K .......... .......... .......... .......... .......... 53% 10.6M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91000K .......... .......... .......... .......... .......... 54% 7.00M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91050K .......... .......... .......... .......... .......... 54% 12.1M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91100K .......... .......... .......... .......... .......... 54% 77.0M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91150K .......... .......... .......... .......... .......... 54% 100M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91200K .......... .......... .......... .......... .......... 54% 54.4M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91250K .......... .......... .......... .......... .......... 54% 89.8M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91300K .......... .......... .......... .......... .......... 54% 58.3M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91350K .......... .......... .......... .......... .......... 54% 63.8M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91400K .......... .......... .......... .......... .......... 54% 90.7M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91450K .......... .......... .......... .......... .......... 54% 84.7M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91500K .......... .......... .......... .......... .......... 54% 75.0M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91550K .......... .......... .......... .......... .......... 54% 80.6M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91600K .......... .......... .......... .......... .......... 54% 28.5M 4s 2024-03-30T13:11:19.663 INFO:teuthology.orchestra.run.smithi066.stderr: 91650K .......... .......... .......... .......... .......... 54% 39.6M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91700K .......... .......... .......... .......... .......... 54% 447K 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91750K .......... .......... .......... .......... .......... 54% 476M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91800K .......... .......... .......... .......... .......... 54% 422M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91850K .......... .......... .......... .......... .......... 54% 452M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91900K .......... .......... .......... .......... .......... 54% 559M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 91950K .......... .......... .......... .......... .......... 54% 527M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92000K .......... .......... .......... .......... .......... 54% 458M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92050K .......... .......... .......... .......... .......... 54% 451M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92100K .......... .......... .......... .......... .......... 54% 472M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92150K .......... .......... .......... .......... .......... 54% 363M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92200K .......... .......... .......... .......... .......... 54% 390M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92250K .......... .......... .......... .......... .......... 54% 444M 4s 2024-03-30T13:11:19.664 INFO:teuthology.orchestra.run.smithi066.stderr: 92300K .......... .......... .......... .......... .......... 54% 573M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92350K .......... .......... .......... .......... .......... 54% 592M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92400K .......... .......... .......... .......... .......... 54% 503M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92450K .......... .......... .......... .......... .......... 54% 567M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92500K .......... .......... .......... .......... .......... 54% 595M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92550K .......... .......... .......... .......... .......... 54% 581M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92600K .......... .......... .......... .......... .......... 54% 522M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92650K .......... .......... .......... .......... .......... 54% 584M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92700K .......... .......... .......... .......... .......... 55% 532M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92750K .......... .......... .......... .......... .......... 55% 585M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92800K .......... .......... .......... .......... .......... 55% 526M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92850K .......... .......... .......... .......... .......... 55% 591M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92900K .......... .......... .......... .......... .......... 55% 579M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 92950K .......... .......... .......... .......... .......... 55% 584M 4s 2024-03-30T13:11:19.665 INFO:teuthology.orchestra.run.smithi066.stderr: 93000K .......... .......... .......... .......... .......... 55% 518M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93050K .......... .......... .......... .......... .......... 55% 607M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93100K .......... .......... .......... .......... .......... 55% 607M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93150K .......... .......... .......... .......... .......... 55% 581M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93200K .......... .......... .......... .......... .......... 55% 506M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93250K .......... .......... .......... .......... .......... 55% 569M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93300K .......... .......... .......... .......... .......... 55% 594M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93350K .......... .......... .......... .......... .......... 55% 592M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93400K .......... .......... .......... .......... .......... 55% 512M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93450K .......... .......... .......... .......... .......... 55% 570M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93500K .......... .......... .......... .......... .......... 55% 523M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93550K .......... .......... .......... .......... .......... 55% 589M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93600K .......... .......... .......... .......... .......... 55% 522M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93650K .......... .......... .......... .......... .......... 55% 588M 4s 2024-03-30T13:11:19.666 INFO:teuthology.orchestra.run.smithi066.stderr: 93700K .......... .......... .......... .......... .......... 55% 533M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 93750K .......... .......... .......... .......... .......... 55% 584M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 93800K .......... .......... .......... .......... .......... 55% 516M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 93850K .......... .......... .......... .......... .......... 55% 486M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 93900K .......... .......... .......... .......... .......... 55% 503M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 93950K .......... .......... .......... .......... .......... 55% 470M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94000K .......... .......... .......... .......... .......... 55% 474M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94050K .......... .......... .......... .......... .......... 55% 550M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94100K .......... .......... .......... .......... .......... 55% 604M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94150K .......... .......... .......... .......... .......... 55% 622M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94200K .......... .......... .......... .......... .......... 55% 158M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94250K .......... .......... .......... .......... .......... 55% 106M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94300K .......... .......... .......... .......... .......... 55% 111M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94350K .......... .......... .......... .......... .......... 56% 105M 4s 2024-03-30T13:11:19.667 INFO:teuthology.orchestra.run.smithi066.stderr: 94400K .......... .......... .......... .......... .......... 56% 69.5M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94450K .......... .......... .......... .......... .......... 56% 82.3M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94500K .......... .......... .......... .......... .......... 56% 50.8M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94550K .......... .......... .......... .......... .......... 56% 78.0M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94600K .......... .......... .......... .......... .......... 56% 88.6M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94650K .......... .......... .......... .......... .......... 56% 79.0M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94700K .......... .......... .......... .......... .......... 56% 104M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94750K .......... .......... .......... .......... .......... 56% 50.0M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94800K .......... .......... .......... .......... .......... 56% 40.2M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94850K .......... .......... .......... .......... .......... 56% 25.0M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94900K .......... .......... .......... .......... .......... 56% 24.5M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 94950K .......... .......... .......... .......... .......... 56% 40.1M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 95000K .......... .......... .......... .......... .......... 56% 25.1M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 95050K .......... .......... .......... .......... .......... 56% 37.8M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 95100K .......... .......... .......... .......... .......... 56% 25.3M 4s 2024-03-30T13:11:19.668 INFO:teuthology.orchestra.run.smithi066.stderr: 95150K .......... .......... .......... .......... .......... 56% 3.71M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95200K .......... .......... .......... .......... .......... 56% 64.1M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95250K .......... .......... .......... .......... .......... 56% 109M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95300K .......... .......... .......... .......... .......... 56% 49.5M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95350K .......... .......... .......... .......... .......... 56% 49.7M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95400K .......... .......... .......... .......... .......... 56% 30.1M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95450K .......... .......... .......... .......... .......... 56% 8.14M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95500K .......... .......... .......... .......... .......... 56% 8.29M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95550K .......... .......... .......... .......... .......... 56% 83.3M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95600K .......... .......... .......... .......... .......... 56% 70.8M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95650K .......... .......... .......... .......... .......... 56% 69.9M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95700K .......... .......... .......... .......... .......... 56% 89.4M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95750K .......... .......... .......... .......... .......... 56% 73.9M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95800K .......... .......... .......... .......... .......... 56% 89.8M 4s 2024-03-30T13:11:19.669 INFO:teuthology.orchestra.run.smithi066.stderr: 95850K .......... .......... .......... .......... .......... 56% 76.5M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 95900K .......... .......... .......... .......... .......... 56% 68.8M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 95950K .......... .......... .......... .......... .......... 56% 73.0M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96000K .......... .......... .......... .......... .......... 56% 73.9M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96050K .......... .......... .......... .......... .......... 57% 96.9M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96100K .......... .......... .......... .......... .......... 57% 68.7M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96150K .......... .......... .......... .......... .......... 57% 81.0M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96200K .......... .......... .......... .......... .......... 57% 35.9M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96250K .......... .......... .......... .......... .......... 57% 25.3M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96300K .......... .......... .......... .......... .......... 57% 70.8M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96350K .......... .......... .......... .......... .......... 57% 24.4M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96400K .......... .......... .......... .......... .......... 57% 27.8M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96450K .......... .......... .......... .......... .......... 57% 23.8M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96500K .......... .......... .......... .......... .......... 57% 28.4M 4s 2024-03-30T13:11:19.670 INFO:teuthology.orchestra.run.smithi066.stderr: 96550K .......... .......... .......... .......... .......... 57% 13.3M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96600K .......... .......... .......... .......... .......... 57% 7.47M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96650K .......... .......... .......... .......... .......... 57% 10.6M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96700K .......... .......... .......... .......... .......... 57% 64.2M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96750K .......... .......... .......... .......... .......... 57% 71.0M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96800K .......... .......... .......... .......... .......... 57% 51.8M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96850K .......... .......... .......... .......... .......... 57% 86.7M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96900K .......... .......... .......... .......... .......... 57% 64.8M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 96950K .......... .......... .......... .......... .......... 57% 52.8M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 97000K .......... .......... .......... .......... .......... 57% 69.5M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 97050K .......... .......... .......... .......... .......... 57% 79.9M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 97100K .......... .......... .......... .......... .......... 57% 108M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 97150K .......... .......... .......... .......... .......... 57% 89.4M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 97200K .......... .......... .......... .......... .......... 57% 29.2M 4s 2024-03-30T13:11:19.671 INFO:teuthology.orchestra.run.smithi066.stderr: 97250K .......... .......... .......... .......... .......... 57% 26.3M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97300K .......... .......... .......... .......... .......... 57% 24.6M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97350K .......... .......... .......... .......... .......... 57% 49.3M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97400K .......... .......... .......... .......... .......... 57% 25.5M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97450K .......... .......... .......... .......... .......... 57% 29.7M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97500K .......... .......... .......... .......... .......... 57% 26.6M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97550K .......... .......... .......... .......... .......... 57% 24.5M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97600K .......... .......... .......... .......... .......... 57% 52.8M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97650K .......... .......... .......... .......... .......... 57% 24.5M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97700K .......... .......... .......... .......... .......... 57% 33.3M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97750K .......... .......... .......... .......... .......... 58% 24.0M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97800K .......... .......... .......... .......... .......... 58% 24.4M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97850K .......... .......... .......... .......... .......... 58% 42.1M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97900K .......... .......... .......... .......... .......... 58% 9.99M 4s 2024-03-30T13:11:19.672 INFO:teuthology.orchestra.run.smithi066.stderr: 97950K .......... .......... .......... .......... .......... 58% 6.92M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98000K .......... .......... .......... .......... .......... 58% 65.8M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98050K .......... .......... .......... .......... .......... 58% 64.0M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98100K .......... .......... .......... .......... .......... 58% 66.9M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98150K .......... .......... .......... .......... .......... 58% 77.5M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98200K .......... .......... .......... .......... .......... 58% 80.8M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98250K .......... .......... .......... .......... .......... 58% 67.2M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98300K .......... .......... .......... .......... .......... 58% 75.4M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98350K .......... .......... .......... .......... .......... 58% 77.0M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98400K .......... .......... .......... .......... .......... 58% 73.9M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98450K .......... .......... .......... .......... .......... 58% 20.8M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98500K .......... .......... .......... .......... .......... 58% 38.6M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98550K .......... .......... .......... .......... .......... 58% 23.7M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98600K .......... .......... .......... .......... .......... 58% 43.1M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98650K .......... .......... .......... .......... .......... 58% 23.6M 4s 2024-03-30T13:11:19.673 INFO:teuthology.orchestra.run.smithi066.stderr: 98700K .......... .......... .......... .......... .......... 58% 35.2M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 98750K .......... .......... .......... .......... .......... 58% 25.1M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 98800K .......... .......... .......... .......... .......... 58% 24.3M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 98850K .......... .......... .......... .......... .......... 58% 50.6M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 98900K .......... .......... .......... .......... .......... 58% 24.1M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 98950K .......... .......... .......... .......... .......... 58% 24.2M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99000K .......... .......... .......... .......... .......... 58% 23.9M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99050K .......... .......... .......... .......... .......... 58% 37.7M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99100K .......... .......... .......... .......... .......... 58% 8.53M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99150K .......... .......... .......... .......... .......... 58% 3.89M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99200K .......... .......... .......... .......... .......... 58% 19.4M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99250K .......... .......... .......... .......... .......... 58% 29.0M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99300K .......... .......... .......... .......... .......... 58% 79.3M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99350K .......... .......... .......... .......... .......... 58% 42.6M 4s 2024-03-30T13:11:19.674 INFO:teuthology.orchestra.run.smithi066.stderr: 99400K .......... .......... .......... .......... .......... 58% 60.9M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99450K .......... .......... .......... .......... .......... 59% 46.4M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99500K .......... .......... .......... .......... .......... 59% 63.9M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99550K .......... .......... .......... .......... .......... 59% 93.5M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99600K .......... .......... .......... .......... .......... 59% 82.5M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99650K .......... .......... .......... .......... .......... 59% 77.4M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99700K .......... .......... .......... .......... .......... 59% 87.5M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99750K .......... .......... .......... .......... .......... 59% 76.5M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99800K .......... .......... .......... .......... .......... 59% 71.2M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99850K .......... .......... .......... .......... .......... 59% 90.5M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99900K .......... .......... .......... .......... .......... 59% 70.9M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr: 99950K .......... .......... .......... .......... .......... 59% 68.3M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:100000K .......... .......... .......... .......... .......... 59% 77.2M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:100050K .......... .......... .......... .......... .......... 59% 76.8M 4s 2024-03-30T13:11:19.675 INFO:teuthology.orchestra.run.smithi066.stderr:100100K .......... .......... .......... .......... .......... 59% 50.5M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100150K .......... .......... .......... .......... .......... 59% 25.1M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100200K .......... .......... .......... .......... .......... 59% 23.4M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100250K .......... .......... .......... .......... .......... 59% 29.8M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100300K .......... .......... .......... .......... .......... 59% 41.3M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100350K .......... .......... .......... .......... .......... 59% 31.0M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100400K .......... .......... .......... .......... .......... 59% 26.6M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100450K .......... .......... .......... .......... .......... 59% 4.55M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100500K .......... .......... .......... .......... .......... 59% 12.0M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100550K .......... .......... .......... .......... .......... 59% 62.4M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100600K .......... .......... .......... .......... .......... 59% 66.2M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100650K .......... .......... .......... .......... .......... 59% 71.1M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100700K .......... .......... .......... .......... .......... 59% 72.8M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100750K .......... .......... .......... .......... .......... 59% 55.8M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100800K .......... .......... .......... .......... .......... 59% 62.7M 4s 2024-03-30T13:11:19.676 INFO:teuthology.orchestra.run.smithi066.stderr:100850K .......... .......... .......... .......... .......... 59% 38.0M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:100900K .......... .......... .......... .......... .......... 59% 70.4M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:100950K .......... .......... .......... .......... .......... 59% 89.3M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101000K .......... .......... .......... .......... .......... 59% 77.1M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101050K .......... .......... .......... .......... .......... 59% 45.9M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101100K .......... .......... .......... .......... .......... 60% 58.3M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101150K .......... .......... .......... .......... .......... 60% 36.7M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101200K .......... .......... .......... .......... .......... 60% 26.3M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101250K .......... .......... .......... .......... .......... 60% 38.3M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101300K .......... .......... .......... .......... .......... 60% 24.3M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101350K .......... .......... .......... .......... .......... 60% 42.1M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101400K .......... .......... .......... .......... .......... 60% 22.9M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101450K .......... .......... .......... .......... .......... 60% 25.3M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101500K .......... .......... .......... .......... .......... 60% 36.6M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101550K .......... .......... .......... .......... .......... 60% 15.2M 4s 2024-03-30T13:11:19.677 INFO:teuthology.orchestra.run.smithi066.stderr:101600K .......... .......... .......... .......... .......... 60% 9.13M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101650K .......... .......... .......... .......... .......... 60% 29.9M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101700K .......... .......... .......... .......... .......... 60% 75.5M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101750K .......... .......... .......... .......... .......... 60% 75.7M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101800K .......... .......... .......... .......... .......... 60% 42.1M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101850K .......... .......... .......... .......... .......... 60% 63.4M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101900K .......... .......... .......... .......... .......... 60% 75.3M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:101950K .......... .......... .......... .......... .......... 60% 49.2M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102000K .......... .......... .......... .......... .......... 60% 27.7M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102050K .......... .......... .......... .......... .......... 60% 25.7M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102100K .......... .......... .......... .......... .......... 60% 24.7M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102150K .......... .......... .......... .......... .......... 60% 51.0M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102200K .......... .......... .......... .......... .......... 60% 27.9M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102250K .......... .......... .......... .......... .......... 60% 31.1M 4s 2024-03-30T13:11:19.678 INFO:teuthology.orchestra.run.smithi066.stderr:102300K .......... .......... .......... .......... .......... 60% 24.1M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102350K .......... .......... .......... .......... .......... 60% 24.4M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102400K .......... .......... .......... .......... .......... 60% 41.8M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102450K .......... .......... .......... .......... .......... 60% 24.7M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102500K .......... .......... .......... .......... .......... 60% 37.3M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102550K .......... .......... .......... .......... .......... 60% 25.1M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102600K .......... .......... .......... .......... .......... 60% 22.6M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102650K .......... .......... .......... .......... .......... 60% 47.3M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102700K .......... .......... .......... .......... .......... 60% 29.5M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102750K .......... .......... .......... .......... .......... 60% 22.3M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102800K .......... .......... .......... .......... .......... 61% 27.2M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102850K .......... .......... .......... .......... .......... 61% 24.5M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102900K .......... .......... .......... .......... .......... 61% 31.1M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:102950K .......... .......... .......... .......... .......... 61% 5.66M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:103000K .......... .......... .......... .......... .......... 61% 9.56M 4s 2024-03-30T13:11:19.679 INFO:teuthology.orchestra.run.smithi066.stderr:103050K .......... .......... .......... .......... .......... 61% 53.1M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103100K .......... .......... .......... .......... .......... 61% 57.7M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103150K .......... .......... .......... .......... .......... 61% 74.0M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103200K .......... .......... .......... .......... .......... 61% 46.7M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103250K .......... .......... .......... .......... .......... 61% 63.7M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103300K .......... .......... .......... .......... .......... 61% 69.7M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103350K .......... .......... .......... .......... .......... 61% 70.6M 4s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103400K .......... .......... .......... .......... .......... 61% 69.0M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103450K .......... .......... .......... .......... .......... 61% 72.7M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103500K .......... .......... .......... .......... .......... 61% 86.6M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103550K .......... .......... .......... .......... .......... 61% 71.3M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103600K .......... .......... .......... .......... .......... 61% 36.0M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103650K .......... .......... .......... .......... .......... 61% 45.9M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103700K .......... .......... .......... .......... .......... 61% 25.1M 3s 2024-03-30T13:11:19.680 INFO:teuthology.orchestra.run.smithi066.stderr:103750K .......... .......... .......... .......... .......... 61% 39.2M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:103800K .......... .......... .......... .......... .......... 61% 23.6M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:103850K .......... .......... .......... .......... .......... 61% 24.3M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:103900K .......... .......... .......... .......... .......... 61% 30.8M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:103950K .......... .......... .......... .......... .......... 61% 31.3M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104000K .......... .......... .......... .......... .......... 61% 15.7M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104050K .......... .......... .......... .......... .......... 61% 4.77M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104100K .......... .......... .......... .......... .......... 61% 17.1M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104150K .......... .......... .......... .......... .......... 61% 79.2M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104200K .......... .......... .......... .......... .......... 61% 56.8M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104250K .......... .......... .......... .......... .......... 61% 57.5M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104300K .......... .......... .......... .......... .......... 61% 56.4M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104350K .......... .......... .......... .......... .......... 61% 37.0M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104400K .......... .......... .......... .......... .......... 61% 63.1M 3s 2024-03-30T13:11:19.681 INFO:teuthology.orchestra.run.smithi066.stderr:104450K .......... .......... .......... .......... .......... 61% 56.1M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104500K .......... .......... .......... .......... .......... 62% 60.6M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104550K .......... .......... .......... .......... .......... 62% 59.0M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104600K .......... .......... .......... .......... .......... 62% 71.8M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104650K .......... .......... .......... .......... .......... 62% 81.5M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104700K .......... .......... .......... .......... .......... 62% 74.0M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104750K .......... .......... .......... .......... .......... 62% 75.5M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104800K .......... .......... .......... .......... .......... 62% 26.2M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104850K .......... .......... .......... .......... .......... 62% 24.6M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104900K .......... .......... .......... .......... .......... 62% 41.8M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:104950K .......... .......... .......... .......... .......... 62% 24.3M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:105000K .......... .......... .......... .......... .......... 62% 23.5M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:105050K .......... .......... .......... .......... .......... 62% 41.4M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:105100K .......... .......... .......... .......... .......... 62% 24.1M 3s 2024-03-30T13:11:19.682 INFO:teuthology.orchestra.run.smithi066.stderr:105150K .......... .......... .......... .......... .......... 62% 42.0M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105200K .......... .......... .......... .......... .......... 62% 24.2M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105250K .......... .......... .......... .......... .......... 62% 14.7M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105300K .......... .......... .......... .......... .......... 62% 7.97M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105350K .......... .......... .......... .......... .......... 62% 65.3M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105400K .......... .......... .......... .......... .......... 62% 89.1M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105450K .......... .......... .......... .......... .......... 62% 8.07M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105500K .......... .......... .......... .......... .......... 62% 8.86M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105550K .......... .......... .......... .......... .......... 62% 70.5M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105600K .......... .......... .......... .......... .......... 62% 60.6M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105650K .......... .......... .......... .......... .......... 62% 82.6M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105700K .......... .......... .......... .......... .......... 62% 63.1M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105750K .......... .......... .......... .......... .......... 62% 84.4M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105800K .......... .......... .......... .......... .......... 62% 52.6M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105850K .......... .......... .......... .......... .......... 62% 43.4M 3s 2024-03-30T13:11:19.683 INFO:teuthology.orchestra.run.smithi066.stderr:105900K .......... .......... .......... .......... .......... 62% 88.4M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:105950K .......... .......... .......... .......... .......... 62% 73.9M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106000K .......... .......... .......... .......... .......... 62% 61.6M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106050K .......... .......... .......... .......... .......... 62% 70.6M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106100K .......... .......... .......... .......... .......... 62% 38.1M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106150K .......... .......... .......... .......... .......... 63% 76.0M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106200K .......... .......... .......... .......... .......... 63% 46.4M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106250K .......... .......... .......... .......... .......... 63% 24.1M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106300K .......... .......... .......... .......... .......... 63% 40.0M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106350K .......... .......... .......... .......... .......... 63% 22.8M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106400K .......... .......... .......... .......... .......... 63% 46.4M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106450K .......... .......... .......... .......... .......... 63% 24.8M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106500K .......... .......... .......... .......... .......... 63% 32.1M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106550K .......... .......... .......... .......... .......... 63% 12.9M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106600K .......... .......... .......... .......... .......... 63% 3.63M 3s 2024-03-30T13:11:19.684 INFO:teuthology.orchestra.run.smithi066.stderr:106650K .......... .......... .......... .......... .......... 63% 10.3M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:106700K .......... .......... .......... .......... .......... 63% 30.1M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:106750K .......... .......... .......... .......... .......... 63% 48.0M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:106800K .......... .......... .......... .......... .......... 63% 47.0M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:106850K .......... .......... .......... .......... .......... 63% 37.8M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:106900K .......... .......... .......... .......... .......... 63% 74.2M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:106950K .......... .......... .......... .......... .......... 63% 49.5M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107000K .......... .......... .......... .......... .......... 63% 54.9M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107050K .......... .......... .......... .......... .......... 63% 38.9M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107100K .......... .......... .......... .......... .......... 63% 65.7M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107150K .......... .......... .......... .......... .......... 63% 66.2M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107200K .......... .......... .......... .......... .......... 63% 65.0M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107250K .......... .......... .......... .......... .......... 63% 82.5M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107300K .......... .......... .......... .......... .......... 63% 75.0M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107350K .......... .......... .......... .......... .......... 63% 61.4M 3s 2024-03-30T13:11:19.685 INFO:teuthology.orchestra.run.smithi066.stderr:107400K .......... .......... .......... .......... .......... 63% 52.7M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107450K .......... .......... .......... .......... .......... 63% 76.7M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107500K .......... .......... .......... .......... .......... 63% 57.5M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107550K .......... .......... .......... .......... .......... 63% 73.2M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107600K .......... .......... .......... .......... .......... 63% 61.2M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107650K .......... .......... .......... .......... .......... 63% 52.4M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107700K .......... .......... .......... .......... .......... 63% 69.6M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107750K .......... .......... .......... .......... .......... 63% 24.8M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107800K .......... .......... .......... .......... .......... 63% 31.0M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107850K .......... .......... .......... .......... .......... 64% 26.2M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107900K .......... .......... .......... .......... .......... 64% 24.9M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:107950K .......... .......... .......... .......... .......... 64% 51.8M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:108000K .......... .......... .......... .......... .......... 64% 32.4M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:108050K .......... .......... .......... .......... .......... 64% 5.89M 3s 2024-03-30T13:11:19.686 INFO:teuthology.orchestra.run.smithi066.stderr:108100K .......... .......... .......... .......... .......... 64% 6.26M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108150K .......... .......... .......... .......... .......... 64% 42.8M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108200K .......... .......... .......... .......... .......... 64% 64.6M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108250K .......... .......... .......... .......... .......... 64% 54.8M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108300K .......... .......... .......... .......... .......... 64% 62.7M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108350K .......... .......... .......... .......... .......... 64% 56.7M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108400K .......... .......... .......... .......... .......... 64% 53.0M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108450K .......... .......... .......... .......... .......... 64% 72.2M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108500K .......... .......... .......... .......... .......... 64% 61.2M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108550K .......... .......... .......... .......... .......... 64% 52.0M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108600K .......... .......... .......... .......... .......... 64% 66.8M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108650K .......... .......... .......... .......... .......... 64% 38.0M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108700K .......... .......... .......... .......... .......... 64% 82.3M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108750K .......... .......... .......... .......... .......... 64% 61.7M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108800K .......... .......... .......... .......... .......... 64% 61.5M 3s 2024-03-30T13:11:19.687 INFO:teuthology.orchestra.run.smithi066.stderr:108850K .......... .......... .......... .......... .......... 64% 57.8M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:108900K .......... .......... .......... .......... .......... 64% 25.4M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:108950K .......... .......... .......... .......... .......... 64% 39.1M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109000K .......... .......... .......... .......... .......... 64% 25.1M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109050K .......... .......... .......... .......... .......... 64% 31.9M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109100K .......... .......... .......... .......... .......... 64% 25.1M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109150K .......... .......... .......... .......... .......... 64% 20.8M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109200K .......... .......... .......... .......... .......... 64% 15.3M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109250K .......... .......... .......... .......... .......... 64% 5.13M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109300K .......... .......... .......... .......... .......... 64% 90.2M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109350K .......... .......... .......... .......... .......... 64% 94.4M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109400K .......... .......... .......... .......... .......... 64% 57.2M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109450K .......... .......... .......... .......... .......... 64% 107M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109500K .......... .......... .......... .......... .......... 64% 92.1M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109550K .......... .......... .......... .......... .......... 65% 86.4M 3s 2024-03-30T13:11:19.688 INFO:teuthology.orchestra.run.smithi066.stderr:109600K .......... .......... .......... .......... .......... 65% 72.1M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109650K .......... .......... .......... .......... .......... 65% 94.2M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109700K .......... .......... .......... .......... .......... 65% 76.7M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109750K .......... .......... .......... .......... .......... 65% 51.4M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109800K .......... .......... .......... .......... .......... 65% 29.6M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109850K .......... .......... .......... .......... .......... 65% 26.2M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109900K .......... .......... .......... .......... .......... 65% 23.0M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:109950K .......... .......... .......... .......... .......... 65% 37.3M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110000K .......... .......... .......... .......... .......... 65% 27.7M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110050K .......... .......... .......... .......... .......... 65% 22.2M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110100K .......... .......... .......... .......... .......... 65% 43.0M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110150K .......... .......... .......... .......... .......... 65% 23.0M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110200K .......... .......... .......... .......... .......... 65% 46.8M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110250K .......... .......... .......... .......... .......... 65% 25.3M 3s 2024-03-30T13:11:19.689 INFO:teuthology.orchestra.run.smithi066.stderr:110300K .......... .......... .......... .......... .......... 65% 29.0M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110350K .......... .......... .......... .......... .......... 65% 24.4M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110400K .......... .......... .......... .......... .......... 65% 22.4M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110450K .......... .......... .......... .......... .......... 65% 90.5M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110500K .......... .......... .......... .......... .......... 65% 24.7M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110550K .......... .......... .......... .......... .......... 65% 28.3M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110600K .......... .......... .......... .......... .......... 65% 8.88M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110650K .......... .......... .......... .......... .......... 65% 5.75M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110700K .......... .......... .......... .......... .......... 65% 12.7M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110750K .......... .......... .......... .......... .......... 65% 108M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110800K .......... .......... .......... .......... .......... 65% 97.8M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110850K .......... .......... .......... .......... .......... 65% 105M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110900K .......... .......... .......... .......... .......... 65% 65.5M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:110950K .......... .......... .......... .......... .......... 65% 95.4M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:111000K .......... .......... .......... .......... .......... 65% 54.6M 3s 2024-03-30T13:11:19.690 INFO:teuthology.orchestra.run.smithi066.stderr:111050K .......... .......... .......... .......... .......... 65% 85.1M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111100K .......... .......... .......... .......... .......... 65% 90.6M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111150K .......... .......... .......... .......... .......... 65% 95.9M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111200K .......... .......... .......... .......... .......... 65% 100M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111250K .......... .......... .......... .......... .......... 66% 77.6M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111300K .......... .......... .......... .......... .......... 66% 37.0M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111350K .......... .......... .......... .......... .......... 66% 37.9M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111400K .......... .......... .......... .......... .......... 66% 24.9M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111450K .......... .......... .......... .......... .......... 66% 42.7M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111500K .......... .......... .......... .......... .......... 66% 24.4M 3s 2024-03-30T13:11:19.691 INFO:teuthology.orchestra.run.smithi066.stderr:111550K .......... .......... .......... .......... .......... 66% 27.4M 3s 2024-03-30T13:11:19.693 INFO:teuthology.orchestra.run.smithi066.stderr:111600K .......... .......... .......... .......... .......... 66% 29.0M 3s 2024-03-30T13:11:19.693 INFO:teuthology.orchestra.run.smithi066.stderr:111650K .......... .......... .......... .......... .......... 66% 22.6M 3s 2024-03-30T13:11:19.693 INFO:teuthology.orchestra.run.smithi066.stderr:111700K .......... .......... .......... .......... .......... 66% 62.4M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:111750K .......... .......... .......... .......... .......... 66% 22.3M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:111800K .......... .......... .......... .......... .......... 66% 7.73M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:111850K .......... .......... .......... .......... .......... 66% 6.41M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:111900K .......... .......... .......... .......... .......... 66% 62.4M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:111950K .......... .......... .......... .......... .......... 66% 110M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112000K .......... .......... .......... .......... .......... 66% 70.8M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112050K .......... .......... .......... .......... .......... 66% 29.8M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112100K .......... .......... .......... .......... .......... 66% 28.9M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112150K .......... .......... .......... .......... .......... 66% 23.0M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112200K .......... .......... .......... .......... .......... 66% 65.0M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112250K .......... .......... .......... .......... .......... 66% 92.5M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112300K .......... .......... .......... .......... .......... 66% 97.8M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112350K .......... .......... .......... .......... .......... 66% 104M 3s 2024-03-30T13:11:19.694 INFO:teuthology.orchestra.run.smithi066.stderr:112400K .......... .......... .......... .......... .......... 66% 93.8M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112450K .......... .......... .......... .......... .......... 66% 98.8M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112500K .......... .......... .......... .......... .......... 66% 96.0M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112550K .......... .......... .......... .......... .......... 66% 26.4M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112600K .......... .......... .......... .......... .......... 66% 37.4M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112650K .......... .......... .......... .......... .......... 66% 24.6M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112700K .......... .......... .......... .......... .......... 66% 23.0M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112750K .......... .......... .......... .......... .......... 66% 42.4M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112800K .......... .......... .......... .......... .......... 66% 24.4M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112850K .......... .......... .......... .......... .......... 66% 35.5M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112900K .......... .......... .......... .......... .......... 67% 24.3M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:112950K .......... .......... .......... .......... .......... 67% 24.5M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:113000K .......... .......... .......... .......... .......... 67% 50.5M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:113050K .......... .......... .......... .......... .......... 67% 25.2M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:113100K .......... .......... .......... .......... .......... 67% 33.5M 3s 2024-03-30T13:11:19.695 INFO:teuthology.orchestra.run.smithi066.stderr:113150K .......... .......... .......... .......... .......... 67% 19.4M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113200K .......... .......... .......... .......... .......... 67% 4.50M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113250K .......... .......... .......... .......... .......... 67% 9.24M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113300K .......... .......... .......... .......... .......... 67% 106M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113350K .......... .......... .......... .......... .......... 67% 111M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113400K .......... .......... .......... .......... .......... 67% 99.8M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113450K .......... .......... .......... .......... .......... 67% 109M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113500K .......... .......... .......... .......... .......... 67% 111M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113550K .......... .......... .......... .......... .......... 67% 60.5M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113600K .......... .......... .......... .......... .......... 67% 102M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113650K .......... .......... .......... .......... .......... 67% 100M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113700K .......... .......... .......... .......... .......... 67% 102M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113750K .......... .......... .......... .......... .......... 67% 109M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113800K .......... .......... .......... .......... .......... 67% 104M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113850K .......... .......... .......... .......... .......... 67% 54.0M 3s 2024-03-30T13:11:19.696 INFO:teuthology.orchestra.run.smithi066.stderr:113900K .......... .......... .......... .......... .......... 67% 24.6M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:113950K .......... .......... .......... .......... .......... 67% 24.9M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114000K .......... .......... .......... .......... .......... 67% 40.2M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114050K .......... .......... .......... .......... .......... 67% 24.9M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114100K .......... .......... .......... .......... .......... 67% 40.1M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114150K .......... .......... .......... .......... .......... 67% 24.2M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114200K .......... .......... .......... .......... .......... 67% 23.2M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114250K .......... .......... .......... .......... .......... 67% 27.0M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114300K .......... .......... .......... .......... .......... 67% 56.2M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114350K .......... .......... .......... .......... .......... 67% 30.0M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114400K .......... .......... .......... .......... .......... 67% 4.79M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114450K .......... .......... .......... .......... .......... 67% 6.89M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114500K .......... .......... .......... .......... .......... 67% 110M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114550K .......... .......... .......... .......... .......... 67% 107M 3s 2024-03-30T13:11:19.697 INFO:teuthology.orchestra.run.smithi066.stderr:114600K .......... .......... .......... .......... .......... 68% 97.4M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114650K .......... .......... .......... .......... .......... 68% 49.1M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114700K .......... .......... .......... .......... .......... 68% 75.0M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114750K .......... .......... .......... .......... .......... 68% 70.4M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114800K .......... .......... .......... .......... .......... 68% 32.3M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114850K .......... .......... .......... .......... .......... 68% 26.3M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114900K .......... .......... .......... .......... .......... 68% 44.3M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:114950K .......... .......... .......... .......... .......... 68% 98.1M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115000K .......... .......... .......... .......... .......... 68% 94.8M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115050K .......... .......... .......... .......... .......... 68% 99.7M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115100K .......... .......... .......... .......... .......... 68% 96.6M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115150K .......... .......... .......... .......... .......... 68% 96.2M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115200K .......... .......... .......... .......... .......... 68% 44.8M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115250K .......... .......... .......... .......... .......... 68% 40.9M 3s 2024-03-30T13:11:19.698 INFO:teuthology.orchestra.run.smithi066.stderr:115300K .......... .......... .......... .......... .......... 68% 24.3M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115350K .......... .......... .......... .......... .......... 68% 24.1M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115400K .......... .......... .......... .......... .......... 68% 35.0M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115450K .......... .......... .......... .......... .......... 68% 24.1M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115500K .......... .......... .......... .......... .......... 68% 51.0M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115550K .......... .......... .......... .......... .......... 68% 25.0M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115600K .......... .......... .......... .......... .......... 68% 30.4M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115650K .......... .......... .......... .......... .......... 68% 25.3M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115700K .......... .......... .......... .......... .......... 68% 50.6M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115750K .......... .......... .......... .......... .......... 68% 10.1M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115800K .......... .......... .......... .......... .......... 68% 5.32M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115850K .......... .......... .......... .......... .......... 68% 102M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115900K .......... .......... .......... .......... .......... 68% 64.9M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:115950K .......... .......... .......... .......... .......... 68% 79.1M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:116000K .......... .......... .......... .......... .......... 68% 97.9M 3s 2024-03-30T13:11:19.699 INFO:teuthology.orchestra.run.smithi066.stderr:116050K .......... .......... .......... .......... .......... 68% 92.0M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116100K .......... .......... .......... .......... .......... 68% 105M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116150K .......... .......... .......... .......... .......... 68% 96.6M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116200K .......... .......... .......... .......... .......... 68% 110M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116250K .......... .......... .......... .......... .......... 68% 106M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116300K .......... .......... .......... .......... .......... 69% 25.9M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116350K .......... .......... .......... .......... .......... 69% 26.2M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116400K .......... .......... .......... .......... .......... 69% 38.5M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116450K .......... .......... .......... .......... .......... 69% 24.3M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116500K .......... .......... .......... .......... .......... 69% 40.2M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116550K .......... .......... .......... .......... .......... 69% 24.5M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116600K .......... .......... .......... .......... .......... 69% 31.4M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116650K .......... .......... .......... .......... .......... 69% 23.0M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116700K .......... .......... .......... .......... .......... 69% 31.2M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116750K .......... .......... .......... .......... .......... 69% 40.8M 3s 2024-03-30T13:11:19.700 INFO:teuthology.orchestra.run.smithi066.stderr:116800K .......... .......... .......... .......... .......... 69% 24.8M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:116850K .......... .......... .......... .......... .......... 69% 32.6M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:116900K .......... .......... .......... .......... .......... 69% 21.0M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:116950K .......... .......... .......... .......... .......... 69% 4.52M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117000K .......... .......... .......... .......... .......... 69% 29.3M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117050K .......... .......... .......... .......... .......... 69% 61.2M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117100K .......... .......... .......... .......... .......... 69% 16.6M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117150K .......... .......... .......... .......... .......... 69% 23.2M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117200K .......... .......... .......... .......... .......... 69% 111M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117250K .......... .......... .......... .......... .......... 69% 86.7M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117300K .......... .......... .......... .......... .......... 69% 126M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117350K .......... .......... .......... .......... .......... 69% 72.2M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117400K .......... .......... .......... .......... .......... 69% 88.6M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117450K .......... .......... .......... .......... .......... 69% 97.1M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117500K .......... .......... .......... .......... .......... 69% 103M 3s 2024-03-30T13:11:19.701 INFO:teuthology.orchestra.run.smithi066.stderr:117550K .......... .......... .......... .......... .......... 69% 98.0M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117600K .......... .......... .......... .......... .......... 69% 58.7M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117650K .......... .......... .......... .......... .......... 69% 41.3M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117700K .......... .......... .......... .......... .......... 69% 24.2M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117750K .......... .......... .......... .......... .......... 69% 24.3M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117800K .......... .......... .......... .......... .......... 69% 40.4M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117850K .......... .......... .......... .......... .......... 69% 24.5M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117900K .......... .......... .......... .......... .......... 69% 39.2M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:117950K .......... .......... .......... .......... .......... 70% 24.1M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118000K .......... .......... .......... .......... .......... 70% 23.1M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118050K .......... .......... .......... .......... .......... 70% 51.5M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118100K .......... .......... .......... .......... .......... 70% 23.3M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118150K .......... .......... .......... .......... .......... 70% 39.0M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118200K .......... .......... .......... .......... .......... 70% 35.4M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118250K .......... .......... .......... .......... .......... 70% 5.25M 3s 2024-03-30T13:11:19.702 INFO:teuthology.orchestra.run.smithi066.stderr:118300K .......... .......... .......... .......... .......... 70% 8.68M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118350K .......... .......... .......... .......... .......... 70% 39.5M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118400K .......... .......... .......... .......... .......... 70% 20.3M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118450K .......... .......... .......... .......... .......... 70% 101M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118500K .......... .......... .......... .......... .......... 70% 100M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118550K .......... .......... .......... .......... .......... 70% 95.9M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118600K .......... .......... .......... .......... .......... 70% 85.6M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118650K .......... .......... .......... .......... .......... 70% 75.7M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118700K .......... .......... .......... .......... .......... 70% 68.2M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118750K .......... .......... .......... .......... .......... 70% 93.1M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118800K .......... .......... .......... .......... .......... 70% 107M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118850K .......... .......... .......... .......... .......... 70% 108M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118900K .......... .......... .......... .......... .......... 70% 102M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:118950K .......... .......... .......... .......... .......... 70% 40.7M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:119000K .......... .......... .......... .......... .......... 70% 19.8M 3s 2024-03-30T13:11:19.703 INFO:teuthology.orchestra.run.smithi066.stderr:119050K .......... .......... .......... .......... .......... 70% 42.9M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119100K .......... .......... .......... .......... .......... 70% 23.9M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119150K .......... .......... .......... .......... .......... 70% 39.4M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119200K .......... .......... .......... .......... .......... 70% 23.8M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119250K .......... .......... .......... .......... .......... 70% 24.8M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119300K .......... .......... .......... .......... .......... 70% 42.4M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119350K .......... .......... .......... .......... .......... 70% 24.6M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119400K .......... .......... .......... .......... .......... 70% 34.0M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119450K .......... .......... .......... .......... .......... 70% 18.4M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119500K .......... .......... .......... .......... .......... 70% 5.28M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119550K .......... .......... .......... .......... .......... 70% 14.2M 3s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119600K .......... .......... .......... .......... .......... 70% 65.3M 2s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119650K .......... .......... .......... .......... .......... 71% 86.9M 2s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119700K .......... .......... .......... .......... .......... 71% 103M 2s 2024-03-30T13:11:19.704 INFO:teuthology.orchestra.run.smithi066.stderr:119750K .......... .......... .......... .......... .......... 71% 98.4M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:119800K .......... .......... .......... .......... .......... 71% 106M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:119850K .......... .......... .......... .......... .......... 71% 103M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:119900K .......... .......... .......... .......... .......... 71% 93.4M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:119950K .......... .......... .......... .......... .......... 71% 75.1M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120000K .......... .......... .......... .......... .......... 71% 53.3M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120050K .......... .......... .......... .......... .......... 71% 48.6M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120100K .......... .......... .......... .......... .......... 71% 26.4M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120150K .......... .......... .......... .......... .......... 71% 23.5M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120200K .......... .......... .......... .......... .......... 71% 53.1M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120250K .......... .......... .......... .......... .......... 71% 20.8M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120300K .......... .......... .......... .......... .......... 71% 42.0M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120350K .......... .......... .......... .......... .......... 71% 24.6M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120400K .......... .......... .......... .......... .......... 71% 23.2M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120450K .......... .......... .......... .......... .......... 71% 41.8M 2s 2024-03-30T13:11:19.705 INFO:teuthology.orchestra.run.smithi066.stderr:120500K .......... .......... .......... .......... .......... 71% 22.9M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120550K .......... .......... .......... .......... .......... 71% 43.6M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120600K .......... .......... .......... .......... .......... 71% 24.5M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120650K .......... .......... .......... .......... .......... 71% 21.1M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120700K .......... .......... .......... .......... .......... 71% 17.2M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120750K .......... .......... .......... .......... .......... 71% 5.58M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120800K .......... .......... .......... .......... .......... 71% 9.39M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120850K .......... .......... .......... .......... .......... 71% 63.8M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120900K .......... .......... .......... .......... .......... 71% 103M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:120950K .......... .......... .......... .......... .......... 71% 95.3M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:121000K .......... .......... .......... .......... .......... 71% 80.1M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:121050K .......... .......... .......... .......... .......... 71% 117M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:121100K .......... .......... .......... .......... .......... 71% 99.3M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:121150K .......... .......... .......... .......... .......... 71% 78.8M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:121200K .......... .......... .......... .......... .......... 71% 95.1M 2s 2024-03-30T13:11:19.706 INFO:teuthology.orchestra.run.smithi066.stderr:121250K .......... .......... .......... .......... .......... 71% 71.3M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121300K .......... .......... .......... .......... .......... 71% 96.8M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121350K .......... .......... .......... .......... .......... 72% 103M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121400K .......... .......... .......... .......... .......... 72% 24.2M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121450K .......... .......... .......... .......... .......... 72% 37.5M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121500K .......... .......... .......... .......... .......... 72% 11.0M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121550K .......... .......... .......... .......... .......... 72% 191M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121600K .......... .......... .......... .......... .......... 72% 108M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121650K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121700K .......... .......... .......... .......... .......... 72% 38.6M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121750K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121800K .......... .......... .......... .......... .......... 72% 44.8M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121850K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121900K .......... .......... .......... .......... .......... 72% 32.9M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:121950K .......... .......... .......... .......... .......... 72% 29.1M 2s 2024-03-30T13:11:19.707 INFO:teuthology.orchestra.run.smithi066.stderr:122000K .......... .......... .......... .......... .......... 72% 4.28M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122050K .......... .......... .......... .......... .......... 72% 12.0M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122100K .......... .......... .......... .......... .......... 72% 11.6M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122150K .......... .......... .......... .......... .......... 72% 86.3M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122200K .......... .......... .......... .......... .......... 72% 110M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122250K .......... .......... .......... .......... .......... 72% 99.7M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122300K .......... .......... .......... .......... .......... 72% 101M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122350K .......... .......... .......... .......... .......... 72% 49.4M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122400K .......... .......... .......... .......... .......... 72% 96.3M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122450K .......... .......... .......... .......... .......... 72% 108M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122500K .......... .......... .......... .......... .......... 72% 77.2M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122550K .......... .......... .......... .......... .......... 72% 85.3M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122600K .......... .......... .......... .......... .......... 72% 81.5M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122650K .......... .......... .......... .......... .......... 72% 77.1M 2s 2024-03-30T13:11:19.708 INFO:teuthology.orchestra.run.smithi066.stderr:122700K .......... .......... .......... .......... .......... 72% 63.9M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:122750K .......... .......... .......... .......... .......... 72% 83.6M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:122800K .......... .......... .......... .......... .......... 72% 26.3M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:122850K .......... .......... .......... .......... .......... 72% 42.7M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:122900K .......... .......... .......... .......... .......... 72% 24.2M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:122950K .......... .......... .......... .......... .......... 72% 40.0M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123000K .......... .......... .......... .......... .......... 72% 23.2M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123050K .......... .......... .......... .......... .......... 73% 25.3M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123100K .......... .......... .......... .......... .......... 73% 43.5M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123150K .......... .......... .......... .......... .......... 73% 24.9M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123200K .......... .......... .......... .......... .......... 73% 39.1M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123250K .......... .......... .......... .......... .......... 73% 20.4M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123300K .......... .......... .......... .......... .......... 73% 5.23M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123350K .......... .......... .......... .......... .......... 73% 11.0M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123400K .......... .......... .......... .......... .......... 73% 76.3M 2s 2024-03-30T13:11:19.709 INFO:teuthology.orchestra.run.smithi066.stderr:123450K .......... .......... .......... .......... .......... 73% 80.8M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123500K .......... .......... .......... .......... .......... 73% 103M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123550K .......... .......... .......... .......... .......... 73% 108M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123600K .......... .......... .......... .......... .......... 73% 83.7M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123650K .......... .......... .......... .......... .......... 73% 77.7M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123700K .......... .......... .......... .......... .......... 73% 107M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123750K .......... .......... .......... .......... .......... 73% 106M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123800K .......... .......... .......... .......... .......... 73% 106M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123850K .......... .......... .......... .......... .......... 73% 48.9M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123900K .......... .......... .......... .......... .......... 73% 25.7M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:123950K .......... .......... .......... .......... .......... 73% 37.2M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:124000K .......... .......... .......... .......... .......... 73% 25.6M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:124050K .......... .......... .......... .......... .......... 73% 24.5M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:124100K .......... .......... .......... .......... .......... 73% 41.2M 2s 2024-03-30T13:11:19.710 INFO:teuthology.orchestra.run.smithi066.stderr:124150K .......... .......... .......... .......... .......... 73% 24.3M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124200K .......... .......... .......... .......... .......... 73% 40.3M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124250K .......... .......... .......... .......... .......... 73% 24.0M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124300K .......... .......... .......... .......... .......... 73% 24.4M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124350K .......... .......... .......... .......... .......... 73% 39.8M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124400K .......... .......... .......... .......... .......... 73% 25.0M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124450K .......... .......... .......... .......... .......... 73% 32.8M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124500K .......... .......... .......... .......... .......... 73% 14.2M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124550K .......... .......... .......... .......... .......... 73% 5.36M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124600K .......... .......... .......... .......... .......... 73% 64.2M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124650K .......... .......... .......... .......... .......... 73% 81.9M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124700K .......... .......... .......... .......... .......... 74% 99.8M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124750K .......... .......... .......... .......... .......... 74% 95.5M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124800K .......... .......... .......... .......... .......... 74% 74.0M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124850K .......... .......... .......... .......... .......... 74% 115M 2s 2024-03-30T13:11:19.711 INFO:teuthology.orchestra.run.smithi066.stderr:124900K .......... .......... .......... .......... .......... 74% 98.9M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:124950K .......... .......... .......... .......... .......... 74% 79.6M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125000K .......... .......... .......... .......... .......... 74% 34.3M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125050K .......... .......... .......... .......... .......... 74% 22.7M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125100K .......... .......... .......... .......... .......... 74% 48.1M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125150K .......... .......... .......... .......... .......... 74% 24.1M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125200K .......... .......... .......... .......... .......... 74% 31.9M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125250K .......... .......... .......... .......... .......... 74% 29.3M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125300K .......... .......... .......... .......... .......... 74% 24.6M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125350K .......... .......... .......... .......... .......... 74% 39.8M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125400K .......... .......... .......... .......... .......... 74% 24.3M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125450K .......... .......... .......... .......... .......... 74% 24.4M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125500K .......... .......... .......... .......... .......... 74% 39.4M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125550K .......... .......... .......... .......... .......... 74% 24.4M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125600K .......... .......... .......... .......... .......... 74% 43.2M 2s 2024-03-30T13:11:19.712 INFO:teuthology.orchestra.run.smithi066.stderr:125650K .......... .......... .......... .......... .......... 74% 24.9M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:125700K .......... .......... .......... .......... .......... 74% 30.7M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:125750K .......... .......... .......... .......... .......... 74% 20.7M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:125800K .......... .......... .......... .......... .......... 74% 4.11M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:125850K .......... .......... .......... .......... .......... 74% 104M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:125900K .......... .......... .......... .......... .......... 74% 98.4M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:125950K .......... .......... .......... .......... .......... 74% 98.2M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126000K .......... .......... .......... .......... .......... 74% 104M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126050K .......... .......... .......... .......... .......... 74% 86.8M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126100K .......... .......... .......... .......... .......... 74% 116M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126150K .......... .......... .......... .......... .......... 74% 82.6M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126200K .......... .......... .......... .......... .......... 74% 73.7M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126250K .......... .......... .......... .......... .......... 74% 82.4M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126300K .......... .......... .......... .......... .......... 74% 29.7M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126350K .......... .......... .......... .......... .......... 74% 49.7M 2s 2024-03-30T13:11:19.713 INFO:teuthology.orchestra.run.smithi066.stderr:126400K .......... .......... .......... .......... .......... 75% 24.8M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126450K .......... .......... .......... .......... .......... 75% 23.8M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126500K .......... .......... .......... .......... .......... 75% 40.8M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126550K .......... .......... .......... .......... .......... 75% 24.1M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126600K .......... .......... .......... .......... .......... 75% 41.8M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126650K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126700K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126750K .......... .......... .......... .......... .......... 75% 37.1M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126800K .......... .......... .......... .......... .......... 75% 24.4M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126850K .......... .......... .......... .......... .......... 75% 47.6M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126900K .......... .......... .......... .......... .......... 75% 10.8M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:126950K .......... .......... .......... .......... .......... 75% 38.7M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:127000K .......... .......... .......... .......... .......... 75% 4.51M 2s 2024-03-30T13:11:19.714 INFO:teuthology.orchestra.run.smithi066.stderr:127050K .......... .......... .......... .......... .......... 75% 45.3M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127100K .......... .......... .......... .......... .......... 75% 84.8M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127150K .......... .......... .......... .......... .......... 75% 53.9M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127200K .......... .......... .......... .......... .......... 75% 105M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127250K .......... .......... .......... .......... .......... 75% 108M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127300K .......... .......... .......... .......... .......... 75% 70.4M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127350K .......... .......... .......... .......... .......... 75% 99.3M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127400K .......... .......... .......... .......... .......... 75% 107M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127450K .......... .......... .......... .......... .......... 75% 104M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127500K .......... .......... .......... .......... .......... 75% 77.9M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127550K .......... .......... .......... .......... .......... 75% 37.0M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127600K .......... .......... .......... .......... .......... 75% 47.1M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127650K .......... .......... .......... .......... .......... 75% 24.0M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127700K .......... .......... .......... .......... .......... 75% 24.4M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127750K .......... .......... .......... .......... .......... 75% 40.5M 2s 2024-03-30T13:11:19.715 INFO:teuthology.orchestra.run.smithi066.stderr:127800K .......... .......... .......... .......... .......... 75% 23.8M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:127850K .......... .......... .......... .......... .......... 75% 38.5M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:127900K .......... .......... .......... .......... .......... 75% 25.4M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:127950K .......... .......... .......... .......... .......... 75% 24.5M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128000K .......... .......... .......... .......... .......... 75% 38.2M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128050K .......... .......... .......... .......... .......... 75% 24.2M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128100K .......... .......... .......... .......... .......... 76% 24.6M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128150K .......... .......... .......... .......... .......... 76% 48.0M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128200K .......... .......... .......... .......... .......... 76% 9.36M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128250K .......... .......... .......... .......... .......... 76% 5.30M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128300K .......... .......... .......... .......... .......... 76% 17.1M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128350K .......... .......... .......... .......... .......... 76% 108M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128400K .......... .......... .......... .......... .......... 76% 93.0M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128450K .......... .......... .......... .......... .......... 76% 62.2M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128500K .......... .......... .......... .......... .......... 76% 104M 2s 2024-03-30T13:11:19.716 INFO:teuthology.orchestra.run.smithi066.stderr:128550K .......... .......... .......... .......... .......... 76% 88.5M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128600K .......... .......... .......... .......... .......... 76% 81.4M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128650K .......... .......... .......... .......... .......... 76% 115M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128700K .......... .......... .......... .......... .......... 76% 97.1M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128750K .......... .......... .......... .......... .......... 76% 75.4M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128800K .......... .......... .......... .......... .......... 76% 56.4M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128850K .......... .......... .......... .......... .......... 76% 36.7M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128900K .......... .......... .......... .......... .......... 76% 33.6M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:128950K .......... .......... .......... .......... .......... 76% 40.5M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129000K .......... .......... .......... .......... .......... 76% 39.4M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129050K .......... .......... .......... .......... .......... 76% 24.3M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129100K .......... .......... .......... .......... .......... 76% 24.5M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129150K .......... .......... .......... .......... .......... 76% 39.0M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129200K .......... .......... .......... .......... .......... 76% 25.5M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129250K .......... .......... .......... .......... .......... 76% 36.9M 2s 2024-03-30T13:11:19.717 INFO:teuthology.orchestra.run.smithi066.stderr:129300K .......... .......... .......... .......... .......... 76% 24.7M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129350K .......... .......... .......... .......... .......... 76% 24.2M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129400K .......... .......... .......... .......... .......... 76% 45.7M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129450K .......... .......... .......... .......... .......... 76% 14.7M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129500K .......... .......... .......... .......... .......... 76% 6.68M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129550K .......... .......... .......... .......... .......... 76% 56.0M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129600K .......... .......... .......... .......... .......... 76% 55.6M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129650K .......... .......... .......... .......... .......... 76% 88.8M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129700K .......... .......... .......... .......... .......... 76% 74.3M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129750K .......... .......... .......... .......... .......... 77% 101M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129800K .......... .......... .......... .......... .......... 77% 83.1M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129850K .......... .......... .......... .......... .......... 77% 48.5M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129900K .......... .......... .......... .......... .......... 77% 53.0M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:129950K .......... .......... .......... .......... .......... 77% 28.8M 2s 2024-03-30T13:11:19.718 INFO:teuthology.orchestra.run.smithi066.stderr:130000K .......... .......... .......... .......... .......... 77% 27.5M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130050K .......... .......... .......... .......... .......... 77% 32.8M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130100K .......... .......... .......... .......... .......... 77% 12.0M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130150K .......... .......... .......... .......... .......... 77% 55.4M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130200K .......... .......... .......... .......... .......... 77% 40.2M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130250K .......... .......... .......... .......... .......... 77% 33.0M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130300K .......... .......... .......... .......... .......... 77% 39.0M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130350K .......... .......... .......... .......... .......... 77% 23.9M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130400K .......... .......... .......... .......... .......... 77% 41.5M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130450K .......... .......... .......... .......... .......... 77% 24.3M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130500K .......... .......... .......... .......... .......... 77% 39.9M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130550K .......... .......... .......... .......... .......... 77% 24.6M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130600K .......... .......... .......... .......... .......... 77% 22.7M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130650K .......... .......... .......... .......... .......... 77% 48.4M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130700K .......... .......... .......... .......... .......... 77% 31.4M 2s 2024-03-30T13:11:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:130750K .......... .......... .......... .......... .......... 77% 5.93M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:130800K .......... .......... .......... .......... .......... 77% 6.24M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:130850K .......... .......... .......... .......... .......... 77% 5.90M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:130900K .......... .......... .......... .......... .......... 77% 99.8M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:130950K .......... .......... .......... .......... .......... 77% 88.2M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131000K .......... .......... .......... .......... .......... 77% 87.0M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131050K .......... .......... .......... .......... .......... 77% 19.7M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131100K .......... .......... .......... .......... .......... 77% 15.2M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131150K .......... .......... .......... .......... .......... 77% 23.6M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131200K .......... .......... .......... .......... .......... 77% 60.1M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131250K .......... .......... .......... .......... .......... 77% 105M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131300K .......... .......... .......... .......... .......... 77% 98.9M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131350K .......... .......... .......... .......... .......... 77% 96.3M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131400K .......... .......... .......... .......... .......... 77% 103M 2s 2024-03-30T13:11:19.720 INFO:teuthology.orchestra.run.smithi066.stderr:131450K .......... .......... .......... .......... .......... 78% 87.5M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131500K .......... .......... .......... .......... .......... 78% 70.0M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131550K .......... .......... .......... .......... .......... 78% 109M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131600K .......... .......... .......... .......... .......... 78% 47.4M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131650K .......... .......... .......... .......... .......... 78% 91.3M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131700K .......... .......... .......... .......... .......... 78% 98.8M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131750K .......... .......... .......... .......... .......... 78% 95.0M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131800K .......... .......... .......... .......... .......... 78% 109M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131850K .......... .......... .......... .......... .......... 78% 92.7M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131900K .......... .......... .......... .......... .......... 78% 110M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:131950K .......... .......... .......... .......... .......... 78% 102M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:132000K .......... .......... .......... .......... .......... 78% 32.6M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:132050K .......... .......... .......... .......... .......... 78% 24.1M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:132100K .......... .......... .......... .......... .......... 78% 11.6M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:132150K .......... .......... .......... .......... .......... 78% 113M 2s 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi066.stderr:132200K .......... .......... .......... .......... .......... 78% 74.6M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132250K .......... .......... .......... .......... .......... 78% 9.55M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132300K .......... .......... .......... .......... .......... 78% 5.26M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132350K .......... .......... .......... .......... .......... 78% 84.1M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132400K .......... .......... .......... .......... .......... 78% 100M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132450K .......... .......... .......... .......... .......... 78% 102M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132500K .......... .......... .......... .......... .......... 78% 52.9M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132550K .......... .......... .......... .......... .......... 78% 92.3M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132600K .......... .......... .......... .......... .......... 78% 93.0M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132650K .......... .......... .......... .......... .......... 78% 108M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132700K .......... .......... .......... .......... .......... 78% 110M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132750K .......... .......... .......... .......... .......... 78% 82.8M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132800K .......... .......... .......... .......... .......... 78% 111M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132850K .......... .......... .......... .......... .......... 78% 33.0M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132900K .......... .......... .......... .......... .......... 78% 41.8M 2s 2024-03-30T13:11:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:132950K .......... .......... .......... .......... .......... 78% 25.4M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133000K .......... .......... .......... .......... .......... 78% 24.5M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133050K .......... .......... .......... .......... .......... 78% 36.0M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133100K .......... .......... .......... .......... .......... 78% 25.3M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133150K .......... .......... .......... .......... .......... 79% 42.2M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133200K .......... .......... .......... .......... .......... 79% 23.3M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133250K .......... .......... .......... .......... .......... 79% 23.1M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133300K .......... .......... .......... .......... .......... 79% 35.0M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133350K .......... .......... .......... .......... .......... 79% 27.6M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133400K .......... .......... .......... .......... .......... 79% 10.1M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133450K .......... .......... .......... .......... .......... 79% 7.34M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133500K .......... .......... .......... .......... .......... 79% 6.68M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133550K .......... .......... .......... .......... .......... 79% 111M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133600K .......... .......... .......... .......... .......... 79% 54.0M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133650K .......... .......... .......... .......... .......... 79% 98.5M 2s 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:133700K .......... .......... .......... .......... .......... 79% 97.6M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:133750K .......... .......... .......... .......... .......... 79% 93.8M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:133800K .......... .......... .......... .......... .......... 79% 127M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:133850K .......... .......... .......... .......... .......... 79% 68.7M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:133900K .......... .......... .......... .......... .......... 79% 78.3M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:133950K .......... .......... .......... .......... .......... 79% 105M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134000K .......... .......... .......... .......... .......... 79% 90.5M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134050K .......... .......... .......... .......... .......... 79% 111M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134100K .......... .......... .......... .......... .......... 79% 79.0M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134150K .......... .......... .......... .......... .......... 79% 27.8M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134200K .......... .......... .......... .......... .......... 79% 41.2M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134250K .......... .......... .......... .......... .......... 79% 24.8M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134300K .......... .......... .......... .......... .......... 79% 39.3M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134350K .......... .......... .......... .......... .......... 79% 24.2M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134400K .......... .......... .......... .......... .......... 79% 24.5M 2s 2024-03-30T13:11:19.724 INFO:teuthology.orchestra.run.smithi066.stderr:134450K .......... .......... .......... .......... .......... 79% 39.1M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134500K .......... .......... .......... .......... .......... 79% 24.5M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134550K .......... .......... .......... .......... .......... 79% 28.6M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134600K .......... .......... .......... .......... .......... 79% 23.1M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134650K .......... .......... .......... .......... .......... 79% 24.6M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134700K .......... .......... .......... .......... .......... 79% 111M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134750K .......... .......... .......... .......... .......... 79% 26.3M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134800K .......... .......... .......... .......... .......... 79% 25.7M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134850K .......... .......... .......... .......... .......... 80% 19.4M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134900K .......... .......... .......... .......... .......... 80% 4.32M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:134950K .......... .......... .......... .......... .......... 80% 16.4M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:135000K .......... .......... .......... .......... .......... 80% 48.7M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:135050K .......... .......... .......... .......... .......... 80% 97.7M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:135100K .......... .......... .......... .......... .......... 80% 71.4M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:135150K .......... .......... .......... .......... .......... 80% 56.3M 2s 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi066.stderr:135200K .......... .......... .......... .......... .......... 80% 105M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135250K .......... .......... .......... .......... .......... 80% 110M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135300K .......... .......... .......... .......... .......... 80% 49.4M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135350K .......... .......... .......... .......... .......... 80% 32.2M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135400K .......... .......... .......... .......... .......... 80% 97.9M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135450K .......... .......... .......... .......... .......... 80% 118M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135500K .......... .......... .......... .......... .......... 80% 73.5M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135550K .......... .......... .......... .......... .......... 80% 91.1M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135600K .......... .......... .......... .......... .......... 80% 36.7M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135650K .......... .......... .......... .......... .......... 80% 24.0M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135700K .......... .......... .......... .......... .......... 80% 41.9M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135750K .......... .......... .......... .......... .......... 80% 24.7M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135800K .......... .......... .......... .......... .......... 80% 38.9M 2s 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi066.stderr:135850K .......... .......... .......... .......... .......... 80% 23.9M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:135900K .......... .......... .......... .......... .......... 80% 24.6M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:135950K .......... .......... .......... .......... .......... 80% 32.2M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136000K .......... .......... .......... .......... .......... 80% 5.19M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136050K .......... .......... .......... .......... .......... 80% 9.34M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136100K .......... .......... .......... .......... .......... 80% 94.3M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136150K .......... .......... .......... .......... .......... 80% 96.4M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136200K .......... .......... .......... .......... .......... 80% 108M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136250K .......... .......... .......... .......... .......... 80% 90.0M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136300K .......... .......... .......... .......... .......... 80% 58.3M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136350K .......... .......... .......... .......... .......... 80% 68.7M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136400K .......... .......... .......... .......... .......... 80% 51.2M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136450K .......... .......... .......... .......... .......... 80% 107M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136500K .......... .......... .......... .......... .......... 81% 66.9M 2s 2024-03-30T13:11:19.727 INFO:teuthology.orchestra.run.smithi066.stderr:136550K .......... .......... .......... .......... .......... 81% 106M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136600K .......... .......... .......... .......... .......... 81% 64.1M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136650K .......... .......... .......... .......... .......... 81% 23.0M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136700K .......... .......... .......... .......... .......... 81% 46.5M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136750K .......... .......... .......... .......... .......... 81% 25.2M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136800K .......... .......... .......... .......... .......... 81% 30.8M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136850K .......... .......... .......... .......... .......... 81% 30.4M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136900K .......... .......... .......... .......... .......... 81% 23.0M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:136950K .......... .......... .......... .......... .......... 81% 44.7M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137000K .......... .......... .......... .......... .......... 81% 24.5M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137050K .......... .......... .......... .......... .......... 81% 23.1M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137100K .......... .......... .......... .......... .......... 81% 41.2M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137150K .......... .......... .......... .......... .......... 81% 24.1M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137200K .......... .......... .......... .......... .......... 81% 41.5M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137250K .......... .......... .......... .......... .......... 81% 25.6M 2s 2024-03-30T13:11:19.728 INFO:teuthology.orchestra.run.smithi066.stderr:137300K .......... .......... .......... .......... .......... 81% 30.5M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137350K .......... .......... .......... .......... .......... 81% 24.2M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137400K .......... .......... .......... .......... .......... 81% 24.8M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137450K .......... .......... .......... .......... .......... 81% 61.9M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137500K .......... .......... .......... .......... .......... 81% 6.61M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137550K .......... .......... .......... .......... .......... 81% 6.38M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137600K .......... .......... .......... .......... .......... 81% 99.2M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137650K .......... .......... .......... .......... .......... 81% 105M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137700K .......... .......... .......... .......... .......... 81% 107M 2s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137750K .......... .......... .......... .......... .......... 81% 108M 1s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137800K .......... .......... .......... .......... .......... 81% 106M 1s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137850K .......... .......... .......... .......... .......... 81% 36.1M 1s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137900K .......... .......... .......... .......... .......... 81% 85.2M 1s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:137950K .......... .......... .......... .......... .......... 81% 103M 1s 2024-03-30T13:11:19.729 INFO:teuthology.orchestra.run.smithi066.stderr:138000K .......... .......... .......... .......... .......... 81% 102M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138050K .......... .......... .......... .......... .......... 81% 77.0M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138100K .......... .......... .......... .......... .......... 81% 37.5M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138150K .......... .......... .......... .......... .......... 81% 24.8M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138200K .......... .......... .......... .......... .......... 82% 40.3M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138250K .......... .......... .......... .......... .......... 82% 24.8M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138300K .......... .......... .......... .......... .......... 82% 24.6M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138350K .......... .......... .......... .......... .......... 82% 36.2M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138400K .......... .......... .......... .......... .......... 82% 24.8M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138450K .......... .......... .......... .......... .......... 82% 37.6M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138500K .......... .......... .......... .......... .......... 82% 32.0M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138550K .......... .......... .......... .......... .......... 82% 25.7M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138600K .......... .......... .......... .......... .......... 82% 7.13M 1s 2024-03-30T13:11:19.730 INFO:teuthology.orchestra.run.smithi066.stderr:138650K .......... .......... .......... .......... .......... 82% 5.35M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:138700K .......... .......... .......... .......... .......... 82% 52.6M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:138750K .......... .......... .......... .......... .......... 82% 75.7M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:138800K .......... .......... .......... .......... .......... 82% 98.1M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:138850K .......... .......... .......... .......... .......... 82% 82.2M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:138900K .......... .......... .......... .......... .......... 82% 32.9M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:138950K .......... .......... .......... .......... .......... 82% 93.2M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139000K .......... .......... .......... .......... .......... 82% 107M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139050K .......... .......... .......... .......... .......... 82% 107M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139100K .......... .......... .......... .......... .......... 82% 106M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139150K .......... .......... .......... .......... .......... 82% 98.2M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139200K .......... .......... .......... .......... .......... 82% 96.7M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139250K .......... .......... .......... .......... .......... 82% 84.5M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139300K .......... .......... .......... .......... .......... 82% 33.6M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139350K .......... .......... .......... .......... .......... 82% 24.6M 1s 2024-03-30T13:11:19.731 INFO:teuthology.orchestra.run.smithi066.stderr:139400K .......... .......... .......... .......... .......... 82% 27.8M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139450K .......... .......... .......... .......... .......... 82% 26.0M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139500K .......... .......... .......... .......... .......... 82% 40.8M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139550K .......... .......... .......... .......... .......... 82% 25.0M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139600K .......... .......... .......... .......... .......... 82% 34.6M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139650K .......... .......... .......... .......... .......... 82% 25.9M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139700K .......... .......... .......... .......... .......... 82% 23.6M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139750K .......... .......... .......... .......... .......... 82% 45.5M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139800K .......... .......... .......... .......... .......... 82% 25.3M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139850K .......... .......... .......... .......... .......... 82% 30.5M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139900K .......... .......... .......... .......... .......... 83% 24.8M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:139950K .......... .......... .......... .......... .......... 83% 18.9M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:140000K .......... .......... .......... .......... .......... 83% 5.26M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:140050K .......... .......... .......... .......... .......... 83% 5.17M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:140100K .......... .......... .......... .......... .......... 83% 7.33M 1s 2024-03-30T13:11:19.732 INFO:teuthology.orchestra.run.smithi066.stderr:140150K .......... .......... .......... .......... .......... 83% 119M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140200K .......... .......... .......... .......... .......... 83% 108M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140250K .......... .......... .......... .......... .......... 83% 100M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140300K .......... .......... .......... .......... .......... 83% 108M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140350K .......... .......... .......... .......... .......... 83% 44.3M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140400K .......... .......... .......... .......... .......... 83% 89.8M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140450K .......... .......... .......... .......... .......... 83% 48.9M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140500K .......... .......... .......... .......... .......... 83% 103M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140550K .......... .......... .......... .......... .......... 83% 33.4M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140600K .......... .......... .......... .......... .......... 83% 75.2M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140650K .......... .......... .......... .......... .......... 83% 79.1M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140700K .......... .......... .......... .......... .......... 83% 64.3M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140750K .......... .......... .......... .......... .......... 83% 103M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140800K .......... .......... .......... .......... .......... 83% 78.5M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140850K .......... .......... .......... .......... .......... 83% 64.1M 1s 2024-03-30T13:11:19.733 INFO:teuthology.orchestra.run.smithi066.stderr:140900K .......... .......... .......... .......... .......... 83% 87.7M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:140950K .......... .......... .......... .......... .......... 83% 50.4M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141000K .......... .......... .......... .......... .......... 83% 93.1M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141050K .......... .......... .......... .......... .......... 83% 65.9M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141100K .......... .......... .......... .......... .......... 83% 110M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141150K .......... .......... .......... .......... .......... 83% 63.9M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141200K .......... .......... .......... .......... .......... 83% 44.3M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141250K .......... .......... .......... .......... .......... 83% 32.9M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141300K .......... .......... .......... .......... .......... 83% 22.9M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141350K .......... .......... .......... .......... .......... 83% 28.8M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141400K .......... .......... .......... .......... .......... 83% 10.7M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141450K .......... .......... .......... .......... .......... 83% 6.37M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141500K .......... .......... .......... .......... .......... 83% 17.7M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141550K .......... .......... .......... .......... .......... 84% 38.2M 1s 2024-03-30T13:11:19.734 INFO:teuthology.orchestra.run.smithi066.stderr:141600K .......... .......... .......... .......... .......... 84% 65.6M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141650K .......... .......... .......... .......... .......... 84% 65.3M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141700K .......... .......... .......... .......... .......... 84% 106M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141750K .......... .......... .......... .......... .......... 84% 103M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141800K .......... .......... .......... .......... .......... 84% 103M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141850K .......... .......... .......... .......... .......... 84% 58.3M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141900K .......... .......... .......... .......... .......... 84% 93.0M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:141950K .......... .......... .......... .......... .......... 84% 109M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142000K .......... .......... .......... .......... .......... 84% 93.9M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142050K .......... .......... .......... .......... .......... 84% 31.1M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142100K .......... .......... .......... .......... .......... 84% 24.3M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142150K .......... .......... .......... .......... .......... 84% 35.8M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142200K .......... .......... .......... .......... .......... 84% 24.4M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142250K .......... .......... .......... .......... .......... 84% 24.0M 1s 2024-03-30T13:11:19.735 INFO:teuthology.orchestra.run.smithi066.stderr:142300K .......... .......... .......... .......... .......... 84% 25.7M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142350K .......... .......... .......... .......... .......... 84% 21.9M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142400K .......... .......... .......... .......... .......... 84% 49.0M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142450K .......... .......... .......... .......... .......... 84% 35.0M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142500K .......... .......... .......... .......... .......... 84% 31.6M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142550K .......... .......... .......... .......... .......... 84% 5.71M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142600K .......... .......... .......... .......... .......... 84% 3.64M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142650K .......... .......... .......... .......... .......... 84% 16.0M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142700K .......... .......... .......... .......... .......... 84% 10.5M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142750K .......... .......... .......... .......... .......... 84% 15.6M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142800K .......... .......... .......... .......... .......... 84% 22.7M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142850K .......... .......... .......... .......... .......... 84% 28.1M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142900K .......... .......... .......... .......... .......... 84% 56.5M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:142950K .......... .......... .......... .......... .......... 84% 23.0M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:143000K .......... .......... .......... .......... .......... 84% 94.8M 1s 2024-03-30T13:11:19.736 INFO:teuthology.orchestra.run.smithi066.stderr:143050K .......... .......... .......... .......... .......... 84% 91.2M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143100K .......... .......... .......... .......... .......... 84% 72.4M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143150K .......... .......... .......... .......... .......... 84% 108M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143200K .......... .......... .......... .......... .......... 84% 80.5M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143250K .......... .......... .......... .......... .......... 85% 103M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143300K .......... .......... .......... .......... .......... 85% 81.2M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143350K .......... .......... .......... .......... .......... 85% 84.6M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143400K .......... .......... .......... .......... .......... 85% 99.6M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143450K .......... .......... .......... .......... .......... 85% 99.8M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143500K .......... .......... .......... .......... .......... 85% 102M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143550K .......... .......... .......... .......... .......... 85% 78.8M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143600K .......... .......... .......... .......... .......... 85% 58.7M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143650K .......... .......... .......... .......... .......... 85% 65.0M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143700K .......... .......... .......... .......... .......... 85% 79.8M 1s 2024-03-30T13:11:19.737 INFO:teuthology.orchestra.run.smithi066.stderr:143750K .......... .......... .......... .......... .......... 85% 98.3M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:143800K .......... .......... .......... .......... .......... 85% 94.0M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:143850K .......... .......... .......... .......... .......... 85% 76.3M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:143900K .......... .......... .......... .......... .......... 85% 76.5M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:143950K .......... .......... .......... .......... .......... 85% 58.0M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144000K .......... .......... .......... .......... .......... 85% 60.1M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144050K .......... .......... .......... .......... .......... 85% 67.8M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144100K .......... .......... .......... .......... .......... 85% 72.8M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144150K .......... .......... .......... .......... .......... 85% 21.7M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144200K .......... .......... .......... .......... .......... 85% 61.9M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144250K .......... .......... .......... .......... .......... 85% 6.86M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144300K .......... .......... .......... .......... .......... 85% 10.1M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144350K .......... .......... .......... .......... .......... 85% 9.06M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144400K .......... .......... .......... .......... .......... 85% 20.1M 1s 2024-03-30T13:11:19.738 INFO:teuthology.orchestra.run.smithi066.stderr:144450K .......... .......... .......... .......... .......... 85% 42.8M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144500K .......... .......... .......... .......... .......... 85% 28.7M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144550K .......... .......... .......... .......... .......... 85% 29.9M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144600K .......... .......... .......... .......... .......... 85% 30.1M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144650K .......... .......... .......... .......... .......... 85% 39.6M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144700K .......... .......... .......... .......... .......... 85% 57.7M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144750K .......... .......... .......... .......... .......... 85% 46.7M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144800K .......... .......... .......... .......... .......... 85% 73.0M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144850K .......... .......... .......... .......... .......... 85% 51.0M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144900K .......... .......... .......... .......... .......... 85% 51.6M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:144950K .......... .......... .......... .......... .......... 86% 62.7M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:145000K .......... .......... .......... .......... .......... 86% 52.8M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:145050K .......... .......... .......... .......... .......... 86% 60.4M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:145100K .......... .......... .......... .......... .......... 86% 77.5M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:145150K .......... .......... .......... .......... .......... 86% 25.1M 1s 2024-03-30T13:11:19.739 INFO:teuthology.orchestra.run.smithi066.stderr:145200K .......... .......... .......... .......... .......... 86% 29.0M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145250K .......... .......... .......... .......... .......... 86% 20.2M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145300K .......... .......... .......... .......... .......... 86% 24.7M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145350K .......... .......... .......... .......... .......... 86% 22.9M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145400K .......... .......... .......... .......... .......... 86% 17.0M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145450K .......... .......... .......... .......... .......... 86% 4.49M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145500K .......... .......... .......... .......... .......... 86% 4.67M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145550K .......... .......... .......... .......... .......... 86% 69.1M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145600K .......... .......... .......... .......... .......... 86% 31.7M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145650K .......... .......... .......... .......... .......... 86% 48.5M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145700K .......... .......... .......... .......... .......... 86% 29.4M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145750K .......... .......... .......... .......... .......... 86% 104M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145800K .......... .......... .......... .......... .......... 86% 91.9M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145850K .......... .......... .......... .......... .......... 86% 103M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145900K .......... .......... .......... .......... .......... 86% 107M 1s 2024-03-30T13:11:19.740 INFO:teuthology.orchestra.run.smithi066.stderr:145950K .......... .......... .......... .......... .......... 86% 108M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146000K .......... .......... .......... .......... .......... 86% 108M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146050K .......... .......... .......... .......... .......... 86% 76.9M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146100K .......... .......... .......... .......... .......... 86% 96.5M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146150K .......... .......... .......... .......... .......... 86% 50.6M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146200K .......... .......... .......... .......... .......... 86% 47.2M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146250K .......... .......... .......... .......... .......... 86% 41.5M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146300K .......... .......... .......... .......... .......... 86% 95.9M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146350K .......... .......... .......... .......... .......... 86% 44.0M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146400K .......... .......... .......... .......... .......... 86% 55.6M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146450K .......... .......... .......... .......... .......... 86% 79.8M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146500K .......... .......... .......... .......... .......... 86% 64.3M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146550K .......... .......... .......... .......... .......... 86% 70.6M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146600K .......... .......... .......... .......... .......... 86% 62.8M 1s 2024-03-30T13:11:19.741 INFO:teuthology.orchestra.run.smithi066.stderr:146650K .......... .......... .......... .......... .......... 87% 85.1M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:146700K .......... .......... .......... .......... .......... 87% 29.8M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:146750K .......... .......... .......... .......... .......... 87% 27.2M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:146800K .......... .......... .......... .......... .......... 87% 72.4M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:146850K .......... .......... .......... .......... .......... 87% 65.2M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:146900K .......... .......... .......... .......... .......... 87% 34.9M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:146950K .......... .......... .......... .......... .......... 87% 14.8M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147000K .......... .......... .......... .......... .......... 87% 49.3M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147050K .......... .......... .......... .......... .......... 87% 49.4M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147100K .......... .......... .......... .......... .......... 87% 5.77M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147150K .......... .......... .......... .......... .......... 87% 4.42M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147200K .......... .......... .......... .......... .......... 87% 46.7M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147250K .......... .......... .......... .......... .......... 87% 29.9M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147300K .......... .......... .......... .......... .......... 87% 32.3M 1s 2024-03-30T13:11:19.742 INFO:teuthology.orchestra.run.smithi066.stderr:147350K .......... .......... .......... .......... .......... 87% 27.5M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147400K .......... .......... .......... .......... .......... 87% 22.2M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147450K .......... .......... .......... .......... .......... 87% 48.6M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147500K .......... .......... .......... .......... .......... 87% 33.4M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147550K .......... .......... .......... .......... .......... 87% 81.3M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147600K .......... .......... .......... .......... .......... 87% 84.1M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147650K .......... .......... .......... .......... .......... 87% 13.9M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147700K .......... .......... .......... .......... .......... 87% 41.2M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147750K .......... .......... .......... .......... .......... 87% 46.7M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147800K .......... .......... .......... .......... .......... 87% 83.7M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147850K .......... .......... .......... .......... .......... 87% 75.1M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147900K .......... .......... .......... .......... .......... 87% 49.9M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:147950K .......... .......... .......... .......... .......... 87% 62.4M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:148000K .......... .......... .......... .......... .......... 87% 43.6M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:148050K .......... .......... .......... .......... .......... 87% 99.0M 1s 2024-03-30T13:11:19.743 INFO:teuthology.orchestra.run.smithi066.stderr:148100K .......... .......... .......... .......... .......... 87% 59.3M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148150K .......... .......... .......... .......... .......... 87% 48.3M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148200K .......... .......... .......... .......... .......... 87% 55.4M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148250K .......... .......... .......... .......... .......... 87% 39.0M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148300K .......... .......... .......... .......... .......... 88% 53.0M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148350K .......... .......... .......... .......... .......... 88% 18.7M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148400K .......... .......... .......... .......... .......... 88% 40.2M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148450K .......... .......... .......... .......... .......... 88% 27.5M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148500K .......... .......... .......... .......... .......... 88% 5.24M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148550K .......... .......... .......... .......... .......... 88% 108M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148600K .......... .......... .......... .......... .......... 88% 35.4M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148650K .......... .......... .......... .......... .......... 88% 37.5M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148700K .......... .......... .......... .......... .......... 88% 48.5M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148750K .......... .......... .......... .......... .......... 88% 47.1M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148800K .......... .......... .......... .......... .......... 88% 93.7M 1s 2024-03-30T13:11:19.744 INFO:teuthology.orchestra.run.smithi066.stderr:148850K .......... .......... .......... .......... .......... 88% 32.8M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:148900K .......... .......... .......... .......... .......... 88% 74.8M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:148950K .......... .......... .......... .......... .......... 88% 104M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149000K .......... .......... .......... .......... .......... 88% 95.6M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149050K .......... .......... .......... .......... .......... 88% 14.9M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149100K .......... .......... .......... .......... .......... 88% 23.1M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149150K .......... .......... .......... .......... .......... 88% 28.4M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149200K .......... .......... .......... .......... .......... 88% 81.6M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149250K .......... .......... .......... .......... .......... 88% 66.4M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149300K .......... .......... .......... .......... .......... 88% 69.7M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149350K .......... .......... .......... .......... .......... 88% 86.4M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149400K .......... .......... .......... .......... .......... 88% 56.9M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149450K .......... .......... .......... .......... .......... 88% 90.4M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149500K .......... .......... .......... .......... .......... 88% 100M 1s 2024-03-30T13:11:19.745 INFO:teuthology.orchestra.run.smithi066.stderr:149550K .......... .......... .......... .......... .......... 88% 60.0M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149600K .......... .......... .......... .......... .......... 88% 56.0M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149650K .......... .......... .......... .......... .......... 88% 37.1M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149700K .......... .......... .......... .......... .......... 88% 34.0M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149750K .......... .......... .......... .......... .......... 88% 22.5M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149800K .......... .......... .......... .......... .......... 88% 20.4M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149850K .......... .......... .......... .......... .......... 88% 13.3M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149900K .......... .......... .......... .......... .......... 88% 3.51M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:149950K .......... .......... .......... .......... .......... 88% 18.0M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150000K .......... .......... .......... .......... .......... 89% 30.1M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150050K .......... .......... .......... .......... .......... 89% 104M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150100K .......... .......... .......... .......... .......... 89% 108M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150150K .......... .......... .......... .......... .......... 89% 35.3M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150200K .......... .......... .......... .......... .......... 89% 17.3M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150250K .......... .......... .......... .......... .......... 89% 21.0M 1s 2024-03-30T13:11:19.746 INFO:teuthology.orchestra.run.smithi066.stderr:150300K .......... .......... .......... .......... .......... 89% 49.3M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150350K .......... .......... .......... .......... .......... 89% 66.2M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150400K .......... .......... .......... .......... .......... 89% 42.6M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150450K .......... .......... .......... .......... .......... 89% 46.7M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150500K .......... .......... .......... .......... .......... 89% 32.6M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150550K .......... .......... .......... .......... .......... 89% 29.7M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150600K .......... .......... .......... .......... .......... 89% 47.9M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150650K .......... .......... .......... .......... .......... 89% 34.6M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150700K .......... .......... .......... .......... .......... 89% 108M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150750K .......... .......... .......... .......... .......... 89% 94.2M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150800K .......... .......... .......... .......... .......... 89% 60.1M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150850K .......... .......... .......... .......... .......... 89% 99.2M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150900K .......... .......... .......... .......... .......... 89% 50.2M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:150950K .......... .......... .......... .......... .......... 89% 104M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:151000K .......... .......... .......... .......... .......... 89% 62.1M 1s 2024-03-30T13:11:19.747 INFO:teuthology.orchestra.run.smithi066.stderr:151050K .......... .......... .......... .......... .......... 89% 24.8M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151100K .......... .......... .......... .......... .......... 89% 11.2M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151150K .......... .......... .......... .......... .......... 89% 108M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151200K .......... .......... .......... .......... .......... 89% 141M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151250K .......... .......... .......... .......... .......... 89% 89.4M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151300K .......... .......... .......... .......... .......... 89% 15.1M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151350K .......... .......... .......... .......... .......... 89% 9.56M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151400K .......... .......... .......... .......... .......... 89% 5.15M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151450K .......... .......... .......... .......... .......... 89% 9.85M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151500K .......... .......... .......... .......... .......... 89% 53.6M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151550K .......... .......... .......... .......... .......... 89% 51.0M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151600K .......... .......... .......... .......... .......... 89% 68.3M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151650K .......... .......... .......... .......... .......... 89% 47.3M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151700K .......... .......... .......... .......... .......... 90% 95.3M 1s 2024-03-30T13:11:19.748 INFO:teuthology.orchestra.run.smithi066.stderr:151750K .......... .......... .......... .......... .......... 90% 22.7M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:151800K .......... .......... .......... .......... .......... 90% 37.0M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:151850K .......... .......... .......... .......... .......... 90% 44.2M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:151900K .......... .......... .......... .......... .......... 90% 44.0M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:151950K .......... .......... .......... .......... .......... 90% 45.5M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152000K .......... .......... .......... .......... .......... 90% 91.2M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152050K .......... .......... .......... .......... .......... 90% 69.5M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152100K .......... .......... .......... .......... .......... 90% 56.5M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152150K .......... .......... .......... .......... .......... 90% 66.6M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152200K .......... .......... .......... .......... .......... 90% 56.4M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152250K .......... .......... .......... .......... .......... 90% 76.1M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152300K .......... .......... .......... .......... .......... 90% 55.0M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152350K .......... .......... .......... .......... .......... 90% 103M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152400K .......... .......... .......... .......... .......... 90% 51.9M 1s 2024-03-30T13:11:19.749 INFO:teuthology.orchestra.run.smithi066.stderr:152450K .......... .......... .......... .......... .......... 90% 48.3M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152500K .......... .......... .......... .......... .......... 90% 104M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152550K .......... .......... .......... .......... .......... 90% 91.6M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152600K .......... .......... .......... .......... .......... 90% 83.7M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152650K .......... .......... .......... .......... .......... 90% 8.99M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152700K .......... .......... .......... .......... .......... 90% 31.4M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152750K .......... .......... .......... .......... .......... 90% 28.7M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152800K .......... .......... .......... .......... .......... 90% 4.06M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152850K .......... .......... .......... .......... .......... 90% 27.0M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152900K .......... .......... .......... .......... .......... 90% 24.1M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:152950K .......... .......... .......... .......... .......... 90% 41.7M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:153000K .......... .......... .......... .......... .......... 90% 42.1M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:153050K .......... .......... .......... .......... .......... 90% 66.0M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:153100K .......... .......... .......... .......... .......... 90% 69.1M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:153150K .......... .......... .......... .......... .......... 90% 25.7M 1s 2024-03-30T13:11:19.750 INFO:teuthology.orchestra.run.smithi066.stderr:153200K .......... .......... .......... .......... .......... 90% 43.9M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153250K .......... .......... .......... .......... .......... 90% 97.1M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153300K .......... .......... .......... .......... .......... 90% 105M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153350K .......... .......... .......... .......... .......... 91% 71.7M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153400K .......... .......... .......... .......... .......... 91% 54.9M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153450K .......... .......... .......... .......... .......... 91% 104M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153500K .......... .......... .......... .......... .......... 91% 78.6M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153550K .......... .......... .......... .......... .......... 91% 11.2M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153600K .......... .......... .......... .......... .......... 91% 55.8M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153650K .......... .......... .......... .......... .......... 91% 36.8M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153700K .......... .......... .......... .......... .......... 91% 55.0M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153750K .......... .......... .......... .......... .......... 91% 75.1M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153800K .......... .......... .......... .......... .......... 91% 98.6M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153850K .......... .......... .......... .......... .......... 91% 110M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153900K .......... .......... .......... .......... .......... 91% 47.2M 1s 2024-03-30T13:11:19.751 INFO:teuthology.orchestra.run.smithi066.stderr:153950K .......... .......... .......... .......... .......... 91% 56.8M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154000K .......... .......... .......... .......... .......... 91% 44.3M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154050K .......... .......... .......... .......... .......... 91% 9.79M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154100K .......... .......... .......... .......... .......... 91% 8.31M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154150K .......... .......... .......... .......... .......... 91% 10.2M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154200K .......... .......... .......... .......... .......... 91% 51.9M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154250K .......... .......... .......... .......... .......... 91% 27.3M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154300K .......... .......... .......... .......... .......... 91% 28.3M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154350K .......... .......... .......... .......... .......... 91% 29.0M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154400K .......... .......... .......... .......... .......... 91% 76.2M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154450K .......... .......... .......... .......... .......... 91% 35.9M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154500K .......... .......... .......... .......... .......... 91% 38.3M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154550K .......... .......... .......... .......... .......... 91% 86.6M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154600K .......... .......... .......... .......... .......... 91% 81.7M 1s 2024-03-30T13:11:19.752 INFO:teuthology.orchestra.run.smithi066.stderr:154650K .......... .......... .......... .......... .......... 91% 105M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:154700K .......... .......... .......... .......... .......... 91% 95.2M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:154750K .......... .......... .......... .......... .......... 91% 10.8M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:154800K .......... .......... .......... .......... .......... 91% 23.2M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:154850K .......... .......... .......... .......... .......... 91% 23.5M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:154900K .......... .......... .......... .......... .......... 91% 86.2M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:154950K .......... .......... .......... .......... .......... 91% 54.6M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155000K .......... .......... .......... .......... .......... 91% 99.8M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155050K .......... .......... .......... .......... .......... 92% 67.1M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155100K .......... .......... .......... .......... .......... 92% 62.5M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155150K .......... .......... .......... .......... .......... 92% 51.1M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155200K .......... .......... .......... .......... .......... 92% 33.9M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155250K .......... .......... .......... .......... .......... 92% 51.8M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155300K .......... .......... .......... .......... .......... 92% 34.1M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155350K .......... .......... .......... .......... .......... 92% 23.8M 1s 2024-03-30T13:11:19.753 INFO:teuthology.orchestra.run.smithi066.stderr:155400K .......... .......... .......... .......... .......... 92% 46.9M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155450K .......... .......... .......... .......... .......... 92% 33.7M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155500K .......... .......... .......... .......... .......... 92% 6.23M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155550K .......... .......... .......... .......... .......... 92% 4.88M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155600K .......... .......... .......... .......... .......... 92% 105M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155650K .......... .......... .......... .......... .......... 92% 106M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155700K .......... .......... .......... .......... .......... 92% 43.1M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155750K .......... .......... .......... .......... .......... 92% 33.1M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155800K .......... .......... .......... .......... .......... 92% 10.6M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155850K .......... .......... .......... .......... .......... 92% 47.7M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155900K .......... .......... .......... .......... .......... 92% 78.4M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:155950K .......... .......... .......... .......... .......... 92% 42.2M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:156000K .......... .......... .......... .......... .......... 92% 36.8M 1s 2024-03-30T13:11:19.754 INFO:teuthology.orchestra.run.smithi066.stderr:156050K .......... .......... .......... .......... .......... 92% 35.3M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156100K .......... .......... .......... .......... .......... 92% 48.9M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156150K .......... .......... .......... .......... .......... 92% 106M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156200K .......... .......... .......... .......... .......... 92% 113M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156250K .......... .......... .......... .......... .......... 92% 68.4M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156300K .......... .......... .......... .......... .......... 92% 89.6M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156350K .......... .......... .......... .......... .......... 92% 52.3M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156400K .......... .......... .......... .......... .......... 92% 104M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156450K .......... .......... .......... .......... .......... 92% 115M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156500K .......... .......... .......... .......... .......... 92% 82.7M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156550K .......... .......... .......... .......... .......... 92% 19.5M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156600K .......... .......... .......... .......... .......... 92% 33.1M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156650K .......... .......... .......... .......... .......... 92% 36.3M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156700K .......... .......... .......... .......... .......... 92% 29.8M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156750K .......... .......... .......... .......... .......... 93% 73.0M 1s 2024-03-30T13:11:19.755 INFO:teuthology.orchestra.run.smithi066.stderr:156800K .......... .......... .......... .......... .......... 93% 84.8M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:156850K .......... .......... .......... .......... .......... 93% 8.50M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:156900K .......... .......... .......... .......... .......... 93% 9.56M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:156950K .......... .......... .......... .......... .......... 93% 3.02M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157000K .......... .......... .......... .......... .......... 93% 22.2M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157050K .......... .......... .......... .......... .......... 93% 18.7M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157100K .......... .......... .......... .......... .......... 93% 23.5M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157150K .......... .......... .......... .......... .......... 93% 100M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157200K .......... .......... .......... .......... .......... 93% 47.0M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157250K .......... .......... .......... .......... .......... 93% 88.8M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157300K .......... .......... .......... .......... .......... 93% 46.0M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157350K .......... .......... .......... .......... .......... 93% 34.2M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157400K .......... .......... .......... .......... .......... 93% 30.5M 1s 2024-03-30T13:11:19.756 INFO:teuthology.orchestra.run.smithi066.stderr:157450K .......... .......... .......... .......... .......... 93% 36.2M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157500K .......... .......... .......... .......... .......... 93% 30.6M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157550K .......... .......... .......... .......... .......... 93% 53.2M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157600K .......... .......... .......... .......... .......... 93% 90.1M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157650K .......... .......... .......... .......... .......... 93% 80.9M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157700K .......... .......... .......... .......... .......... 93% 74.0M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157750K .......... .......... .......... .......... .......... 93% 53.1M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157800K .......... .......... .......... .......... .......... 93% 57.1M 1s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157850K .......... .......... .......... .......... .......... 93% 83.5M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157900K .......... .......... .......... .......... .......... 93% 104M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:157950K .......... .......... .......... .......... .......... 93% 87.4M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:158000K .......... .......... .......... .......... .......... 93% 104M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:158050K .......... .......... .......... .......... .......... 93% 70.0M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:158100K .......... .......... .......... .......... .......... 93% 42.4M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:158150K .......... .......... .......... .......... .......... 93% 75.5M 0s 2024-03-30T13:11:19.757 INFO:teuthology.orchestra.run.smithi066.stderr:158200K .......... .......... .......... .......... .......... 93% 11.0M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158250K .......... .......... .......... .......... .......... 93% 17.4M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158300K .......... .......... .......... .......... .......... 93% 30.8M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158350K .......... .......... .......... .......... .......... 93% 32.2M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158400K .......... .......... .......... .......... .......... 93% 50.2M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158450K .......... .......... .......... .......... .......... 94% 45.6M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158500K .......... .......... .......... .......... .......... 94% 77.9M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158550K .......... .......... .......... .......... .......... 94% 44.8M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158600K .......... .......... .......... .......... .......... 94% 18.7M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158650K .......... .......... .......... .......... .......... 94% 8.56M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158700K .......... .......... .......... .......... .......... 94% 12.8M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158750K .......... .......... .......... .......... .......... 94% 54.0M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158800K .......... .......... .......... .......... .......... 94% 51.9M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158850K .......... .......... .......... .......... .......... 94% 72.6M 0s 2024-03-30T13:11:19.758 INFO:teuthology.orchestra.run.smithi066.stderr:158900K .......... .......... .......... .......... .......... 94% 109M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:158950K .......... .......... .......... .......... .......... 94% 82.5M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159000K .......... .......... .......... .......... .......... 94% 92.0M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159050K .......... .......... .......... .......... .......... 94% 69.5M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159100K .......... .......... .......... .......... .......... 94% 104M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159150K .......... .......... .......... .......... .......... 94% 102M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159200K .......... .......... .......... .......... .......... 94% 93.1M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159250K .......... .......... .......... .......... .......... 94% 96.5M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159300K .......... .......... .......... .......... .......... 94% 108M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159350K .......... .......... .......... .......... .......... 94% 106M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159400K .......... .......... .......... .......... .......... 94% 111M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159450K .......... .......... .......... .......... .......... 94% 26.1M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159500K .......... .......... .......... .......... .......... 94% 11.7M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159550K .......... .......... .......... .......... .......... 94% 38.4M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159600K .......... .......... .......... .......... .......... 94% 23.3M 0s 2024-03-30T13:11:19.759 INFO:teuthology.orchestra.run.smithi066.stderr:159650K .......... .......... .......... .......... .......... 94% 30.3M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:159700K .......... .......... .......... .......... .......... 94% 30.0M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:159750K .......... .......... .......... .......... .......... 94% 7.42M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:159800K .......... .......... .......... .......... .......... 94% 6.66M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:159850K .......... .......... .......... .......... .......... 94% 5.44M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:159900K .......... .......... .......... .......... .......... 94% 32.2M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:159950K .......... .......... .......... .......... .......... 94% 49.3M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160000K .......... .......... .......... .......... .......... 94% 24.4M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160050K .......... .......... .......... .......... .......... 94% 29.5M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160100K .......... .......... .......... .......... .......... 95% 60.9M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160150K .......... .......... .......... .......... .......... 95% 97.2M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160200K .......... .......... .......... .......... .......... 95% 106M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160250K .......... .......... .......... .......... .......... 95% 79.3M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160300K .......... .......... .......... .......... .......... 95% 106M 0s 2024-03-30T13:11:19.760 INFO:teuthology.orchestra.run.smithi066.stderr:160350K .......... .......... .......... .......... .......... 95% 95.8M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160400K .......... .......... .......... .......... .......... 95% 101M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160450K .......... .......... .......... .......... .......... 95% 23.8M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160500K .......... .......... .......... .......... .......... 95% 12.1M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160550K .......... .......... .......... .......... .......... 95% 31.2M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160600K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160650K .......... .......... .......... .......... .......... 95% 27.1M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160700K .......... .......... .......... .......... .......... 95% 47.5M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160750K .......... .......... .......... .......... .......... 95% 23.6M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160800K .......... .......... .......... .......... .......... 95% 43.0M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160850K .......... .......... .......... .......... .......... 95% 44.6M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160900K .......... .......... .......... .......... .......... 95% 86.7M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:160950K .......... .......... .......... .......... .......... 95% 92.4M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:161000K .......... .......... .......... .......... .......... 95% 41.4M 0s 2024-03-30T13:11:19.761 INFO:teuthology.orchestra.run.smithi066.stderr:161050K .......... .......... .......... .......... .......... 95% 38.3M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161100K .......... .......... .......... .......... .......... 95% 64.6M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161150K .......... .......... .......... .......... .......... 95% 53.3M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161200K .......... .......... .......... .......... .......... 95% 95.1M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161250K .......... .......... .......... .......... .......... 95% 79.5M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161300K .......... .......... .......... .......... .......... 95% 45.5M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161350K .......... .......... .......... .......... .......... 95% 96.1M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161400K .......... .......... .......... .......... .......... 95% 83.5M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161450K .......... .......... .......... .......... .......... 95% 106M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161500K .......... .......... .......... .......... .......... 95% 99.7M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161550K .......... .......... .......... .......... .......... 95% 17.4M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161600K .......... .......... .......... .......... .......... 95% 14.3M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161650K .......... .......... .......... .......... .......... 95% 6.99M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161700K .......... .......... .......... .......... .......... 95% 8.26M 0s 2024-03-30T13:11:19.762 INFO:teuthology.orchestra.run.smithi066.stderr:161750K .......... .......... .......... .......... .......... 95% 7.29M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:161800K .......... .......... .......... .......... .......... 96% 29.6M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:161850K .......... .......... .......... .......... .......... 96% 52.2M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:161900K .......... .......... .......... .......... .......... 96% 61.9M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:161950K .......... .......... .......... .......... .......... 96% 90.4M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162000K .......... .......... .......... .......... .......... 96% 64.7M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162050K .......... .......... .......... .......... .......... 96% 70.7M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162100K .......... .......... .......... .......... .......... 96% 48.1M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162150K .......... .......... .......... .......... .......... 96% 61.7M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162200K .......... .......... .......... .......... .......... 96% 109M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162250K .......... .......... .......... .......... .......... 96% 69.0M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162300K .......... .......... .......... .......... .......... 96% 94.4M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162350K .......... .......... .......... .......... .......... 96% 60.6M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162400K .......... .......... .......... .......... .......... 96% 96.2M 0s 2024-03-30T13:11:19.763 INFO:teuthology.orchestra.run.smithi066.stderr:162450K .......... .......... .......... .......... .......... 96% 93.3M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162500K .......... .......... .......... .......... .......... 96% 30.6M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162550K .......... .......... .......... .......... .......... 96% 42.5M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162600K .......... .......... .......... .......... .......... 96% 30.0M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162650K .......... .......... .......... .......... .......... 96% 60.5M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162700K .......... .......... .......... .......... .......... 96% 60.3M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162750K .......... .......... .......... .......... .......... 96% 34.5M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162800K .......... .......... .......... .......... .......... 96% 6.18M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162850K .......... .......... .......... .......... .......... 96% 5.61M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162900K .......... .......... .......... .......... .......... 96% 8.39M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:162950K .......... .......... .......... .......... .......... 96% 49.9M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:163000K .......... .......... .......... .......... .......... 96% 23.0M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:163050K .......... .......... .......... .......... .......... 96% 17.8M 0s 2024-03-30T13:11:19.764 INFO:teuthology.orchestra.run.smithi066.stderr:163100K .......... .......... .......... .......... .......... 96% 32.5M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163150K .......... .......... .......... .......... .......... 96% 23.0M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163200K .......... .......... .......... .......... .......... 96% 45.1M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163250K .......... .......... .......... .......... .......... 96% 30.9M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163300K .......... .......... .......... .......... .......... 96% 60.4M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163350K .......... .......... .......... .......... .......... 96% 104M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163400K .......... .......... .......... .......... .......... 96% 89.2M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163450K .......... .......... .......... .......... .......... 96% 108M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163500K .......... .......... .......... .......... .......... 97% 100M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163550K .......... .......... .......... .......... .......... 97% 91.0M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163600K .......... .......... .......... .......... .......... 97% 103M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163650K .......... .......... .......... .......... .......... 97% 12.0M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163700K .......... .......... .......... .......... .......... 97% 24.1M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163750K .......... .......... .......... .......... .......... 97% 29.7M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163800K .......... .......... .......... .......... .......... 97% 47.6M 0s 2024-03-30T13:11:19.765 INFO:teuthology.orchestra.run.smithi066.stderr:163850K .......... .......... .......... .......... .......... 97% 88.4M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:163900K .......... .......... .......... .......... .......... 97% 79.0M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:163950K .......... .......... .......... .......... .......... 97% 115M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164000K .......... .......... .......... .......... .......... 97% 76.8M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164050K .......... .......... .......... .......... .......... 97% 30.6M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164100K .......... .......... .......... .......... .......... 97% 7.19M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164150K .......... .......... .......... .......... .......... 97% 45.4M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164200K .......... .......... .......... .......... .......... 97% 39.3M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164250K .......... .......... .......... .......... .......... 97% 38.6M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164300K .......... .......... .......... .......... .......... 97% 32.9M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164350K .......... .......... .......... .......... .......... 97% 25.4M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164400K .......... .......... .......... .......... .......... 97% 49.5M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164450K .......... .......... .......... .......... .......... 97% 111M 0s 2024-03-30T13:11:19.766 INFO:teuthology.orchestra.run.smithi066.stderr:164500K .......... .......... .......... .......... .......... 97% 48.7M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164550K .......... .......... .......... .......... .......... 97% 60.3M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164600K .......... .......... .......... .......... .......... 97% 71.9M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164650K .......... .......... .......... .......... .......... 97% 8.38M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164700K .......... .......... .......... .......... .......... 97% 11.8M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164750K .......... .......... .......... .......... .......... 97% 13.1M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164800K .......... .......... .......... .......... .......... 97% 64.1M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164850K .......... .......... .......... .......... .......... 97% 96.7M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164900K .......... .......... .......... .......... .......... 97% 105M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:164950K .......... .......... .......... .......... .......... 97% 82.6M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:165000K .......... .......... .......... .......... .......... 97% 101M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:165050K .......... .......... .......... .......... .......... 97% 98.7M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:165100K .......... .......... .......... .......... .......... 97% 110M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:165150K .......... .......... .......... .......... .......... 98% 59.7M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:165200K .......... .......... .......... .......... .......... 98% 74.7M 0s 2024-03-30T13:11:19.767 INFO:teuthology.orchestra.run.smithi066.stderr:165250K .......... .......... .......... .......... .......... 98% 95.8M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165300K .......... .......... .......... .......... .......... 98% 112M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165350K .......... .......... .......... .......... .......... 98% 102M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165400K .......... .......... .......... .......... .......... 98% 107M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165450K .......... .......... .......... .......... .......... 98% 12.7M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165500K .......... .......... .......... .......... .......... 98% 111M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165550K .......... .......... .......... .......... .......... 98% 97.0M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165600K .......... .......... .......... .......... .......... 98% 109M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165650K .......... .......... .......... .......... .......... 98% 103M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165700K .......... .......... .......... .......... .......... 98% 30.6M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165750K .......... .......... .......... .......... .......... 98% 22.9M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165800K .......... .......... .......... .......... .......... 98% 4.19M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165850K .......... .......... .......... .......... .......... 98% 30.9M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165900K .......... .......... .......... .......... .......... 98% 51.1M 0s 2024-03-30T13:11:19.768 INFO:teuthology.orchestra.run.smithi066.stderr:165950K .......... .......... .......... .......... .......... 98% 58.3M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166000K .......... .......... .......... .......... .......... 98% 61.4M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166050K .......... .......... .......... .......... .......... 98% 65.1M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166100K .......... .......... .......... .......... .......... 98% 47.6M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166150K .......... .......... .......... .......... .......... 98% 65.4M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166200K .......... .......... .......... .......... .......... 98% 107M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166250K .......... .......... .......... .......... .......... 98% 108M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166300K .......... .......... .......... .......... .......... 98% 101M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166350K .......... .......... .......... .......... .......... 98% 89.8M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166400K .......... .......... .......... .......... .......... 98% 68.3M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166450K .......... .......... .......... .......... .......... 98% 24.6M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166500K .......... .......... .......... .......... .......... 98% 41.3M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166550K .......... .......... .......... .......... .......... 98% 25.1M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166600K .......... .......... .......... .......... .......... 98% 40.3M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166650K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-03-30T13:11:19.769 INFO:teuthology.orchestra.run.smithi066.stderr:166700K .......... .......... .......... .......... .......... 98% 22.3M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:166750K .......... .......... .......... .......... .......... 98% 46.6M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:166800K .......... .......... .......... .......... .......... 98% 25.1M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:166850K .......... .......... .......... .......... .......... 99% 32.3M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:166900K .......... .......... .......... .......... .......... 99% 24.6M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:166950K .......... .......... .......... .......... .......... 99% 24.0M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167000K .......... .......... .......... .......... .......... 99% 60.7M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167050K .......... .......... .......... .......... .......... 99% 24.4M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167100K .......... .......... .......... .......... .......... 99% 28.6M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167150K .......... .......... .......... .......... .......... 99% 25.0M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167200K .......... .......... .......... .......... .......... 99% 13.5M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167250K .......... .......... .......... .......... .......... 99% 31.7M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167300K .......... .......... .......... .......... .......... 99% 3.78M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167350K .......... .......... .......... .......... .......... 99% 14.7M 0s 2024-03-30T13:11:19.770 INFO:teuthology.orchestra.run.smithi066.stderr:167400K .......... .......... .......... .......... .......... 99% 81.1M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167450K .......... .......... .......... .......... .......... 99% 66.2M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167500K .......... .......... .......... .......... .......... 99% 107M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167550K .......... .......... .......... .......... .......... 99% 108M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167600K .......... .......... .......... .......... .......... 99% 58.9M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167650K .......... .......... .......... .......... .......... 99% 34.0M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167700K .......... .......... .......... .......... .......... 99% 35.0M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167750K .......... .......... .......... .......... .......... 99% 108M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167800K .......... .......... .......... .......... .......... 99% 98.9M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167850K .......... .......... .......... .......... .......... 99% 110M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167900K .......... .......... .......... .......... .......... 99% 97.3M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:167950K .......... .......... .......... .......... .......... 99% 87.9M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:168000K .......... .......... .......... .......... .......... 99% 111M 0s 2024-03-30T13:11:19.771 INFO:teuthology.orchestra.run.smithi066.stderr:168050K .......... .......... .......... .......... .......... 99% 97.2M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168100K .......... .......... .......... .......... .......... 99% 66.4M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168150K .......... .......... .......... .......... .......... 99% 24.8M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168200K .......... .......... .......... .......... .......... 99% 23.6M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168250K .......... .......... .......... .......... .......... 99% 66.3M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168300K .......... .......... .......... .......... .......... 99% 24.4M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168350K .......... .......... .......... .......... .......... 99% 36.3M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168400K .......... .......... .......... .......... .......... 99% 23.9M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168450K .......... .......... .......... .......... .......... 99% 6.52M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168500K .......... .......... .......... .......... .......... 99% 8.64M 0s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:168550K .......... .......... . 100% 6.20M=7.8s 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30 13:11:18 (21.2 MB/s) - ‘linux-5.4.tar.gz’ saved [172616875/172616875] 2024-03-30T13:11:19.772 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:11:19.773 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-30T13:11:19.773 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-30T13:11:19.773 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 Processor -- start 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 -- start start 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8354072d50 0x7f83540b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f835415f800 0x7f83540ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835415fcb0 con 0x7f8354144cb0 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f83540b3e80 con 0x7f8354072d50 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.880+0000 7f835afba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f83540b35e0 con 0x7f835415f800 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8354072d50 0x7f83540b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8354072d50 0x7f83540b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53502/0 (socket says 172.21.15.66:53502) 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 -- 172.21.15.66:0/3626047482 learned_addr learned my addr 172.21.15.66:0/3626047482 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:19.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8358d56640 1 --1- 172.21.15.66:0/3626047482 >> v1:172.21.15.66:6789/0 conn(0x7f8354144cb0 0x7f8354072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44854/0 (socket says 172.21.15.66:44854) 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8359557640 1 --2- 172.21.15.66:0/3626047482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f835415f800 0x7f83540ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 -- 172.21.15.66:0/3626047482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f835415f800 msgr2=0x7f83540ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 --2- 172.21.15.66:0/3626047482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f835415f800 0x7f83540ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 -- 172.21.15.66:0/3626047482 >> v1:172.21.15.66:6789/0 conn(0x7f8354144cb0 legacy=0x7f8354072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.881+0000 7f8353fff640 1 -- 172.21.15.66:0/3626047482 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83540b38c0 con 0x7f8354072d50 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f8353fff640 1 --2- 172.21.15.66:0/3626047482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354072d50 0x7f83540b2f20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8348009870 tx=0x7f834802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5c891e6d0058d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f83537fe640 1 -- 172.21.15.66:0/3626047482 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348034070 con 0x7f8354072d50 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f83537fe640 1 -- 172.21.15.66:0/3626047482 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348030840 con 0x7f8354072d50 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f83537fe640 1 -- 172.21.15.66:0/3626047482 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348030c10 con 0x7f8354072d50 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f835afba640 1 -- 172.21.15.66:0/3626047482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354072d50 msgr2=0x7f83540b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f835afba640 1 --2- 172.21.15.66:0/3626047482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354072d50 0x7f83540b2f20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8348009870 tx=0x7f834802b980 comp rx=0 tx=0).stop 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.882+0000 7f835afba640 1 -- 172.21.15.66:0/3626047482 shutdown_connections 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.883+0000 7f835afba640 1 --2- 172.21.15.66:0/3626047482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f835415f800 0x7f83540ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.883+0000 7f835afba640 1 --2- 172.21.15.66:0/3626047482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8354072d50 0x7f83540b2f20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.883+0000 7f835afba640 1 -- 172.21.15.66:0/3626047482 >> 172.21.15.66:0/3626047482 conn(0x7f83540b5b00 msgr2=0x7f835406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.883+0000 7f835afba640 1 -- 172.21.15.66:0/3626047482 shutdown_connections 2024-03-30T13:11:19.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.883+0000 7f835afba640 1 -- 172.21.15.66:0/3626047482 wait complete. 2024-03-30T13:11:19.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.884+0000 7f835afba640 1 Processor -- start 2024-03-30T13:11:19.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.884+0000 7f835afba640 1 -- start start 2024-03-30T13:11:19.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f835afba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8354072d50 0x7f83541593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f835afba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354144cb0 0x7f835415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f835afba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 0x7f835413f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f835afba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83540ce590 con 0x7f835415f800 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f835afba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83540cdb30 con 0x7f8354144cb0 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f835afba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f83540ce710 con 0x7f8354072d50 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 0x7f835413f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 0x7f835413f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53504/0 (socket says 172.21.15.66:53504) 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 -- 172.21.15.66:0/1903221074 learned_addr learned my addr 172.21.15.66:0/1903221074 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8353fff640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354144cb0 0x7f835415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8358d56640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8354072d50 0x7f83541593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 -- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8354072d50 msgr2=0x7f83541593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8354072d50 0x7f83541593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 -- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354144cb0 msgr2=0x7f835415b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354144cb0 0x7f835415b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 -- 172.21.15.66:0/1903221074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835415b9d0 con 0x7f835415f800 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.886+0000 7f8359557640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 0x7f835413f550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f834c007e60 tx=0x7f834c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.887+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c014420 con 0x7f835415f800 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.887+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834c019070 con 0x7f835415f800 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.887+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835413e1d0 con 0x7f835415f800 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.887+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c014820 con 0x7f835415f800 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.887+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835413e790 con 0x7f835415f800 2024-03-30T13:11:19.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.888+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8354075890 con 0x7f835415f800 2024-03-30T13:11:19.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.888+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f834c0149c0 con 0x7f835415f800 2024-03-30T13:11:19.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.889+0000 7f83517fa640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f834006c530 0x7f834006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.889+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f834c08bf70 con 0x7f835415f800 2024-03-30T13:11:19.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.889+0000 7f8358d56640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f834006c530 0x7f834006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.889+0000 7f8358d56640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f834006c530 0x7f834006e9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8344007fd0 tx=0x7f8344007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:19.893+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f834c059db0 con 0x7f835415f800 2024-03-30T13:11:20.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.180+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f83540b4030 con 0x7f835415f800 2024-03-30T13:11:20.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.887+0000 7f83517fa640 1 -- 172.21.15.66:0/1903221074 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f834c05e0b0 con 0x7f835415f800 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f834006c530 msgr2=0x7f834006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f834006c530 0x7f834006e9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8344007fd0 tx=0x7f8344007e80 comp rx=0 tx=0).stop 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 msgr2=0x7f835413f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 0x7f835413f550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f834c007e60 tx=0x7f834c00e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 shutdown_connections 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f834006c530 0x7f834006e9f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835415f800 0x7f835413f550 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354144cb0 0x7f835415b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 --2- 172.21.15.66:0/1903221074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8354072d50 0x7f83541593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 >> 172.21.15.66:0/1903221074 conn(0x7f83540b5b00 msgr2=0x7f8354071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:20.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 shutdown_connections 2024-03-30T13:11:20.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:20.890+0000 7f835afba640 1 -- 172.21.15.66:0/1903221074 wait complete. 2024-03-30T13:11:20.904 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-30T13:11:20.904 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-30T13:11:20.904 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-30T13:11:20.905 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-30T13:11:25.275 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-30T13:11:25.275 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-30T13:11:25.309 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph 2024-03-30T13:11:25.310 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2 2024-03-30T13:11:25.310 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs 2024-03-30T13:11:25.310 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-30T13:11:25.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:25.310 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-30T13:11:25.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:25.365 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-30T13:11:25.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:25.421 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-30T13:11:25.477 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-30T13:11:25.477 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-30T13:11:25.545 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/Makefile 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/io.h 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/Makefile 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/acl.h 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:11:25.546 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:11:25.547 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:11:25.547 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-30T13:11:25.547 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-30T13:11:25.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:25.548 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-30T13:11:25.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.693+0000 7f9db4fba640 1 Processor -- start 2024-03-30T13:11:25.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.693+0000 7f9db4fba640 1 -- start start 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.694+0000 7f9db4fba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db00b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.694+0000 7f9db4fba640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9db006d4a0 0x7f9db00d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.694+0000 7f9db4fba640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db00d2910 con 0x7f9db00d3ef0 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.694+0000 7f9db4fba640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f9db00d3520 con 0x7f9db006d4a0 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.694+0000 7f9db4fba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9db00d2bf0 con 0x7f9db015e440 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dadd9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9db006d4a0 0x7f9db00d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9daed9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9db00d3ef0 0x7f9db00cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:34808/0 (socket says 172.21.15.66:34808) 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9daed9d640 1 -- 172.21.15.66:0/3937959285 learned_addr learned my addr 172.21.15.66:0/3937959285 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:25.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dae59c640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db00b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dad59a640 1 -- 172.21.15.66:0/3937959285 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1087976743 0 0) 0x7f9db00d2910 con 0x7f9db00d3ef0 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dad59a640 1 -- 172.21.15.66:0/3937959285 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d98003520 con 0x7f9db00d3ef0 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dadd9b640 1 -- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 msgr2=0x7f9db00b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db00b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dadd9b640 1 -- 172.21.15.66:0/3937959285 >> v1:172.21.15.117:6789/0 conn(0x7f9db00d3ef0 legacy=0x7f9db00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dadd9b640 1 -- 172.21.15.66:0/3937959285 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db00d2ed0 con 0x7f9db006d4a0 2024-03-30T13:11:25.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.695+0000 7f9dae59c640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db00b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:25.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.696+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00d39b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9da40097a0 tx=0x7f9da402b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe90fe3bc8108c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.696+0000 7f9dad59a640 1 -- 172.21.15.66:0/3937959285 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4034070 con 0x7f9db006d4a0 2024-03-30T13:11:25.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.696+0000 7f9dad59a640 1 -- 172.21.15.66:0/3937959285 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da4030800 con 0x7f9db006d4a0 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.696+0000 7f9dad59a640 1 -- 172.21.15.66:0/3937959285 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4030bd0 con 0x7f9db006d4a0 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 -- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 msgr2=0x7f9db00d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00d39b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9da40097a0 tx=0x7f9da402b940 comp rx=0 tx=0).stop 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 -- 172.21.15.66:0/3937959285 shutdown_connections 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00d39b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 --2- 172.21.15.66:0/3937959285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db00b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 -- 172.21.15.66:0/3937959285 >> 172.21.15.66:0/3937959285 conn(0x7f9db00676b0 msgr2=0x7f9db0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 -- 172.21.15.66:0/3937959285 shutdown_connections 2024-03-30T13:11:25.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.697+0000 7f9db4fba640 1 -- 172.21.15.66:0/3937959285 wait complete. 2024-03-30T13:11:25.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.699+0000 7f9db4fba640 1 Processor -- start 2024-03-30T13:11:25.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.699+0000 7f9db4fba640 1 -- start start 2024-03-30T13:11:25.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.700+0000 7f9db4fba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.700+0000 7f9db4fba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db01530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.700+0000 7f9db4fba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 0x7f9db0151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9db4fba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9db015e820 con 0x7f9db015c440 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9db4fba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00ba220 con 0x7f9db006d4a0 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9db4fba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9db015e9a0 con 0x7f9db015e440 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 0x7f9db0151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 0x7f9db0151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51562/0 (socket says 172.21.15.66:51562) 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 -- 172.21.15.66:0/2670744937 learned_addr learned my addr 172.21.15.66:0/2670744937 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db01530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 -- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 msgr2=0x7f9db01530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db01530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 -- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 msgr2=0x7f9db00c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 -- 172.21.15.66:0/2670744937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db0153800 con 0x7f9db015c440 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 0x7f9db0151b40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9da800cae0 tx=0x7f9da800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9d9effd640 1 -- 172.21.15.66:0/2670744937 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8017070 con 0x7f9db015c440 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9d9effd640 1 -- 172.21.15.66:0/2670744937 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da800ac40 con 0x7f9db015c440 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db0153ae0 con 0x7f9db015c440 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9d9effd640 1 -- 172.21.15.66:0/2670744937 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8012640 con 0x7f9db015c440 2024-03-30T13:11:25.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.701+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db01540a0 con 0x7f9db015c440 2024-03-30T13:11:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.703+0000 7f9d9effd640 1 -- 172.21.15.66:0/2670744937 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9da801a070 con 0x7f9db015c440 2024-03-30T13:11:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.703+0000 7f9d9effd640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9d9006c560 0x7f9d9006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.704+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9d9006c560 0x7f9d9006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.704+0000 7f9d9effd640 1 -- 172.21.15.66:0/2670744937 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9da808b5c0 con 0x7f9db015c440 2024-03-30T13:11:25.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.704+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9d9006c560 0x7f9d9006ea20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009560 tx=0x7f9da0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.704+0000 7f9db7ba2740 1 Processor -- start 2024-03-30T13:11:25.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 -- start start 2024-03-30T13:11:25.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1deedac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df1d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1deee000 0x557a1df1ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557a1de57410 con 0x557a1df1ab20 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a1ddf5a80 con 0x557a1df1af00 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x557a1de533c0 con 0x557a1deee000 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9dae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1deedac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9dae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1deedac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51570/0 (socket says 172.21.15.66:51570) 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9dae59c640 1 -- 172.21.15.66:0/3597019088 learned_addr learned my addr 172.21.15.66:0/3597019088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df1d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.705+0000 7f9daed9d640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1deee000 0x557a1df1ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dae59c640 1 -- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1deee000 msgr2=0x557a1df1ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dae59c640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1deee000 0x557a1df1ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dae59c640 1 -- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 msgr2=0x557a1df1d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dae59c640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df1d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dae59c640 1 -- 172.21.15.66:0/3597019088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1dc377b0 con 0x557a1df1ab20 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df1d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9daed9d640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1deee000 0x557a1df1ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9dae59c640 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1deedac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9da0011df0 tx=0x7f9da0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2f2038b71e9738 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9d8ffff640 1 -- 172.21.15.66:0/3597019088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0014c90 con 0x557a1df1ab20 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9d8ffff640 1 -- 172.21.15.66:0/3597019088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da001b070 con 0x557a1df1ab20 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.706+0000 7f9d8ffff640 1 -- 172.21.15.66:0/3597019088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0004230 con 0x557a1df1ab20 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 -- 172.21.15.66:0/3597019088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 msgr2=0x557a1deedac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1deedac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9da0011df0 tx=0x7f9da0014000 comp rx=0 tx=0).stop 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 -- 172.21.15.66:0/3597019088 shutdown_connections 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1deee000 0x557a1df1ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df1d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/3597019088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1deedac0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 -- 172.21.15.66:0/3597019088 >> 172.21.15.66:0/3597019088 conn(0x557a1deefd30 msgr2=0x557a1df388b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 -- 172.21.15.66:0/3597019088 shutdown_connections 2024-03-30T13:11:25.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 -- 172.21.15.66:0/3597019088 wait complete. 2024-03-30T13:11:25.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 -- start start 2024-03-30T13:11:25.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.707+0000 7f9db7ba2740 1 Processor -- start 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1df21420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df4c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557a1def23d0 con 0x557a1df1ab20 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a1ddf3740 con 0x557a1df1af00 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x557a1def4c10 con 0x557a1df4cab0 2024-03-30T13:11:25.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9db7ba2740 10 client.0 _ll_register_callbacks cb 0x557a1de00b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9dae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1df21420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9dae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1df21420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51574/0 (socket says 172.21.15.66:51574) 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.708+0000 7f9dae59c640 1 -- 172.21.15.66:0/1852549783 learned_addr learned my addr 172.21.15.66:0/1852549783 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df4c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9daed9d640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df4cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dae59c640 1 -- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 msgr2=0x557a1df4cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dae59c640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df4cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dae59c640 1 -- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 msgr2=0x557a1df4c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dae59c640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df4c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dae59c640 1 -- 172.21.15.66:0/1852549783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1dc97f40 con 0x557a1df1ab20 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9dae59c640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1df21420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009890 tx=0x7f9da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ce33283bc8ecaa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9d8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:25.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/1852549783 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0037060 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/1852549783 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9da0003610 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da001b070 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/1852549783 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9da0005990 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9db7ba2740 10 client.5628 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.709+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557a1db2b0a0 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/1852549783 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da001b070 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/1852549783 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f9da0040080 con 0x557a1df1ab20 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9db7ba2740 10 client.5628 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:25.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9db7ba2740 10 client.5628 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9db7ba2740 20 client.5628 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df4a680 0x557a1df4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9db7ba2740 4 client.5628 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1de64950 con 0x557a1df4a680 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df4a680 0x557a1df4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9d8d7fa640 10 client.5628 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.710+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df4a680 0x557a1df4aad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef9d488cbfaf29f server_cookie=58151f8c45416510 in_seq=0 out_seq=0 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/1852549783 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x557a1de64950 con 0x557a1df4a680 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9d8d7fa640 10 client.5628 handle_command_reply: tid=0 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 client.5628 shutdown 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 msgr2=0x557a1df21420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1df21420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009890 tx=0x7f9da0004730 comp rx=0 tx=0).stop 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 shutdown_connections 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df4a680 0x557a1df4aad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1af00 0x557a1df4c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/1852549783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1df21420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9d8d7fa640 0 client.5628 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 >> 172.21.15.66:0/1852549783 conn(0x557a1deee000 msgr2=0x557a1def1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.711+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 shutdown_connections 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.712+0000 7f9db7ba2740 1 -- 172.21.15.66:0/1852549783 wait complete. 2024-03-30T13:11:25.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.712+0000 7f9db7ba2740 20 client.5628 trim_cache size 0 max 16384 2024-03-30T13:11:25.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 Processor -- start 2024-03-30T13:11:25.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 -- start start 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1def3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df1af00 0x557a1df38a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df4cab0 0x557a1df38f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9db02010d0 con 0x557a1df1ab20 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9db0200130 con 0x557a1df4cab0 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9db00cbe20 con 0x557a1df1af00 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9dae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1def3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9dae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1def3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51588/0 (socket says 172.21.15.66:51588) 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.730+0000 7f9dae59c640 1 -- 172.21.15.66:0/2335558931 learned_addr learned my addr 172.21.15.66:0/2335558931 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9dae59c640 1 -- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df1af00 msgr2=0x557a1df38a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df4cab0 0x557a1df38f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df1af00 0x557a1df38a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9dae59c640 1 -- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df4cab0 msgr2=0x557a1df38f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df4cab0 0x557a1df38f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9dae59c640 1 -- 172.21.15.66:0/2335558931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1df22860 con 0x557a1df1ab20 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df4cab0 0x557a1df38f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1def3ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9da0011710 tx=0x7f9da002b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01e11322b803fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9d8f7fe640 1 -- 172.21.15.66:0/2335558931 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0006c10 con 0x557a1df1ab20 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9d8f7fe640 1 -- 172.21.15.66:0/2335558931 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0043070 con 0x557a1df1ab20 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9d8f7fe640 1 -- 172.21.15.66:0/2335558931 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0003850 con 0x557a1df1ab20 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2335558931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 msgr2=0x557a1def3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.731+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1def3ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9da0011710 tx=0x7f9da002b050 comp rx=0 tx=0).stop 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2335558931 shutdown_connections 2024-03-30T13:11:25.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df4cab0 0x557a1df38f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df1af00 0x557a1df38a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2335558931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1ab20 0x557a1def3ae0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2335558931 >> 172.21.15.66:0/2335558931 conn(0x557a1deee000 msgr2=0x557a1df4c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2335558931 shutdown_connections 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2335558931 wait complete. 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 -- start start 2024-03-30T13:11:25.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.732+0000 7f9db7ba2740 1 Processor -- start 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1ab20 0x557a1dfb4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 0x557a1df3b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df3bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00cc470 con 0x557a1df1af00 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a1df18aa0 con 0x557a1df1ab20 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x557a1df46ff0 con 0x557a1df4cab0 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9db7ba2740 10 client.0 _ll_register_callbacks cb 0x7f9da0011cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:25.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9dadd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 0x557a1df3b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9dadd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 0x557a1df3b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51590/0 (socket says 172.21.15.66:51590) 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9dadd9b640 1 -- 172.21.15.66:0/2260294689 learned_addr learned my addr 172.21.15.66:0/2260294689 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1ab20 0x557a1dfb4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df3bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9dadd9b640 1 -- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 msgr2=0x557a1df3bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.733+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df3bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9dadd9b640 1 -- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1ab20 msgr2=0x557a1dfb4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1ab20 0x557a1dfb4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9dadd9b640 1 -- 172.21.15.66:0/2260294689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1df53230 con 0x557a1df1af00 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1ab20 0x557a1dfb4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9daed9d640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df3bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9dadd9b640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 0x557a1df3b5c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000c00 tx=0x7f9da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63a271ceed5ed7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9d8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:25.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/2260294689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4034040 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/2260294689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9da404fc50 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da0043070 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/2260294689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9da404fe50 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 10 client.5634 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x557a1df47f80 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/2260294689 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0043070 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/2260294689 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f9da404d030 con 0x557a1df1af00 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 10 client.5634 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 10 client.5634 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 20 client.5634 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df190c0 0x557a1df194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 4 client.5634 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.734+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x557a1de5eca0 con 0x557a1df190c0 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.735+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df190c0 0x557a1df194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.735+0000 7f9dae59c640 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df190c0 0x557a1df194a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a32ede6e2a2d45 server_cookie=1c873a99f2141d6e in_seq=0 out_seq=0 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.735+0000 7f9d8d7fa640 10 client.5634 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9d8d7fa640 1 -- 172.21.15.66:0/2260294689 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a1de5eca0 con 0x557a1df190c0 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9d8d7fa640 10 client.5634 handle_command_reply: tid=0 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 client.5634 shutdown 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 msgr2=0x557a1df3b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 0x557a1df3b5c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000c00 tx=0x7f9da40046c0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 shutdown_connections 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557a1df190c0 0x557a1df194a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557a1df4cab0 0x557a1df3bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557a1df1af00 0x557a1df3b5c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9d8d7fa640 0 client.5634 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 --2- 172.21.15.66:0/2260294689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a1df1ab20 0x557a1dfb4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.736+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 >> 172.21.15.66:0/2260294689 conn(0x557a1deee000 msgr2=0x557a1df38930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 shutdown_connections 2024-03-30T13:11:25.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db7ba2740 1 -- 172.21.15.66:0/2260294689 wait complete. 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db7ba2740 20 client.5634 trim_cache size 0 max 16384 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "c7ed1273-d477-4170-a25d-5b1410281d2c", 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:25.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9d9006c560 msgr2=0x7f9d9006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9d9006c560 0x7f9d9006ea20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009560 tx=0x7f9da0009190 comp rx=0 tx=0).stop 2024-03-30T13:11:25.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 msgr2=0x7f9db0151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 0x7f9db0151b40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9da800cae0 tx=0x7f9da800a060 comp rx=0 tx=0).stop 2024-03-30T13:11:25.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 shutdown_connections 2024-03-30T13:11:25.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9d9006c560 0x7f9d9006ea20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015c440 0x7f9db0151b40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9db015e440 0x7f9db01530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 --2- 172.21.15.66:0/2670744937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9db006d4a0 0x7f9db00c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.737+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 >> 172.21.15.66:0/2670744937 conn(0x7f9db00676b0 msgr2=0x7f9db00b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.738+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 shutdown_connections 2024-03-30T13:11:25.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.738+0000 7f9db4fba640 1 -- 172.21.15.66:0/2670744937 wait complete. 2024-03-30T13:11:25.756 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': 'c7ed1273-d477-4170-a25d-5b1410281d2c', 'mode': 'asynchronous'} 2024-03-30T13:11:25.756 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-30T13:11:25.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:25.756 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:11:25.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.910+0000 7f74ec063640 1 Processor -- start 2024-03-30T13:11:25.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.910+0000 7f74ec063640 1 -- start start 2024-03-30T13:11:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.911+0000 7f74ec063640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74e415e4d0 0x7f74e4162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.911+0000 7f74ec063640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74e40d3e00 0x7f74e40b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74ec063640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e40cf650 con 0x7f74e40d1350 2024-03-30T13:11:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74ec063640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f74e40d29e0 con 0x7f74e40d3e00 2024-03-30T13:11:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74ec063640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f74e40cf930 con 0x7f74e415e4d0 2024-03-30T13:11:25.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e95fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74e40d3e00 0x7f74e40b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e9dff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74e415e4d0 0x7f74e4162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e95fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74e40d3e00 0x7f74e40b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51606/0 (socket says 172.21.15.66:51606) 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e95fe640 1 -- 172.21.15.66:0/1100270372 learned_addr learned my addr 172.21.15.66:0/1100270372 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e9dff640 1 -- 172.21.15.66:0/1100270372 >> v2:172.21.15.66:3300/0 conn(0x7f74e40d3e00 msgr2=0x7f74e40b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e9dff640 1 --2- 172.21.15.66:0/1100270372 >> v2:172.21.15.66:3300/0 conn(0x7f74e40d3e00 0x7f74e40b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e9dff640 1 -- 172.21.15.66:0/1100270372 >> v1:172.21.15.117:6789/0 conn(0x7f74e40d1350 legacy=0x7f74e40d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.912+0000 7f74e9dff640 1 -- 172.21.15.66:0/1100270372 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e40b3bf0 con 0x7f74e415e4d0 2024-03-30T13:11:25.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.913+0000 7f74e8dfd640 1 -- 172.21.15.66:0/1100270372 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163864954 0 0) 0x7f74e40cf650 con 0x7f74e40d1350 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.913+0000 7f74e9dff640 1 --2- 172.21.15.66:0/1100270372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e415e4d0 0x7f74e4162a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74e40d2780 tx=0x7f74e002b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7115d829ddefb662 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.914+0000 7f74e8dfd640 1 -- 172.21.15.66:0/1100270372 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0038070 con 0x7f74e415e4d0 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.914+0000 7f74e8dfd640 1 -- 172.21.15.66:0/1100270372 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e0031710 con 0x7f74e415e4d0 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.914+0000 7f74e8dfd640 1 -- 172.21.15.66:0/1100270372 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0031ae0 con 0x7f74e415e4d0 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 -- 172.21.15.66:0/1100270372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e415e4d0 msgr2=0x7f74e4162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 --2- 172.21.15.66:0/1100270372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e415e4d0 0x7f74e4162a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f74e40d2780 tx=0x7f74e002b7b0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 -- 172.21.15.66:0/1100270372 shutdown_connections 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 --2- 172.21.15.66:0/1100270372 >> v2:172.21.15.66:3300/0 conn(0x7f74e40d3e00 0x7f74e40b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 --2- 172.21.15.66:0/1100270372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e415e4d0 0x7f74e4162a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 -- 172.21.15.66:0/1100270372 >> 172.21.15.66:0/1100270372 conn(0x7f74e40676e0 msgr2=0x7f74e4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 -- 172.21.15.66:0/1100270372 shutdown_connections 2024-03-30T13:11:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.915+0000 7f74ec063640 1 -- 172.21.15.66:0/1100270372 wait complete. 2024-03-30T13:11:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.916+0000 7f74ec063640 1 Processor -- start 2024-03-30T13:11:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.916+0000 7f74ec063640 1 -- start start 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.917+0000 7f74ec063640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 0x7f74e41511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.917+0000 7f74ec063640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74e40d3e00 0x7f74e41e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.917+0000 7f74ec063640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74e415e4d0 0x7f74e41e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.917+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 0x7f74e41511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.917+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 0x7f74e41511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:48832/0 (socket says 172.21.15.66:48832) 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e95fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74e40d3e00 0x7f74e41e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 -- 172.21.15.66:0/3061234555 learned_addr learned my addr 172.21.15.66:0/3061234555 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74e40d2ff0 con 0x7f74e40d3e00 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74e40d2e70 con 0x7f74e40d1350 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74e40d3170 con 0x7f74e415e4d0 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74ea600640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74e415e4d0 0x7f74e41e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 -- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74e415e4d0 msgr2=0x7f74e41e7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74e415e4d0 0x7f74e41e7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 -- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74e40d3e00 msgr2=0x7f74e41e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74e40d3e00 0x7f74e41e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 -- 172.21.15.66:0/3061234555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e41f4570 con 0x7f74e40d1350 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.918+0000 7f74e9dff640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 0x7f74e41511d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f74e0000c00 tx=0x7f74e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.919+0000 7f74da7fc640 1 -- 172.21.15.66:0/3061234555 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0042070 con 0x7f74e40d1350 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.919+0000 7f74da7fc640 1 -- 172.21.15.66:0/3061234555 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e003d870 con 0x7f74e40d1350 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.919+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e4153ff0 con 0x7f74e40d1350 2024-03-30T13:11:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.919+0000 7f74da7fc640 1 -- 172.21.15.66:0/3061234555 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0038040 con 0x7f74e40d1350 2024-03-30T13:11:25.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.919+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e41e8e50 con 0x7f74e40d1350 2024-03-30T13:11:25.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.921+0000 7f74da7fc640 1 -- 172.21.15.66:0/3061234555 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f74e003b070 con 0x7f74e40d1350 2024-03-30T13:11:25.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.921+0000 7f74da7fc640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74c806c530 0x7f74c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.922+0000 7f74da7fc640 1 -- 172.21.15.66:0/3061234555 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f74e00acdf0 con 0x7f74e40d1350 2024-03-30T13:11:25.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.922+0000 7f74e95fe640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74c806c530 0x7f74c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.923+0000 7f74e95fe640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74c806c530 0x7f74c806e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f74e40b2fe0 tx=0x7f74d4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.923+0000 7f74eec54740 1 Processor -- start 2024-03-30T13:11:25.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.923+0000 7f74eec54740 1 -- start start 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470c6d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470c6d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470c6d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:48842/0 (socket says 172.21.15.66:48842) 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74e9dff640 1 -- 172.21.15.66:0/1985555716 learned_addr learned my addr 172.21.15.66:0/1985555716 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74eec54740 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c6d6e0 0x563470c9d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74e95fe640 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563470bd7880 con 0x563470c6d6e0 2024-03-30T13:11:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563470b7f8f0 con 0x563470c9a200 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x563470bd3830 con 0x563470c9a5e0 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.925+0000 7f74ea600640 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c6d6e0 0x563470c9d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74e9dff640 1 -- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 msgr2=0x563470c9cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74e9dff640 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74e9dff640 1 -- 172.21.15.66:0/1985555716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c6d6e0 msgr2=0x563470c9d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74e9dff640 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c6d6e0 0x563470c9d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74e9dff640 1 -- 172.21.15.66:0/1985555716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563470b8a140 con 0x563470c9a200 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74ea600640 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c6d6e0 0x563470c9d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74e9dff640 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470c6d1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74e002bfd0 tx=0x7f74e007a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b635d4783aaa45 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74c77fe640 1 -- 172.21.15.66:0/1985555716 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00836a0 con 0x563470c9a200 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74c77fe640 1 -- 172.21.15.66:0/1985555716 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e0078dc0 con 0x563470c9a200 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.926+0000 7f74c77fe640 1 -- 172.21.15.66:0/1985555716 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0077130 con 0x563470c9a200 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 msgr2=0x563470c6d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470c6d1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74e002bfd0 tx=0x7f74e007a9c0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 shutdown_connections 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c6d6e0 0x563470c9d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 --2- 172.21.15.66:0/1985555716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470c6d1a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 >> 172.21.15.66:0/1985555716 conn(0x563470c6f410 msgr2=0x563470cb7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 shutdown_connections 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.927+0000 7f74eec54740 1 -- 172.21.15.66:0/1985555716 wait complete. 2024-03-30T13:11:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.928+0000 7f74eec54740 1 -- start start 2024-03-30T13:11:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.928+0000 7f74eec54740 1 Processor -- start 2024-03-30T13:11:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 0x563470c99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a5e0 0x563470cc09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 0x563470c99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 0x563470c99830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:38996/0 (socket says 172.21.15.66:38996) 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74e95fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a5e0 0x563470cc09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74e95fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a5e0 0x563470cc09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:48848/0 (socket says 172.21.15.66:48848) 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74eec54740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470c9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.930+0000 7f74e9dff640 1 -- 172.21.15.66:0/2556128437 learned_addr learned my addr 172.21.15.66:0/2556128437 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74ea600640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470c9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563470b7f8f0 con 0x563470cd0ca0 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563470b92cf0 con 0x563470c9a5e0 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x563470c71ab0 con 0x563470c9a200 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74eec54740 10 client.0 _ll_register_callbacks cb 0x563470b84f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74ea600640 1 -- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 msgr2=0x563470c99830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74ea600640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 0x563470c99830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74ea600640 1 -- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a5e0 msgr2=0x563470cc09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74ea600640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a5e0 0x563470cc09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74ea600640 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634709b57d0 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.931+0000 7f74e9dff640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 0x563470c99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74ea600640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470c9fc90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0012e0 tx=0x7f74dc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e1a102949f250b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74c4ff9640 1 -- 172.21.15.66:0/2556128437 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc01b060 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74c4ff9640 1 -- 172.21.15.66:0/2556128437 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f74dc016cc0 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e0078dc0 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74c4ff9640 1 -- 172.21.15.66:0/2556128437 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f74dc0124d0 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74eec54740 10 client.5646 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5634708eb3f0 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.932+0000 7f74c4ff9640 1 -- 172.21.15.66:0/2556128437 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74dc012670 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74c4ff9640 1 -- 172.21.15.66:0/2556128437 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f74dc0128a0 con 0x563470cd0ca0 2024-03-30T13:11:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74eec54740 10 client.5646 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74eec54740 10 client.5646 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:25.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74eec54740 20 client.5646 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74eec54740 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470cbed00 0x563470cbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74eec54740 4 client.5646 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f74e0078dc0 con 0x563470cbed00 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.933+0000 7f74e9dff640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470cbed00 0x563470cbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.934+0000 7f74e9dff640 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470cbed00 0x563470cbf110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b0383f11b8915ac server_cookie=f40056beb4809ca2 in_seq=0 out_seq=0 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.934+0000 7f74c4ff9640 10 client.5646 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74c4ff9640 1 -- 172.21.15.66:0/2556128437 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x7f74e0078dc0 con 0x563470cbed00 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74c4ff9640 10 client.5646 handle_command_reply: tid=0 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 client.5646 shutdown 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 msgr2=0x563470c9fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470c9fc90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0012e0 tx=0x7f74dc001a00 comp rx=0 tx=0).stop 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 shutdown_connections 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470cbed00 0x563470cbf110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470c9fc90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a5e0 0x563470cc09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 --2- 172.21.15.66:0/2556128437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a200 0x563470c99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74c4ff9640 0 client.5646 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.937+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 >> 172.21.15.66:0/2556128437 conn(0x563470c6d6e0 msgr2=0x563470c73fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.938+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 shutdown_connections 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.938+0000 7f74eec54740 1 -- 172.21.15.66:0/2556128437 wait complete. 2024-03-30T13:11:25.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.938+0000 7f74eec54740 20 client.5646 trim_cache size 0 max 16384 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.968+0000 7f74eec54740 1 Processor -- start 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.968+0000 7f74eec54740 1 -- start start 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74eec54740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 0x563470cb8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 0x563470cb8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 0x563470cb8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51630/0 (socket says 172.21.15.66:51630) 2024-03-30T13:11:25.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470cd0ca0 0x563470cbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74e9dff640 1 -- 172.21.15.66:0/1503369655 learned_addr learned my addr 172.21.15.66:0/1503369655 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4070c20 con 0x563470c9a200 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74e406fc20 con 0x563470cd0ca0 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74e4158820 con 0x563470c9a5e0 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.969+0000 7f74e95fe640 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74ea600640 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470cd0ca0 0x563470cbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e95fe640 1 -- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470cd0ca0 msgr2=0x563470cbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e95fe640 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470cd0ca0 0x563470cbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e95fe640 1 -- 172.21.15.66:0/1503369655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 msgr2=0x563470cb8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e95fe640 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 0x563470cb8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e95fe640 1 -- 172.21.15.66:0/1503369655 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563470c99010 con 0x563470c9a5e0 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e9dff640 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 0x563470cb8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:25.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74e95fe640 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9fa20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009a90 tx=0x7f74d4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d462fb200b397c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74c6ffd640 1 -- 172.21.15.66:0/1503369655 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4014de0 con 0x563470c9a5e0 2024-03-30T13:11:25.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74c6ffd640 1 -- 172.21.15.66:0/1503369655 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d401b050 con 0x563470c9a5e0 2024-03-30T13:11:25.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.970+0000 7f74c6ffd640 1 -- 172.21.15.66:0/1503369655 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d402c680 con 0x563470c9a5e0 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 msgr2=0x563470c9fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9fa20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009a90 tx=0x7f74d4014000 comp rx=0 tx=0).stop 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 shutdown_connections 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470cd0ca0 0x563470cbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470c9fa20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 --2- 172.21.15.66:0/1503369655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470c9a200 0x563470cb8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 >> 172.21.15.66:0/1503369655 conn(0x563470c6d6e0 msgr2=0x563470ce48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 shutdown_connections 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.971+0000 7f74eec54740 1 -- 172.21.15.66:0/1503369655 wait complete. 2024-03-30T13:11:25.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.972+0000 7f74eec54740 1 -- start start 2024-03-30T13:11:25.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.972+0000 7f74eec54740 1 Processor -- start 2024-03-30T13:11:25.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470d39bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470cbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470cbb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563470c98ca0 con 0x563470cd0ca0 2024-03-30T13:11:25.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4158e70 con 0x563470c9a200 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x563470d3dba0 con 0x563470c9a5e0 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470d39bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74e9dff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470d39bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:48858/0 (socket says 172.21.15.66:48858) 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74e9dff640 1 -- 172.21.15.66:0/596345428 learned_addr learned my addr 172.21.15.66:0/596345428 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74e95fe640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470cbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74eec54740 10 client.0 _ll_register_callbacks cb 0x7f74e406d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.974+0000 7f74ea600640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470cbb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74e9dff640 1 -- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 msgr2=0x563470cbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74e9dff640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470cbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74e9dff640 1 -- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 msgr2=0x563470cbb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74e9dff640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470cbb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74e9dff640 1 -- 172.21.15.66:0/596345428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634708ab0f0 con 0x563470c9a200 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74ea600640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470cbb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:25.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74e9dff640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470d39bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74e007a7c0 tx=0x7f74e00ac000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5434a59fc7fa1d39 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74c4ff9640 1 -- 172.21.15.66:0/596345428 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0077130 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d401b050 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74c4ff9640 1 -- 172.21.15.66:0/596345428 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f74e003dc60 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74c4ff9640 1 -- 172.21.15.66:0/596345428 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f74e00a4050 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 10 client.5530 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x563470d3d8e0 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74c4ff9640 1 -- 172.21.15.66:0/596345428 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d401b050 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74c4ff9640 1 -- 172.21.15.66:0/596345428 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f74e009d020 con 0x563470c9a200 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 10 client.5530 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 10 client.5530 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 20 client.5530 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.975+0000 7f74eec54740 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470c99640 0x563470cda490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 4 client.5530 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563470cc2310 con 0x563470c99640 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74e95fe640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470c99640 0x563470cda490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74c4ff9640 10 client.5530 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74e95fe640 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470c99640 0x563470cda490 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf5ddba6f8a4fd5 server_cookie=b5cdfb4e752697de in_seq=0 out_seq=0 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74c4ff9640 1 -- 172.21.15.66:0/596345428 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x563470cc2310 con 0x563470c99640 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74c4ff9640 10 client.5530 handle_command_reply: tid=0 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 client.5530 shutdown 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 msgr2=0x563470d39bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470d39bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74e007a7c0 tx=0x7f74e00ac000 comp rx=0 tx=0).stop 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 shutdown_connections 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563470c99640 0x563470cda490 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563470cd0ca0 0x563470cbb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563470c9a5e0 0x563470cbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 --2- 172.21.15.66:0/596345428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563470c9a200 0x563470d39bf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74c4ff9640 0 client.5530 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:25.986 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:25.986 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T13:11:25.986 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T13:11:25.986 INFO:teuthology.orchestra.run.smithi066.stdout: "c7ed1273-d477-4170-a25d-5b1410281d2c": { 2024-03-30T13:11:25.986 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "c7ed1273-d477-4170-a25d-5b1410281d2c", 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive" 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_stats": { 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "c7ed1273-d477-4170-a25d-5b1410281d2c": { 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "start_time": "2024-03-30T13:11:25.735190+0000", 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 0, 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_passed": 0, 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_failed": 0, 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_skipped": 0 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:25.987 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.976+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 >> 172.21.15.66:0/596345428 conn(0x563470c6d6e0 msgr2=0x563470ce49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 shutdown_connections 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74eec54740 1 -- 172.21.15.66:0/596345428 wait complete. 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74eec54740 20 client.5530 trim_cache size 0 max 16384 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74c806c530 msgr2=0x7f74c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74c806c530 0x7f74c806e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f74e40b2fe0 tx=0x7f74d4009240 comp rx=0 tx=0).stop 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 msgr2=0x7f74e41511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 0x7f74e41511d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f74e0000c00 tx=0x7f74e00046c0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 shutdown_connections 2024-03-30T13:11:25.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74c806c530 0x7f74c806e9f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74e415e4d0 0x7f74e41e7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74e40d3e00 0x7f74e41e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 --2- 172.21.15.66:0/3061234555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74e40d1350 0x7f74e41511d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 >> 172.21.15.66:0/3061234555 conn(0x7f74e40676e0 msgr2=0x7f74e40d1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 shutdown_connections 2024-03-30T13:11:25.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:25.977+0000 7f74ec063640 1 -- 172.21.15.66:0/3061234555 wait complete. 2024-03-30T13:11:25.992 INFO:tasks.cephfs.filesystem:scrub status for tag:c7ed1273-d477-4170-a25d-5b1410281d2c - {'path': '/', 'tag': 'c7ed1273-d477-4170-a25d-5b1410281d2c', 'options': 'recursive'} 2024-03-30T13:11:42.260 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:42.264 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:55.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:55.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:11:56.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.098+0000 7fc3c9fdb640 1 Processor -- start 2024-03-30T13:11:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.098+0000 7fc3c9fdb640 1 -- start start 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.098+0000 7fc3c9fdb640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3c4158e00 0x7fc3c40b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.098+0000 7fc3c9fdb640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c4072570 con 0x7fc3c406bc90 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c9fdb640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c4072850 con 0x7fc3c4158a20 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c9fdb640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c4072b30 con 0x7fc3c4158e00 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c2ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3c4158e00 0x7fc3c40b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c2ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3c4158e00 0x7fc3c40b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56848/0 (socket says 172.21.15.66:56848) 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c3fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc3c406bc90 0x7fc3c40b0570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37450/0 (socket says 172.21.15.66:37450) 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2921190634 learned_addr learned my addr 172.21.15.66:0/2921190634 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2921190634 >> v1:172.21.15.117:6789/0 conn(0x7fc3c4158a20 legacy=0x7fc3c406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2921190634 >> v1:172.21.15.66:6789/0 conn(0x7fc3c406bc90 legacy=0x7fc3c40b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c27fc640 1 -- 172.21.15.66:0/2921190634 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262444377 0 0) 0x7fc3c4072570 con 0x7fc3c406bc90 2024-03-30T13:11:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.099+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2921190634 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c4072e10 con 0x7fc3c4158e00 2024-03-30T13:11:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.100+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/2921190634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c4158e00 0x7fc3c40b0030 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009870 tx=0x7fc3b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5543e3efe345116f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.100+0000 7fc3c27fc640 1 -- 172.21.15.66:0/2921190634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8034070 con 0x7fc3c4158e00 2024-03-30T13:11:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.100+0000 7fc3c27fc640 1 -- 172.21.15.66:0/2921190634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8030860 con 0x7fc3c4158e00 2024-03-30T13:11:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.100+0000 7fc3c27fc640 1 -- 172.21.15.66:0/2921190634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8030c30 con 0x7fc3c4158e00 2024-03-30T13:11:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/2921190634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c4158e00 msgr2=0x7fc3c40b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/2921190634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c4158e00 0x7fc3c40b0030 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009870 tx=0x7fc3b802b9a0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/2921190634 shutdown_connections 2024-03-30T13:11:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/2921190634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c4158e00 0x7fc3c40b0030 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/2921190634 >> 172.21.15.66:0/2921190634 conn(0x7fc3c40d1160 msgr2=0x7fc3c40d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/2921190634 shutdown_connections 2024-03-30T13:11:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.101+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/2921190634 wait complete. 2024-03-30T13:11:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.102+0000 7fc3c9fdb640 1 Processor -- start 2024-03-30T13:11:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.102+0000 7fc3c9fdb640 1 -- start start 2024-03-30T13:11:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c9fdb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 0x7fc3c414e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c9fdb640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c4158a20 0x7fc3c4154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c9fdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c4158e00 0x7fc3c4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c9fdb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41692b0 con 0x7fc3c406bc90 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c9fdb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c40b5120 con 0x7fc3c4158e00 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c9fdb640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c4169430 con 0x7fc3c4158a20 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 0x7fc3c414e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 0x7fc3c414e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56860/0 (socket says 172.21.15.66:56860) 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c37fe640 1 -- 172.21.15.66:0/326075660 learned_addr learned my addr 172.21.15.66:0/326075660 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c4158e00 0x7fc3c4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.103+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c4158a20 0x7fc3c4154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c37fe640 1 -- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c4158a20 msgr2=0x7fc3c4154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c4158a20 0x7fc3c4154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c37fe640 1 -- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c4158e00 msgr2=0x7fc3c4154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c4158e00 0x7fc3c4154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c37fe640 1 -- 172.21.15.66:0/326075660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c4154ff0 con 0x7fc3c406bc90 2024-03-30T13:11:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c4158e00 0x7fc3c4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 0x7fc3c414e6b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c430 tx=0x7fc3b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3abfff640 1 -- 172.21.15.66:0/326075660 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4011790 con 0x7fc3c406bc90 2024-03-30T13:11:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3abfff640 1 -- 172.21.15.66:0/326075660 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b4017070 con 0x7fc3c406bc90 2024-03-30T13:11:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c40adaf0 con 0x7fc3c406bc90 2024-03-30T13:11:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3abfff640 1 -- 172.21.15.66:0/326075660 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4011bb0 con 0x7fc3c406bc90 2024-03-30T13:11:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.104+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c40ae0b0 con 0x7fc3c406bc90 2024-03-30T13:11:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.106+0000 7fc3abfff640 1 -- 172.21.15.66:0/326075660 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc3b401c460 con 0x7fc3c406bc90 2024-03-30T13:11:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.107+0000 7fc3abfff640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc3b006c530 0x7fc3b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.107+0000 7fc3abfff640 1 -- 172.21.15.66:0/326075660 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc3b408b6a0 con 0x7fc3c406bc90 2024-03-30T13:11:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.107+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc3b006c530 0x7fc3b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.107+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc3b006c530 0x7fc3b006e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8036000 tx=0x7fc3b803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.107+0000 7fc3ccbc6740 1 Processor -- start 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.107+0000 7fc3ccbc6740 1 -- start start 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf200 0x565225e821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225eb1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225e826e0 0x565225eb24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565225dec880 con 0x565225eaf5e0 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565225d948f0 con 0x565225e826e0 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x565225de8830 con 0x565225eaf200 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225eb1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225eb1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56868/0 (socket says 172.21.15.66:56868) 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/168355532 learned_addr learned my addr 172.21.15.66:0/168355532 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf200 0x565225e821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225e826e0 0x565225eb24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf200 msgr2=0x565225e821a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf200 0x565225e821a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225e826e0 msgr2=0x565225eb24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225e826e0 0x565225eb24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/168355532 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565225d9f140 con 0x565225eaf5e0 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf200 0x565225e821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225e826e0 0x565225eb24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.109+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225eb1f90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc3b802bfb0 tx=0x7fc3b8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3fb1d3e26ca7c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3a97fa640 1 -- 172.21.15.66:0/168355532 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8034040 con 0x565225eaf5e0 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3a97fa640 1 -- 172.21.15.66:0/168355532 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8003790 con 0x565225eaf5e0 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3a97fa640 1 -- 172.21.15.66:0/168355532 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8003b40 con 0x565225eaf5e0 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/168355532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 msgr2=0x565225eb1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225eb1f90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc3b802bfb0 tx=0x7fc3b8030000 comp rx=0 tx=0).stop 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/168355532 shutdown_connections 2024-03-30T13:11:56.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225e826e0 0x565225eb24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225eb1f90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/168355532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf200 0x565225e821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/168355532 >> 172.21.15.66:0/168355532 conn(0x565225e84410 msgr2=0x565225eccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/168355532 shutdown_connections 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.110+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/168355532 wait complete. 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.111+0000 7fc3ccbc6740 1 -- start start 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.111+0000 7fc3ccbc6740 1 Processor -- start 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225ea7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf5e0 0x565225eaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225ea8a70 0x565225eaaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565225e86ab0 con 0x565225eaf200 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565225da7cf0 con 0x565225ea8a70 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x565225e892f0 con 0x565225eaf5e0 2024-03-30T13:11:56.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 10 client.0 _ll_register_callbacks cb 0x565225d99f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225ea7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225ea7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56878/0 (socket says 172.21.15.66:56878) 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 -- 172.21.15.66:0/953746891 learned_addr learned my addr 172.21.15.66:0/953746891 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf5e0 0x565225eaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225ea8a70 0x565225eaaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 -- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf5e0 msgr2=0x565225eaa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf5e0 0x565225eaa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 -- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225ea8a70 msgr2=0x565225eaaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225ea8a70 0x565225eaaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 -- 172.21.15.66:0/953746891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565225bca7d0 con 0x565225eaf200 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf5e0 0x565225eaa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225ea7a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4059110 tx=0x7fc3b405d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8f81a23ae51b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:56.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3a27fc640 1 -- 172.21.15.66:0/953746891 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4071020 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3a27fc640 1 -- 172.21.15.66:0/953746891 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc3b4055a80 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.112+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b8003790 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3a27fc640 1 -- 172.21.15.66:0/953746891 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc3b4058860 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 10 client.5667 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x565225b003f0 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3a27fc640 1 -- 172.21.15.66:0/953746891 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8003790 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3a27fc640 1 -- 172.21.15.66:0/953746891 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fc3b4058a00 con 0x565225eaf200 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 10 client.5667 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 10 client.5667 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 20 client.5667 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225ee0ee0 0x565225ee1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 4 client.5667 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565225df9dc0 con 0x565225ee0ee0 2024-03-30T13:11:56.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225ee0ee0 0x565225ee1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3a27fc640 10 client.5667 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.113+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225ee0ee0 0x565225ee1300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef9e47e342beb37 server_cookie=e1ce5564db5ad64e in_seq=0 out_seq=0 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3a27fc640 1 -- 172.21.15.66:0/953746891 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x7fc3b805dad0 con 0x565225ee0ee0 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3a27fc640 10 client.5667 handle_command_reply: tid=0 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 client.5667 shutdown 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 msgr2=0x565225ea7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225ea7a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4059110 tx=0x7fc3b405d7c0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 shutdown_connections 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225ee0ee0 0x565225ee1300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225ea8a70 0x565225eaaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3a27fc640 0 client.5667 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225eaf5e0 0x565225eaa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/953746891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225ea7a50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 >> 172.21.15.66:0/953746891 conn(0x565225e826e0 msgr2=0x565225e81d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 shutdown_connections 2024-03-30T13:11:56.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.114+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/953746891 wait complete. 2024-03-30T13:11:56.127 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.115+0000 7fc3ccbc6740 20 client.5667 trim_cache size 0 max 16384 2024-03-30T13:11:56.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.144+0000 7fc3ccbc6740 1 Processor -- start 2024-03-30T13:11:56.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.144+0000 7fc3ccbc6740 1 -- start start 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225e88ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf200 0x565225eb3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225e81050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4201080 con 0x565225eaf5e0 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c42001a0 con 0x565225eaf200 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c41f8aa0 con 0x565225ea8a70 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225e81050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225e81050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56890/0 (socket says 172.21.15.66:56890) 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 -- 172.21.15.66:0/238286753 learned_addr learned my addr 172.21.15.66:0/238286753 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf200 0x565225eb3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225e88ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 -- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 msgr2=0x565225e88ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225e88ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 -- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf200 msgr2=0x565225eb3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf200 0x565225eb3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c3fff640 1 -- 172.21.15.66:0/238286753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565225ea92f0 con 0x565225eaf5e0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.145+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225e88ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.146+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225e81050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc000f80 tx=0x7fc3bc007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e203c4f8c2e5d34d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.146+0000 7fc3a8ff9640 1 -- 172.21.15.66:0/238286753 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc010040 con 0x565225eaf5e0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.146+0000 7fc3a8ff9640 1 -- 172.21.15.66:0/238286753 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc01a070 con 0x565225eaf5e0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.146+0000 7fc3a8ff9640 1 -- 172.21.15.66:0/238286753 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc001fa0 con 0x565225eaf5e0 2024-03-30T13:11:56.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/238286753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 msgr2=0x565225e81050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225e81050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc000f80 tx=0x7fc3bc007940 comp rx=0 tx=0).stop 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/238286753 shutdown_connections 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf5e0 0x565225e81050 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf200 0x565225eb3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/238286753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225e88ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/238286753 >> 172.21.15.66:0/238286753 conn(0x565225e826e0 msgr2=0x565225edcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/238286753 shutdown_connections 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/238286753 wait complete. 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 -- start start 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.147+0000 7fc3ccbc6740 1 Processor -- start 2024-03-30T13:11:56.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225f3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225f40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf5e0 0x565225e86d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565225ef16e0 con 0x565225eaf200 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4156b60 con 0x565225eaf5e0 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x565225eb5ec0 con 0x565225ea8a70 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3ccbc6740 10 client.0 _ll_register_callbacks cb 0x7fc3b40591e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225f40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3c2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225f40330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56900/0 (socket says 172.21.15.66:56900) 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2141785712 learned_addr learned my addr 172.21.15.66:0/2141785712 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225f3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.148+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf5e0 0x565225e86d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 msgr2=0x565225f3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225f3fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf5e0 msgr2=0x565225e86d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf5e0 0x565225e86d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c2ffd640 1 -- 172.21.15.66:0/2141785712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565225ac00f0 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c3fff640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf5e0 0x565225e86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3c2ffd640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225f40330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009870 tx=0x7fc3b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa95bb13efa05a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3a27fc640 1 -- 172.21.15.66:0/2141785712 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8056040 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3a27fc640 1 -- 172.21.15.66:0/2141785712 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc3b804d040 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3bc01a070 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3a27fc640 1 -- 172.21.15.66:0/2141785712 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc3b8034040 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 10 client.5673 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x565225e88e80 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3a27fc640 1 -- 172.21.15.66:0/2141785712 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc01a070 con 0x565225eaf200 2024-03-30T13:11:56.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3a27fc640 1 -- 172.21.15.66:0/2141785712 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fc3b806e080 con 0x565225eaf200 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 10 client.5673 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 10 client.5673 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 20 client.5673 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225eaa840 0x565225e872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 4 client.5673 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.149+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565225df4110 con 0x565225eaa840 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225eaa840 0x565225e872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3a27fc640 10 client.5673 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3c37fe640 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225eaa840 0x565225e872b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91a55ed7b982a36 server_cookie=236d77eb42029291 in_seq=0 out_seq=0 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3a27fc640 1 -- 172.21.15.66:0/2141785712 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+376 (crc 0 0 0) 0x565225df4110 con 0x565225eaa840 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3a27fc640 10 client.5673 handle_command_reply: tid=0 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 client.5673 shutdown 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 msgr2=0x565225f40330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225f40330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009870 tx=0x7fc3b8004730 comp rx=0 tx=0).stop 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 shutdown_connections 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x565225eaa840 0x565225e872b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565225eaf5e0 0x565225e86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565225eaf200 0x565225f40330 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3a27fc640 0 client.5673 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 --2- 172.21.15.66:0/2141785712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565225ea8a70 0x565225f3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 >> 172.21.15.66:0/2141785712 conn(0x565225e826e0 msgr2=0x565225e81050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 shutdown_connections 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.150+0000 7fc3ccbc6740 1 -- 172.21.15.66:0/2141785712 wait complete. 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3ccbc6740 20 client.5673 trim_cache size 0 max 16384 2024-03-30T13:11:56.161 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {}, 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_stats": { 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "c7ed1273-d477-4170-a25d-5b1410281d2c": { 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "start_time": "2024-03-30T13:11:25.735190+0000", 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 75, 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_passed": 75, 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_failed": 0, 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_skipped": 0 2024-03-30T13:11:56.162 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:56.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:56.163 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc3b006c530 msgr2=0x7fc3b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc3b006c530 0x7fc3b006e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8036000 tx=0x7fc3b803c040 comp rx=0 tx=0).stop 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 msgr2=0x7fc3c414e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 0x7fc3c414e6b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c430 tx=0x7fc3b400c8f0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 shutdown_connections 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc3b006c530 0x7fc3b006e9f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c4158e00 0x7fc3c4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c4158a20 0x7fc3c4154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 --2- 172.21.15.66:0/326075660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c406bc90 0x7fc3c414e6b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 >> 172.21.15.66:0/326075660 conn(0x7fc3c40d1160 msgr2=0x7fc3c40b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 shutdown_connections 2024-03-30T13:11:56.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:56.151+0000 7fc3c9fdb640 1 -- 172.21.15.66:0/326075660 wait complete. 2024-03-30T13:11:56.167 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T13:11:56.167 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-30T13:11:56.167 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:11:56.167 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-30T13:11:56.185 INFO:teuthology.orchestra.run.smithi066.stdout:1099511627780 linux-5.4/fs/ceph/Kconfig 2024-03-30T13:11:56.186 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-30T13:11:56.186 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-30T13:11:56.240 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628281 linux-5.4/fs/ceph/Makefile 2024-03-30T13:11:56.240 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:11:56.241 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:11:56.295 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628287 linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:11:56.295 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-30T13:11:56.296 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-30T13:11:56.350 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628294 linux-5.4/fs/ceph/io.h 2024-03-30T13:11:56.351 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:11:56.351 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:11:56.405 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628296 linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:11:56.406 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:11:56.406 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-30T13:11:56.461 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628308 linux-5.4/fs/ext2/Kconfig 2024-03-30T13:11:56.461 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:11:56.461 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:11:56.517 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628326 linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:11:56.517 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:11:56.517 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:11:56.572 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628324 linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:11:56.572 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-30T13:11:56.572 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628309 linux-5.4/fs/ext2/Makefile 2024-03-30T13:11:56.627 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:11:56.627 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-30T13:11:56.681 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628321 linux-5.4/fs/ext2/symlink.c 2024-03-30T13:11:56.681 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-30T13:11:56.682 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-30T13:11:56.736 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628311 linux-5.4/fs/ext2/acl.h 2024-03-30T13:11:56.737 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:11:56.737 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:11:56.791 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628325 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:11:56.792 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:11:56.792 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-30T13:11:56.846 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628323 linux-5.4/fs/ext2/xattr.h 2024-03-30T13:11:56.847 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:11:56.847 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:11:56.901 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628328 linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:11:56.902 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:11:56.902 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:11:56.956 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628341 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:11:56.956 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:11:56.957 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:11:57.011 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628348 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:11:57.012 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:11:57.012 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:11:57.066 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628331 linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:11:57.066 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:11:57.067 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:11:57.124 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628339 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:11:57.125 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:11:57.125 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:11:57.180 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628343 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:11:57.181 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:11:57.181 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:11:57.236 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628344 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:11:57.236 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:11:57.236 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-30T13:11:57.292 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628329 linux-5.4/fs/orangefs/Makefile 2024-03-30T13:11:57.292 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:11:57.293 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:11:57.348 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628352 linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:11:57.348 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:11:57.349 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:11:57.404 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628334 linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:11:57.404 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-30T13:11:57.404 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511627780 2024-03-30T13:11:57.404 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511627780 2024-03-30T13:11:57.405 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511627780 2024-03-30T13:11:57.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.550+0000 7f3f44022640 1 Processor -- start 2024-03-30T13:11:57.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.551+0000 7f3f44022640 1 -- start start 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.551+0000 7f3f44022640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0ba090 0x7f3f3c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.551+0000 7f3f44022640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0d2ff0 con 0x7f3f3c0b73d0 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.551+0000 7f3f44022640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0d32d0 con 0x7f3f3c0b5dc0 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.551+0000 7f3f44022640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3c0d35b0 con 0x7f3f3c0ba090 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f425bf640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3f3c0b73d0 0x7f3f3c0b7820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37462/0 (socket says 172.21.15.66:37462) 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f425bf640 1 -- 172.21.15.66:0/1818450916 learned_addr learned my addr 172.21.15.66:0/1818450916 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f415bd640 1 --2- 172.21.15.66:0/1818450916 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0ba090 0x7f3f3c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610990534 0 0) 0x7f3f3c0d2ff0 con 0x7f3f3c0b73d0 2024-03-30T13:11:57.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f28003520 con 0x7f3f3c0b73d0 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 87242016 0 0) 0x7f3f28003520 con 0x7f3f3c0b73d0 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0ba090 msgr2=0x7f3f3c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 --2- 172.21.15.66:0/1818450916 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0ba090 0x7f3f3c0d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 >> v1:172.21.15.117:6789/0 conn(0x7f3f3c0b5dc0 legacy=0x7f3f3c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0d3890 con 0x7f3f3c0b73d0 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f3f380019f0 con 0x7f3f3c0b73d0 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 >> v1:172.21.15.66:6789/0 conn(0x7f3f3c0b73d0 legacy=0x7f3f3c0b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f280039e0 0x7f3f28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f28004800 0x7f3f28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f280051c0 0x7f3f2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c0d35b0 con 0x7f3f280051c0 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800c010 con 0x7f3f28004800 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2800c210 con 0x7f3f280039e0 2024-03-30T13:11:57.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f425bf640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f280051c0 0x7f3f2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.552+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f280039e0 0x7f3f28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.553+0000 7f3f415bd640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f28004800 0x7f3f28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.553+0000 7f3f425bf640 1 -- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f280039e0 msgr2=0x7f3f28003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.553+0000 7f3f425bf640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f280039e0 0x7f3f28003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.553+0000 7f3f425bf640 1 -- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f28004800 msgr2=0x7f3f28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.553+0000 7f3f425bf640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f28004800 0x7f3f28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.553+0000 7f3f425bf640 1 -- 172.21.15.66:0/1818450916 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f28003520 con 0x7f3f280051c0 2024-03-30T13:11:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.554+0000 7f3f425bf640 1 -- 172.21.15.66:0/1818450916 reap_dead start 2024-03-30T13:11:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.554+0000 7f3f425bf640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f280051c0 0x7f3f2800b9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800a880 tx=0x7f3f28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90df23620fdb8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.554+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800a4a0 con 0x7f3f280051c0 2024-03-30T13:11:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.554+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38034650 con 0x7f3f280051c0 2024-03-30T13:11:57.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.554+0000 7f3f40dbc640 1 -- 172.21.15.66:0/1818450916 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38034a40 con 0x7f3f280051c0 2024-03-30T13:11:57.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 -- 172.21.15.66:0/1818450916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f280051c0 msgr2=0x7f3f2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f280051c0 0x7f3f2800b9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800a880 tx=0x7f3f28005830 comp rx=0 tx=0).stop 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 -- 172.21.15.66:0/1818450916 shutdown_connections 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 --2- 172.21.15.66:0/1818450916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f280051c0 0x7f3f2800b9c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 -- 172.21.15.66:0/1818450916 >> 172.21.15.66:0/1818450916 conn(0x7f3f3c0678f0 msgr2=0x7f3f3c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 -- 172.21.15.66:0/1818450916 shutdown_connections 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.555+0000 7f3f44022640 1 -- 172.21.15.66:0/1818450916 wait complete. 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.556+0000 7f3f44022640 1 Processor -- start 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.556+0000 7f3f44022640 1 -- start start 2024-03-30T13:11:57.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f44022640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f280051c0 0x7f3f3c15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f44022640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c15c530 0x7f3f3c0aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f44022640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 0x7f3f3c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f44022640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c15e880 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f44022640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c0d4090 con 0x7f3f280051c0 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f44022640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c15ea00 con 0x7f3f3c15c530 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 0x7f3f3c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 0x7f3f3c160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56922/0 (socket says 172.21.15.66:56922) 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 -- 172.21.15.66:0/28384026 learned_addr learned my addr 172.21.15.66:0/28384026 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f415bd640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c15c530 0x7f3f3c0aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f280051c0 0x7f3f3c15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 -- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c15c530 msgr2=0x7f3f3c0aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c15c530 0x7f3f3c0aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 -- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f280051c0 msgr2=0x7f3f3c15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f280051c0 0x7f3f3c15eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.557+0000 7f3f425bf640 1 -- 172.21.15.66:0/28384026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0ad4c0 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.558+0000 7f3f425bf640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 0x7f3f3c160f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f3802cc70 tx=0x7f3f3c0d31c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.558+0000 7f3f327fc640 1 -- 172.21.15.66:0/28384026 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38039040 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.558+0000 7f3f327fc640 1 -- 172.21.15.66:0/28384026 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38043070 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.558+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c0aadd0 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.558+0000 7f3f327fc640 1 -- 172.21.15.66:0/28384026 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800a4a0 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.558+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c0ab390 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.560+0000 7f3f327fc640 1 -- 172.21.15.66:0/28384026 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3f38034650 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.560+0000 7f3f327fc640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3f2006c560 0x7f3f2006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.560+0000 7f3f327fc640 1 -- 172.21.15.66:0/28384026 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3f380ad380 con 0x7f3f3c15c9b0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3f2006c560 0x7f3f2006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 Processor -- start 2024-03-30T13:11:57.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 -- start start 2024-03-30T13:11:57.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3f2006c560 0x7f3f2006ea20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0d51f0 tx=0x7f3f34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c570 0x55895315f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c950 0x55895318f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 0x55895318f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5589530c9590 con 0x55895315fa50 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558953067d30 con 0x55895318c950 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.561+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5589530c5540 con 0x55895318c570 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 0x55895318f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 0x55895318f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56928/0 (socket says 172.21.15.66:56928) 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 -- 172.21.15.66:0/814685194 learned_addr learned my addr 172.21.15.66:0/814685194 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f415bd640 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c950 0x55895318f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c570 0x55895315f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 -- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c570 msgr2=0x55895315f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c570 0x55895315f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 -- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c950 msgr2=0x55895318f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c950 0x55895318f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 -- 172.21.15.66:0/814685194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558952ea9920 con 0x55895315fa50 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f425bf640 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 0x55895318f840 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3f380837b0 tx=0x7f3f3807ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c6c1a4b68dccd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f1f7fe640 1 -- 172.21.15.66:0/814685194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3807b100 con 0x55895315fa50 2024-03-30T13:11:57.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f1f7fe640 1 -- 172.21.15.66:0/814685194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3807f4b0 con 0x55895315fa50 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.562+0000 7f3f1f7fe640 1 -- 172.21.15.66:0/814685194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38083ce0 con 0x55895315fa50 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 -- 172.21.15.66:0/814685194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 msgr2=0x55895318f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 0x55895318f840 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3f380837b0 tx=0x7f3f3807ad40 comp rx=0 tx=0).stop 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 -- 172.21.15.66:0/814685194 shutdown_connections 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895315fa50 0x55895318f840 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c950 0x55895318f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/814685194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c570 0x55895315f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 -- 172.21.15.66:0/814685194 >> 172.21.15.66:0/814685194 conn(0x558953161780 msgr2=0x5589531aa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 -- 172.21.15.66:0/814685194 shutdown_connections 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.563+0000 7f3f46c0d740 1 -- 172.21.15.66:0/814685194 wait complete. 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.564+0000 7f3f46c0d740 1 -- start start 2024-03-30T13:11:57.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.564+0000 7f3f46c0d740 1 Processor -- start 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x558953192f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953190b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x558953180790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558953067d30 con 0x55895318c570 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589530659f0 con 0x558953181ae0 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5589530c9590 con 0x55895318c950 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x558953192f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f46c0d740 10 client.0 _ll_register_callbacks cb 0x558953069c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x558953192f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56934/0 (socket says 172.21.15.66:56934) 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 -- 172.21.15.66:0/4174364462 learned_addr learned my addr 172.21.15.66:0/4174364462 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f415bd640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953190b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f425bf640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x558953180790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 -- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 msgr2=0x558953190b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953190b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 -- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 msgr2=0x558953180790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x558953180790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 -- 172.21.15.66:0/4174364462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558952d9d210 con 0x55895318c570 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f425bf640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x558953180790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x558953192f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3f34015a20 tx=0x7f3f34007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4414a01e3c1e533 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f1cff9640 1 -- 172.21.15.66:0/4174364462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34019040 con 0x55895318c570 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.566+0000 7f3f1cff9640 1 -- 172.21.15.66:0/4174364462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3f34022780 con 0x55895318c570 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f3807f4b0 con 0x55895318c570 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f1cff9640 1 -- 172.21.15.66:0/4174364462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3f34032050 con 0x55895318c570 2024-03-30T13:11:57.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 10 client.5685 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558952f0a0b0 con 0x55895318c570 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f1cff9640 1 -- 172.21.15.66:0/4174364462 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3807f4b0 con 0x55895318c570 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f1cff9640 1 -- 172.21.15.66:0/4174364462 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f3f34035020 con 0x55895318c570 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 10 client.5685 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 10 client.5685 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 20 client.5685 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531837e0 0x558953183bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 4 client.5685 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589530d6ad0 con 0x5589531837e0 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.567+0000 7f3f415bd640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531837e0 0x558953183bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f1cff9640 10 client.5685 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f415bd640 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531837e0 0x558953183bf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479cbd3ce8263ab0 server_cookie=e8d6274e64084929 in_seq=0 out_seq=0 2024-03-30T13:11:57.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f1cff9640 1 -- 172.21.15.66:0/4174364462 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5589530d6ad0 con 0x5589531837e0 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f1cff9640 10 client.5685 handle_command_reply: tid=0 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 client.5685 shutdown 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 msgr2=0x558953192f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x558953192f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3f34015a20 tx=0x7f3f34007ca0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 shutdown_connections 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531837e0 0x558953183bf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x558953180790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953190b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f1cff9640 0 client.5685 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/4174364462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x558953192f10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.568+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 >> 172.21.15.66:0/4174364462 conn(0x55895315fa50 msgr2=0x5589531ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.569+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 shutdown_connections 2024-03-30T13:11:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.569+0000 7f3f46c0d740 1 -- 172.21.15.66:0/4174364462 wait complete. 2024-03-30T13:11:57.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.569+0000 7f3f46c0d740 20 client.5685 trim_cache size 0 max 16384 2024-03-30T13:11:57.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.583+0000 7f3f46c0d740 1 Processor -- start 2024-03-30T13:11:57.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.583+0000 7f3f46c0d740 1 -- start start 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 0x5589531abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c570 0x5589531ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x5589531bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c201640 con 0x558953181ae0 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c2006a0 con 0x55895318c570 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c1ffa00 con 0x55895318c950 2024-03-30T13:11:57.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f41dbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 0x5589531abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f41dbe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 0x5589531abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56940/0 (socket says 172.21.15.66:56940) 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f41dbe640 1 -- 172.21.15.66:0/2152187874 learned_addr learned my addr 172.21.15.66:0/2152187874 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f415bd640 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c570 0x5589531ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.584+0000 7f3f425bf640 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x5589531bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f41dbe640 1 -- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 msgr2=0x5589531bc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x5589531bc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f41dbe640 1 -- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c570 msgr2=0x5589531ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c570 0x5589531ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f41dbe640 1 -- 172.21.15.66:0/2152187874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558952d762d0 con 0x558953181ae0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 0x5589531abe80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3f34012790 tx=0x7f3f34024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e304cda714b6ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f1effd640 1 -- 172.21.15.66:0/2152187874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34019070 con 0x558953181ae0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f1effd640 1 -- 172.21.15.66:0/2152187874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34026030 con 0x558953181ae0 2024-03-30T13:11:57.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f1effd640 1 -- 172.21.15.66:0/2152187874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34022b80 con 0x558953181ae0 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2152187874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 msgr2=0x5589531abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 0x5589531abe80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3f34012790 tx=0x7f3f34024000 comp rx=0 tx=0).stop 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2152187874 shutdown_connections 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x5589531bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55895318c570 0x5589531ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.585+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2152187874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558953181ae0 0x5589531abe80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.586+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2152187874 >> 172.21.15.66:0/2152187874 conn(0x55895315fa50 msgr2=0x558953181060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.586+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2152187874 shutdown_connections 2024-03-30T13:11:57.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.586+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2152187874 wait complete. 2024-03-30T13:11:57.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.586+0000 7f3f46c0d740 1 -- start start 2024-03-30T13:11:57.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.586+0000 7f3f46c0d740 1 Processor -- start 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x5589531ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x5589531adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953225d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5589531c4670 con 0x55895318c570 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c200050 con 0x558953181ae0 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558953226f90 con 0x55895318c950 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f46c0d740 10 client.0 _ll_register_callbacks cb 0x7f3f340129a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x5589531adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x5589531adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56948/0 (socket says 172.21.15.66:56948) 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 -- 172.21.15.66:0/2962537587 learned_addr learned my addr 172.21.15.66:0/2962537587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x5589531ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f425bf640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953225d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 -- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 msgr2=0x558953225d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953225d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 -- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 msgr2=0x5589531ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.587+0000 7f3f415bd640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x5589531ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f415bd640 1 -- 172.21.15.66:0/2962537587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895315f3e0 con 0x55895318c570 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f415bd640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x5589531adcf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x558953190020 tx=0x7f3f2c017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec39e5cde95aeae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f1cff9640 1 -- 172.21.15.66:0/2962537587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c004030 con 0x55895318c570 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f1cff9640 1 -- 172.21.15.66:0/2962537587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3f2c0041d0 con 0x55895318c570 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f34026030 con 0x55895318c570 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f1cff9640 1 -- 172.21.15.66:0/2962537587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3f2c034050 con 0x55895318c570 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 10 client.5691 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558953226cd0 con 0x55895318c570 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f1cff9640 1 -- 172.21.15.66:0/2962537587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34026030 con 0x55895318c570 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f1cff9640 1 -- 172.21.15.66:0/2962537587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f3f2c017940 con 0x55895318c570 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 10 client.5691 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 10 client.5691 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 20 client.5691 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531bd170 0x5589532262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 4 client.5691 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511627780}] 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.588+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511627780}) -- 0x5589530d0e20 con 0x5589531bd170 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531bd170 0x5589532262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f1cff9640 10 client.5691 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f41dbe640 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531bd170 0x5589532262a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76fa6d520a5651fd server_cookie=4619ece987bb4680 in_seq=0 out_seq=0 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f1cff9640 1 -- 172.21.15.66:0/2962537587 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4356 (crc 0 0 0) 0x5589530d0e20 con 0x5589531bd170 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f1cff9640 10 client.5691 handle_command_reply: tid=0 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 client.5691 shutdown 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 msgr2=0x5589531adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x5589531adcf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x558953190020 tx=0x7f3f2c017730 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 shutdown_connections 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5589531bd170 0x5589532262a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55895318c950 0x558953225d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55895318c570 0x5589531adcf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f1cff9640 0 client.5691 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 --2- 172.21.15.66:0/2962537587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558953181ae0 0x5589531ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 >> 172.21.15.66:0/2962537587 conn(0x55895315fa50 msgr2=0x55895315a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.589+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 shutdown_connections 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f46c0d740 1 -- 172.21.15.66:0/2962537587 wait complete. 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f46c0d740 20 client.5691 trim_cache size 0 max 16384 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/Kconfig", 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627780, 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.757987+0000", 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.283203+0000", 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1481, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.283203+0000", 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1481, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.757987+0000" 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1481, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.757987+0000" 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 111, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 2, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:57.609 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:57.610 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyrstat": 0, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:57.612 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.613 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:57.614 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.617 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:57.617 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:57.617 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:57.617 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.617 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.618 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.619 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:57.620 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:57.621 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3f2006c560 msgr2=0x7f3f2006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3f2006c560 0x7f3f2006ea20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0d51f0 tx=0x7f3f34009040 comp rx=0 tx=0).stop 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 msgr2=0x7f3f3c160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 0x7f3f3c160f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f3802cc70 tx=0x7f3f3c0d31c0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 shutdown_connections 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3f2006c560 0x7f3f2006ea20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f3c15c9b0 0x7f3f3c160f20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c15c530 0x7f3f3c0aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 --2- 172.21.15.66:0/28384026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f280051c0 0x7f3f3c15eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 >> 172.21.15.66:0/28384026 conn(0x7f3f3c0678f0 msgr2=0x7f3f3c0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 shutdown_connections 2024-03-30T13:11:57.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.590+0000 7f3f44022640 1 -- 172.21.15.66:0/28384026 wait complete. 2024-03-30T13:11:57.624 DEBUG:tasks.cephfs.test_uninlining:getting inode info #2 of 23:1099511628281 2024-03-30T13:11:57.624 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628281 2024-03-30T13:11:57.624 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628281 2024-03-30T13:11:57.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.731+0000 7f55d2daf640 1 Processor -- start 2024-03-30T13:11:57.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.731+0000 7f55d2daf640 1 -- start start 2024-03-30T13:11:57.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.732+0000 7f55d2daf640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f55cc0d0ad0 0x7f55cc15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.732+0000 7f55d2daf640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55cc0b5ff0 con 0x7f55cc155b10 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.732+0000 7f55d2daf640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55cc0b41b0 con 0x7f55cc0b4c90 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.732+0000 7f55d2daf640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f55cc155ef0 con 0x7f55cc0d0ad0 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55d0b4b640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f55cc155b10 0x7f55cc0b57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37468/0 (socket says 172.21.15.66:37468) 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55d0b4b640 1 -- 172.21.15.66:0/314773612 learned_addr learned my addr 172.21.15.66:0/314773612 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55d134c640 1 --2- 172.21.15.66:0/314773612 >> v2:172.21.15.117:3300/0 conn(0x7f55cc0d0ad0 0x7f55cc15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773409923 0 0) 0x7f55cc0b5ff0 con 0x7f55cc155b10 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55bc005730 con 0x7f55cc155b10 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2297999205 0 0) 0x7f55bc005730 con 0x7f55cc155b10 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 >> v2:172.21.15.117:3300/0 conn(0x7f55cc0d0ad0 msgr2=0x7f55cc15fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 --2- 172.21.15.66:0/314773612 >> v2:172.21.15.117:3300/0 conn(0x7f55cc0d0ad0 0x7f55cc15fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 >> v1:172.21.15.117:6789/0 conn(0x7f55cc0b4c90 legacy=0x7f55cc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc0b4490 con 0x7f55cc155b10 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f55c0001670 con 0x7f55cc155b10 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 >> v1:172.21.15.66:6789/0 conn(0x7f55cc155b10 legacy=0x7f55cc0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55bc005bc0 0x7f55bc005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55bc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0073d0 0x7f55bc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00e2f0 con 0x7f55bc006a10 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00e4a0 con 0x7f55bc0073d0 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc00e6a0 con 0x7f55bc005bc0 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.733+0000 7f55cbfff640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55bc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55bc005bc0 0x7f55bc005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55d134c640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0073d0 0x7f55bc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55cbfff640 1 -- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55bc005bc0 msgr2=0x7f55bc005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55cbfff640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55bc005bc0 0x7f55bc005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55cbfff640 1 -- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0073d0 msgr2=0x7f55bc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55cbfff640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc0073d0 0x7f55bc00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55cbfff640 1 -- 172.21.15.66:0/314773612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc005730 con 0x7f55bc006a10 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55d134c640 1 -- 172.21.15.66:0/314773612 reap_dead start 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.734+0000 7f55cbfff640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55bc006e90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f55b8001750 tx=0x7f55bc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e45908bd915da5c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8001370 con 0x7f55bc006a10 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b80052d0 con 0x7f55bc006a10 2024-03-30T13:11:57.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55cb7fe640 1 -- 172.21.15.66:0/314773612 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8003830 con 0x7f55bc006a10 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 -- 172.21.15.66:0/314773612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 msgr2=0x7f55bc006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55bc006e90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f55b8001750 tx=0x7f55bc007a40 comp rx=0 tx=0).stop 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 -- 172.21.15.66:0/314773612 shutdown_connections 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 --2- 172.21.15.66:0/314773612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55bc006e90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 -- 172.21.15.66:0/314773612 >> 172.21.15.66:0/314773612 conn(0x7f55cc0d1160 msgr2=0x7f55cc0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 -- 172.21.15.66:0/314773612 shutdown_connections 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.735+0000 7f55d2daf640 1 -- 172.21.15.66:0/314773612 wait complete. 2024-03-30T13:11:57.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.736+0000 7f55d2daf640 1 Processor -- start 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 -- start start 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55cc14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc1f08e0 0x7f55cc0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc152470 0x7f55cc1528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc15fae0 con 0x7f55bc006a10 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0b62d0 con 0x7f55cc152470 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.737+0000 7f55d2daf640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc15fc60 con 0x7f55cc1f08e0 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55cc14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55cc14be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56966/0 (socket says 172.21.15.66:56966) 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 -- 172.21.15.66:0/1531813998 learned_addr learned my addr 172.21.15.66:0/1531813998 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc1f08e0 0x7f55cc0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d134c640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc152470 0x7f55cc1528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 -- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc1f08e0 msgr2=0x7f55cc0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc1f08e0 0x7f55cc0b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 -- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc152470 msgr2=0x7f55cc1528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc152470 0x7f55cc1528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 -- 172.21.15.66:0/1531813998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc14cfc0 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55cc14be50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f55c00072c0 tx=0x7f55c0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55c97fa640 1 -- 172.21.15.66:0/1531813998 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000f860 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55c97fa640 1 -- 172.21.15.66:0/1531813998 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c0018070 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc14d260 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55c97fa640 1 -- 172.21.15.66:0/1531813998 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000fc60 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.738+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc14d820 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.740+0000 7f55c97fa640 1 -- 172.21.15.66:0/1531813998 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f55c001d460 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.740+0000 7f55c97fa640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f55b006c530 0x7f55b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.740+0000 7f55c97fa640 1 -- 172.21.15.66:0/1531813998 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f55c008b8c0 con 0x7f55bc006a10 2024-03-30T13:11:57.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.740+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f55b006c530 0x7f55b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.741+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f55b006c530 0x7f55b006e9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55b8033000 tx=0x7f55b8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 Processor -- start 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 -- start start 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df35fff0 0x5565df35dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 0x5565df317620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df33f930 0x5565df317b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5565df27c590 con 0x5565df3603d0 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565df21ad30 con 0x5565df35fff0 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.742+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5565df278540 con 0x5565df33f930 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 0x5565df317620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 0x5565df317620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56974/0 (socket says 172.21.15.66:56974) 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 -- 172.21.15.66:0/1596008695 learned_addr learned my addr 172.21.15.66:0/1596008695 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55d134c640 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df33f930 0x5565df317b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df35fff0 0x5565df35dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 -- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df33f930 msgr2=0x5565df317b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df33f930 0x5565df317b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 -- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df35fff0 msgr2=0x5565df35dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df35fff0 0x5565df35dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 -- 172.21.15.66:0/1596008695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565df05c920 con 0x5565df3603d0 2024-03-30T13:11:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 0x5565df317620 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f55b8001840 tx=0x7f55b802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be9ebce654a8c21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55b67fc640 1 -- 172.21.15.66:0/1596008695 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8042070 con 0x5565df3603d0 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55b67fc640 1 -- 172.21.15.66:0/1596008695 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b8043040 con 0x5565df3603d0 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.743+0000 7f55b67fc640 1 -- 172.21.15.66:0/1596008695 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80067d0 con 0x5565df3603d0 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 -- 172.21.15.66:0/1596008695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 msgr2=0x5565df317620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 0x5565df317620 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f55b8001840 tx=0x7f55b802f000 comp rx=0 tx=0).stop 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 -- 172.21.15.66:0/1596008695 shutdown_connections 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df33f930 0x5565df317b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df3603d0 0x5565df317620 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 --2- 172.21.15.66:0/1596008695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df35fff0 0x5565df35dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 -- 172.21.15.66:0/1596008695 >> 172.21.15.66:0/1596008695 conn(0x5565df314780 msgr2=0x5565df35d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 -- 172.21.15.66:0/1596008695 shutdown_connections 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 -- 172.21.15.66:0/1596008695 wait complete. 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.744+0000 7f55d599d740 1 -- start start 2024-03-30T13:11:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 Processor -- start 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df32f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df3603d0 0x5565df33ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df374f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5565df312db0 con 0x5565df33cd80 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565df2189f0 con 0x5565df3603d0 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5565df316bb0 con 0x5565df35fff0 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.745+0000 7f55d599d740 10 client.0 _ll_register_callbacks cb 0x5565df21cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df374f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df374f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56988/0 (socket says 172.21.15.66:56988) 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 -- 172.21.15.66:0/1503966290 learned_addr learned my addr 172.21.15.66:0/1503966290 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df32f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df3603d0 0x5565df33ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 -- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 msgr2=0x5565df32f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df32f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 -- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df3603d0 msgr2=0x5565df33ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df3603d0 0x5565df33ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 -- 172.21.15.66:0/1503966290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565def50210 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df32f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d134c640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df374f10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fa0 tx=0x7f55c40044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac36b8ffc8ca1a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1503966290 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c401c020 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1503966290 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f55c4007820 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.746+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b8043040 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1503966290 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f55c4026050 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 10 client.5703 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5565df0bd0b0 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1503966290 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b8043040 con 0x5565df33cd80 2024-03-30T13:11:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1503966290 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f55c4029020 con 0x5565df33cd80 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 10 client.5703 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 10 client.5703 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 20 client.5703 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df334240 0x5565df35edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 4 client.5703 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565df289ad0 con 0x5565df334240 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df334240 0x5565df35edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55ab7fe640 10 client.5703 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.747+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df334240 0x5565df35edb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6941a92b06c0207 server_cookie=f878970388264284 in_seq=0 out_seq=0 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1503966290 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5565df289ad0 con 0x5565df334240 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55ab7fe640 10 client.5703 handle_command_reply: tid=0 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 client.5703 shutdown 2024-03-30T13:11:57.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 msgr2=0x5565df374f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df374f10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fa0 tx=0x7f55c40044d0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 shutdown_connections 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df334240 0x5565df35edb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df374f10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df3603d0 0x5565df33ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 --2- 172.21.15.66:0/1503966290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df32f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 >> 172.21.15.66:0/1503966290 conn(0x5565df33f930 msgr2=0x5565df3170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55ab7fe640 0 client.5703 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 shutdown_connections 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 1 -- 172.21.15.66:0/1503966290 wait complete. 2024-03-30T13:11:57.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.748+0000 7f55d599d740 20 client.5703 trim_cache size 0 max 16384 2024-03-30T13:11:57.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.763+0000 7f55d599d740 1 Processor -- start 2024-03-30T13:11:57.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.763+0000 7f55d599d740 1 -- start start 2024-03-30T13:11:57.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33f930 0x5565df385540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d599d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df3339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df316280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d599d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1fd2f0 con 0x5565df33cd80 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1fc2d0 con 0x5565df33f930 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc1fb660 con 0x5565df35fff0 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df3339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df3339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57002/0 (socket says 172.21.15.66:57002) 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 -- 172.21.15.66:0/1700461203 learned_addr learned my addr 172.21.15.66:0/1700461203 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55d134c640 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df316280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 -- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 msgr2=0x5565df316280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df316280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 -- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33f930 msgr2=0x5565df385540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33f930 0x5565df385540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 -- 172.21.15.66:0/1700461203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565def292d0 con 0x5565df33cd80 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df3339e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f55b804b040 tx=0x7f55b803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a424e468538df39f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55b5ffb640 1 -- 172.21.15.66:0/1700461203 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8056070 con 0x5565df33cd80 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55b5ffb640 1 -- 172.21.15.66:0/1700461203 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b8005260 con 0x5565df33cd80 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.764+0000 7f55b5ffb640 1 -- 172.21.15.66:0/1700461203 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8042040 con 0x5565df33cd80 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 -- 172.21.15.66:0/1700461203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 msgr2=0x5565df3339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df3339e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f55b804b040 tx=0x7f55b803e000 comp rx=0 tx=0).stop 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 -- 172.21.15.66:0/1700461203 shutdown_connections 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df316280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33cd80 0x5565df3339e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 --2- 172.21.15.66:0/1700461203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33f930 0x5565df385540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 -- 172.21.15.66:0/1700461203 >> 172.21.15.66:0/1700461203 conn(0x5565df32f420 msgr2=0x5565df3570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 -- 172.21.15.66:0/1700461203 shutdown_connections 2024-03-30T13:11:57.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.765+0000 7f55d599d740 1 -- 172.21.15.66:0/1700461203 wait complete. 2024-03-30T13:11:57.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.766+0000 7f55d599d740 1 -- start start 2024-03-30T13:11:57.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.766+0000 7f55d599d740 1 Processor -- start 2024-03-30T13:11:57.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.766+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33cd80 0x5565df318130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 0x5565df318670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df311a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5565df35fde0 con 0x5565df33f930 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1fbcb0 con 0x5565df33cd80 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5565df201860 con 0x5565df35fff0 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 10 client.0 _ll_register_callbacks cb 0x7f55c4009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 0x5565df318670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 0x5565df318670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57008/0 (socket says 172.21.15.66:57008) 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 -- 172.21.15.66:0/1801500349 learned_addr learned my addr 172.21.15.66:0/1801500349 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33cd80 0x5565df318130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d134c640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df311a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 -- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 msgr2=0x5565df311a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df311a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 -- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33cd80 msgr2=0x5565df318130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33cd80 0x5565df318130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 -- 172.21.15.66:0/1801500349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565df3465c0 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55cbfff640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 0x5565df318670 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f55b8007810 tx=0x7f55b8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154705a17e699ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1801500349 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8056070 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1801500349 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f55b804e040 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.767+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b8005260 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1801500349 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f55b8042040 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 10 client.5709 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5565df311540 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1801500349 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b8005260 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1801500349 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f55b8006ae0 con 0x5565df33f930 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 10 client.5709 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 10 client.5709 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 20 client.5709 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:57.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df3603d0 0x5565df311f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 4 client.5709 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628281}] 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628281}) -- 0x5565df283e20 con 0x5565df3603d0 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df3603d0 0x5565df311f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55d0b4b640 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df3603d0 0x5565df311f60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71360c2461d7ec6c server_cookie=f6e00fd104e054c8 in_seq=0 out_seq=0 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.768+0000 7f55ab7fe640 10 client.5709 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55ab7fe640 1 -- 172.21.15.66:0/1801500349 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4354 (crc 0 0 0) 0x5565df283e20 con 0x5565df3603d0 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55ab7fe640 10 client.5709 handle_command_reply: tid=0 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 client.5709 shutdown 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 msgr2=0x5565df318670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 0x5565df318670 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f55b8007810 tx=0x7f55b8001190 comp rx=0 tx=0).stop 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 shutdown_connections 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5565df3603d0 0x5565df311f60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5565df35fff0 0x5565df311a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5565df33f930 0x5565df318670 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 --2- 172.21.15.66:0/1801500349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565df33cd80 0x5565df318130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 >> 172.21.15.66:0/1801500349 conn(0x5565df32f420 msgr2=0x5565df3570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55ab7fe640 0 client.5709 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 shutdown_connections 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 1 -- 172.21.15.66:0/1801500349 wait complete. 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d599d740 20 client.5709 trim_cache size 0 max 16384 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:57.779 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/Makefile", 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628281, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.797590+0000", 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.284203+0000", 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:57.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 356, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.284203+0000", 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:11:57.783 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.797590+0000" 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.797590+0000" 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 160, 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 4, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:57.786 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:57.787 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyrstat": 0, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.788 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.790 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:57.791 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:57.792 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:57.793 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:57.793 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:57.793 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f55b006c530 msgr2=0x7f55b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d2daf640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f55b006c530 0x7f55b006e9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55b8033000 tx=0x7f55b8009030 comp rx=0 tx=0).stop 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 msgr2=0x7f55cc14be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.769+0000 7f55d2daf640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55cc14be50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f55c00072c0 tx=0x7f55c0007780 comp rx=0 tx=0).stop 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 shutdown_connections 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f55b006c530 0x7f55b006e9f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc152470 0x7f55cc1528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc1f08e0 0x7f55cc0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 --2- 172.21.15.66:0/1531813998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55bc006a10 0x7f55cc14be50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 >> 172.21.15.66:0/1531813998 conn(0x7f55cc0d1160 msgr2=0x7f55cc154020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 shutdown_connections 2024-03-30T13:11:57.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.770+0000 7f55d2daf640 1 -- 172.21.15.66:0/1531813998 wait complete. 2024-03-30T13:11:57.795 DEBUG:tasks.cephfs.test_uninlining:getting inode info #3 of 23:1099511628287 2024-03-30T13:11:57.795 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628287 2024-03-30T13:11:57.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628287 2024-03-30T13:11:57.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.906+0000 7f873cf04640 1 Processor -- start 2024-03-30T13:11:57.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.906+0000 7f873cf04640 1 -- start start 2024-03-30T13:11:57.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f873cf04640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8738144cb0 0x7f87380b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f873cf04640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f873815f800 0x7f8738072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f873cf04640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87380b35e0 con 0x7f8738072b80 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f873cf04640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8738072fd0 con 0x7f873815f800 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f873cf04640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f87380b3e80 con 0x7f8738144cb0 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f8735d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f873815f800 0x7f8738072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f8735d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f873815f800 0x7f8738072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57018/0 (socket says 172.21.15.66:57018) 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.908+0000 7f8735d9b640 1 -- 172.21.15.66:0/1011454649 learned_addr learned my addr 172.21.15.66:0/1011454649 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.909+0000 7f873659c640 1 --2- 172.21.15.66:0/1011454649 >> v2:172.21.15.117:3300/0 conn(0x7f8738144cb0 0x7f87380b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.909+0000 7f8735d9b640 1 -- 172.21.15.66:0/1011454649 >> v2:172.21.15.117:3300/0 conn(0x7f8738144cb0 msgr2=0x7f87380b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.909+0000 7f8735d9b640 1 --2- 172.21.15.66:0/1011454649 >> v2:172.21.15.117:3300/0 conn(0x7f8738144cb0 0x7f87380b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.909+0000 7f8735d9b640 1 -- 172.21.15.66:0/1011454649 >> v1:172.21.15.117:6789/0 conn(0x7f8738072b80 legacy=0x7f8738071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.909+0000 7f8735d9b640 1 -- 172.21.15.66:0/1011454649 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87380b38c0 con 0x7f873815f800 2024-03-30T13:11:57.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.909+0000 7f8735d9b640 1 --2- 172.21.15.66:0/1011454649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873815f800 0x7f8738072640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f873806e3e0 tx=0x7f872002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e697e9a0649306d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873559a640 1 -- 172.21.15.66:0/1011454649 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720035020 con 0x7f873815f800 2024-03-30T13:11:57.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873559a640 1 -- 172.21.15.66:0/1011454649 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8720031870 con 0x7f873815f800 2024-03-30T13:11:57.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873559a640 1 -- 172.21.15.66:0/1011454649 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720031c40 con 0x7f873815f800 2024-03-30T13:11:57.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 -- 172.21.15.66:0/1011454649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873815f800 msgr2=0x7f8738072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 --2- 172.21.15.66:0/1011454649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873815f800 0x7f8738072640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f873806e3e0 tx=0x7f872002b910 comp rx=0 tx=0).stop 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 -- 172.21.15.66:0/1011454649 shutdown_connections 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 --2- 172.21.15.66:0/1011454649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f873815f800 0x7f8738072640 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 --2- 172.21.15.66:0/1011454649 >> v2:172.21.15.117:3300/0 conn(0x7f8738144cb0 0x7f87380b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 -- 172.21.15.66:0/1011454649 >> 172.21.15.66:0/1011454649 conn(0x7f87380b5b00 msgr2=0x7f873806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 -- 172.21.15.66:0/1011454649 shutdown_connections 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.910+0000 7f873cf04640 1 -- 172.21.15.66:0/1011454649 wait complete. 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.912+0000 7f873cf04640 1 Processor -- start 2024-03-30T13:11:57.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.913+0000 7f873cf04640 1 -- start start 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873cf04640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 0x7f8738139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873cf04640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8738144cb0 0x7f8738136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873cf04640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873815f800 0x7f87381371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873cf04640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f873815fc40 con 0x7f8738072b80 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873cf04640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87380cdb30 con 0x7f873815f800 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873cf04640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f87380cf060 con 0x7f8738144cb0 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 0x7f8738139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 0x7f8738139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57026/0 (socket says 172.21.15.66:57026) 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 -- 172.21.15.66:0/4083609472 learned_addr learned my addr 172.21.15.66:0/4083609472 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f8736d9d640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873815f800 0x7f87381371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f8735d9b640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8738144cb0 0x7f8738136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 -- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8738144cb0 msgr2=0x7f8738136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8738144cb0 0x7f8738136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 -- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873815f800 msgr2=0x7f87381371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873815f800 0x7f87381371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.914+0000 7f873659c640 1 -- 172.21.15.66:0/4083609472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8738155420 con 0x7f8738072b80 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.915+0000 7f873659c640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 0x7f8738139940 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f872800d830 tx=0x7f872800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.915+0000 7f872effd640 1 -- 172.21.15.66:0/4083609472 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872800b6c0 con 0x7f8738072b80 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.915+0000 7f872effd640 1 -- 172.21.15.66:0/4083609472 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728019070 con 0x7f8738072b80 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.915+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8738154b80 con 0x7f8738072b80 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.915+0000 7f872effd640 1 -- 172.21.15.66:0/4083609472 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872800ba80 con 0x7f8738072b80 2024-03-30T13:11:57.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.915+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8738155140 con 0x7f8738072b80 2024-03-30T13:11:57.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.916+0000 7f872effd640 1 -- 172.21.15.66:0/4083609472 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f872801e460 con 0x7f8738072b80 2024-03-30T13:11:57.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.917+0000 7f872effd640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f871806c530 0x7f871806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.917+0000 7f8735d9b640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f871806c530 0x7f871806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.917+0000 7f872effd640 1 -- 172.21.15.66:0/4083609472 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f872808be60 con 0x7f8738072b80 2024-03-30T13:11:57.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.918+0000 7f8735d9b640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f871806c530 0x7f871806e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8720036000 tx=0x7f872003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.918+0000 7f873fbca740 1 Processor -- start 2024-03-30T13:11:57.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.918+0000 7f873fbca740 1 -- start start 2024-03-30T13:11:57.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6cf2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f873fbca740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 0x5590c6d22300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6cf2a50 0x5590c6d22840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f873fbca740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5590c6c5c590 con 0x5590c6d1f950 2024-03-30T13:11:57.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5590c6bfad30 con 0x5590c6cf2a50 2024-03-30T13:11:57.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5590c6c58540 con 0x5590c6d1f570 2024-03-30T13:11:57.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 0x5590c6d22300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 0x5590c6d22300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57042/0 (socket says 172.21.15.66:57042) 2024-03-30T13:11:57.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 -- 172.21.15.66:0/284112649 learned_addr learned my addr 172.21.15.66:0/284112649 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 -- 172.21.15.66:0/284112649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 msgr2=0x5590c6cf2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6cf2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 -- 172.21.15.66:0/284112649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6cf2a50 msgr2=0x5590c6d22840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6cf2a50 0x5590c6d22840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 -- 172.21.15.66:0/284112649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c6a3c920 con 0x5590c6d1f950 2024-03-30T13:11:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8735d9b640 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 0x5590c6d22300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8720009960 tx=0x7f8720030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b476a784e218280 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8717fff640 1 -- 172.21.15.66:0/284112649 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720054070 con 0x5590c6d1f950 2024-03-30T13:11:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8717fff640 1 -- 172.21.15.66:0/284112649 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f872003aa70 con 0x5590c6d1f950 2024-03-30T13:11:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.919+0000 7f8717fff640 1 -- 172.21.15.66:0/284112649 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720035040 con 0x5590c6d1f950 2024-03-30T13:11:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 -- 172.21.15.66:0/284112649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 msgr2=0x5590c6d22300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 0x5590c6d22300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8720009960 tx=0x7f8720030000 comp rx=0 tx=0).stop 2024-03-30T13:11:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 -- 172.21.15.66:0/284112649 shutdown_connections 2024-03-30T13:11:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6cf2a50 0x5590c6d22840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f950 0x5590c6d22300 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 --2- 172.21.15.66:0/284112649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6cf2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 -- 172.21.15.66:0/284112649 >> 172.21.15.66:0/284112649 conn(0x5590c6cf4780 msgr2=0x5590c6d3d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 -- 172.21.15.66:0/284112649 shutdown_connections 2024-03-30T13:11:57.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.920+0000 7f873fbca740 1 -- 172.21.15.66:0/284112649 wait complete. 2024-03-30T13:11:57.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.921+0000 7f873fbca740 1 -- start start 2024-03-30T13:11:57.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.921+0000 7f873fbca740 1 Processor -- start 2024-03-30T13:11:57.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6d25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d50c90 0x5590c6d24950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5590c6cf6e20 con 0x5590c6d1f570 2024-03-30T13:11:57.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5590c6bf89f0 con 0x5590c6d1f950 2024-03-30T13:11:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5590c6cf9660 con 0x5590c6d50c90 2024-03-30T13:11:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873fbca740 10 client.0 _ll_register_callbacks cb 0x5590c6bfcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d4e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57052/0 (socket says 172.21.15.66:57052) 2024-03-30T13:11:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 -- 172.21.15.66:0/378212117 learned_addr learned my addr 172.21.15.66:0/378212117 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f8735d9b640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6d25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f8736d9d640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d50c90 0x5590c6d24950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 -- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d50c90 msgr2=0x5590c6d24950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d50c90 0x5590c6d24950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 -- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 msgr2=0x5590c6d25f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6d25f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 -- 172.21.15.66:0/378212117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c6930210 con 0x5590c6d1f570 2024-03-30T13:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f8735d9b640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6d25f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f8736d9d640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d50c90 0x5590c6d24950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f873659c640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d4e120 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8728056050 tx=0x7f8728059d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172d6c3971830e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f87157fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.922+0000 7f87157fa640 1 -- 172.21.15.66:0/378212117 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728089060 con 0x5590c6d1f570 2024-03-30T13:11:57.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f87157fa640 1 -- 172.21.15.66:0/378212117 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f8728083b40 con 0x5590c6d1f570 2024-03-30T13:11:57.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872003aa70 con 0x5590c6d1f570 2024-03-30T13:11:57.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f87157fa640 1 -- 172.21.15.66:0/378212117 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8728078050 con 0x5590c6d1f570 2024-03-30T13:11:57.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 10 client.5721 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:57.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5590c6a9d0b0 con 0x5590c6d1f570 2024-03-30T13:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f87157fa640 1 -- 172.21.15.66:0/378212117 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f872003aa70 con 0x5590c6d1f570 2024-03-30T13:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f87157fa640 1 -- 172.21.15.66:0/378212117 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f8728013070 con 0x5590c6d1f570 2024-03-30T13:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 10 client.5721 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 10 client.5721 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 20 client.5721 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:57.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d217b0 0x5590c6d234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 4 client.5721 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590c6c69ad0 con 0x5590c6d217b0 2024-03-30T13:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f8735d9b640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d217b0 0x5590c6d234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f8735d9b640 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d217b0 0x5590c6d234a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d675c6ac439e20 server_cookie=cb715864cf025bb8 in_seq=0 out_seq=0 2024-03-30T13:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.923+0000 7f87157fa640 10 client.5721 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f87157fa640 1 -- 172.21.15.66:0/378212117 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5590c6c69ad0 con 0x5590c6d217b0 2024-03-30T13:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f87157fa640 10 client.5721 handle_command_reply: tid=0 2024-03-30T13:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 client.5721 shutdown 2024-03-30T13:11:57.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 msgr2=0x5590c6d4e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d4e120 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8728056050 tx=0x7f8728059d30 comp rx=0 tx=0).stop 2024-03-30T13:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 shutdown_connections 2024-03-30T13:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d217b0 0x5590c6d234a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d50c90 0x5590c6d24950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6d25f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f87157fa640 0 client.5721 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 --2- 172.21.15.66:0/378212117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d4e120 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 >> 172.21.15.66:0/378212117 conn(0x5590c6cf2a50 msgr2=0x5590c6cf6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 shutdown_connections 2024-03-30T13:11:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 1 -- 172.21.15.66:0/378212117 wait complete. 2024-03-30T13:11:57.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.924+0000 7f873fbca740 20 client.5721 trim_cache size 0 max 16384 2024-03-30T13:11:57.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.939+0000 7f873fbca740 1 Processor -- start 2024-03-30T13:11:57.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.939+0000 7f873fbca740 1 -- start start 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873fbca740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d25e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f950 0x5590c6cf1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d50c90 0x5590c6d23300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873fbca740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87381fef00 con 0x5590c6d1f570 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87381fdf30 con 0x5590c6d50c90 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8738143af0 con 0x5590c6d1f950 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d25e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d25e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57060/0 (socket says 172.21.15.66:57060) 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f873659c640 1 -- 172.21.15.66:0/2174627755 learned_addr learned my addr 172.21.15.66:0/2174627755 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f8736d9d640 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d50c90 0x5590c6d23300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.940+0000 7f8735d9b640 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f950 0x5590c6cf1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f873659c640 1 -- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f950 msgr2=0x5590c6cf1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f873659c640 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f950 0x5590c6cf1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f873659c640 1 -- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d50c90 msgr2=0x5590c6d23300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f873659c640 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d50c90 0x5590c6d23300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f873659c640 1 -- 172.21.15.66:0/2174627755 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c69092d0 con 0x5590c6d1f570 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f873659c640 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d25e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8728059b80 tx=0x7f872805d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2c63039982ce82 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f87177fe640 1 -- 172.21.15.66:0/2174627755 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872808a070 con 0x5590c6d1f570 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f87177fe640 1 -- 172.21.15.66:0/2174627755 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728083b40 con 0x5590c6d1f570 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.941+0000 7f87177fe640 1 -- 172.21.15.66:0/2174627755 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728059280 con 0x5590c6d1f570 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 -- 172.21.15.66:0/2174627755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 msgr2=0x5590c6d25e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d25e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8728059b80 tx=0x7f872805d000 comp rx=0 tx=0).stop 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 -- 172.21.15.66:0/2174627755 shutdown_connections 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d50c90 0x5590c6d23300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f950 0x5590c6cf1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 --2- 172.21.15.66:0/2174627755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d1f570 0x5590c6d25e10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 -- 172.21.15.66:0/2174627755 >> 172.21.15.66:0/2174627755 conn(0x5590c6cf2a50 msgr2=0x5590c6d4dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 -- 172.21.15.66:0/2174627755 shutdown_connections 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 -- 172.21.15.66:0/2174627755 wait complete. 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 -- start start 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.942+0000 7f873fbca740 1 Processor -- start 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6d3fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6db8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 0x5590c6d239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5590c6d24cb0 con 0x5590c6d50c90 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8738144140 con 0x5590c6d1f950 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5590c6d50ab0 con 0x5590c6d1f570 2024-03-30T13:11:57.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f873fbca740 10 client.0 _ll_register_callbacks cb 0x7f8728055e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f8736d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 0x5590c6d239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f8736d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 0x5590c6d239c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57070/0 (socket says 172.21.15.66:57070) 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.943+0000 7f8736d9d640 1 -- 172.21.15.66:0/4208694587 learned_addr learned my addr 172.21.15.66:0/4208694587 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8735d9b640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6db8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f873659c640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6d3fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8736d9d640 1 -- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 msgr2=0x5590c6d3fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8736d9d640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6d3fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8736d9d640 1 -- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 msgr2=0x5590c6db8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8736d9d640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6db8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8736d9d640 1 -- 172.21.15.66:0/4208694587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c6d4e0a0 con 0x5590c6d50c90 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f8736d9d640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 0x5590c6d239c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f87300138f0 tx=0x7f8730013db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6107aac7f89d1ff0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f87157fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f87157fa640 1 -- 172.21.15.66:0/4208694587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873001c070 con 0x5590c6d50c90 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f87157fa640 1 -- 172.21.15.66:0/4208694587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f8730011960 con 0x5590c6d50c90 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8728083b40 con 0x5590c6d50c90 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f87157fa640 1 -- 172.21.15.66:0/4208694587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8730027050 con 0x5590c6d50c90 2024-03-30T13:11:57.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f873fbca740 10 client.5727 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5590c6d40360 con 0x5590c6d50c90 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.944+0000 7f87157fa640 1 -- 172.21.15.66:0/4208694587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728083b40 con 0x5590c6d50c90 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f87157fa640 1 -- 172.21.15.66:0/4208694587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f873002a020 con 0x5590c6d50c90 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 10 client.5727 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 10 client.5727 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 20 client.5727 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d23f00 0x5590c6d3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 4 client.5727 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628287}] 2024-03-30T13:11:57.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628287}) -- 0x5590c6c63e20 con 0x5590c6d23f00 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873659c640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d23f00 0x5590c6d3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873659c640 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d23f00 0x5590c6d3e4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5492697b5e4e0d8c server_cookie=779e448010cf0ff1 in_seq=0 out_seq=0 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f87157fa640 10 client.5727 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f87157fa640 1 -- 172.21.15.66:0/4208694587 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4358 (crc 0 0 0) 0x5590c6c63e20 con 0x5590c6d23f00 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f87157fa640 10 client.5727 handle_command_reply: tid=0 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 1 client.5727 shutdown 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 msgr2=0x5590c6d239c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.945+0000 7f873fbca740 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 0x5590c6d239c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f87300138f0 tx=0x7f8730013db0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 shutdown_connections 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5590c6d23f00 0x5590c6d3e4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5590c6d50c90 0x5590c6d239c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5590c6d1f950 0x5590c6db8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f87157fa640 0 client.5727 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 --2- 172.21.15.66:0/4208694587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5590c6d1f570 0x5590c6d3fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 >> 172.21.15.66:0/4208694587 conn(0x5590c6cf2a50 msgr2=0x5590c6d3d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 shutdown_connections 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 1 -- 172.21.15.66:0/4208694587 wait complete. 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873fbca740 20 client.5727 trim_cache size 0 max 16384 2024-03-30T13:11:57.970 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:57.970 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/ceph_frag.c", 2024-03-30T13:11:57.970 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628287, 2024-03-30T13:11:57.970 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:57.970 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.820066+0000", 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.291203+0000", 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:57.972 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:57.973 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:57.973 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:57.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.973 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:57.973 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 401, 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.291203+0000", 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:57.977 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:57.977 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:57.977 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:57.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.977 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:57.977 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:57.978 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:57.978 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:57.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.978 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:57.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.978 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 401, 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.820066+0000" 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 401, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.820066+0000" 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 174, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 16, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:57.983 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:57.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.983 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:57.983 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:57.984 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:57.984 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:57.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.984 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:57.984 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:57.986 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyrstat": 0, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.988 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.989 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:57.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:57.991 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:57.992 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:57.992 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:57.992 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:57.992 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:57.992 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f871806c530 msgr2=0x7f871806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f871806c530 0x7f871806e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8720036000 tx=0x7f872003c040 comp rx=0 tx=0).stop 2024-03-30T13:11:57.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 msgr2=0x7f8738139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 0x7f8738139940 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f872800d830 tx=0x7f872800dcf0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 shutdown_connections 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f871806c530 0x7f871806e9f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873815f800 0x7f87381371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8738144cb0 0x7f8738136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 --2- 172.21.15.66:0/4083609472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8738072b80 0x7f8738139940 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 >> 172.21.15.66:0/4083609472 conn(0x7f87380b5b00 msgr2=0x7f873813a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 shutdown_connections 2024-03-30T13:11:57.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:57.946+0000 7f873cf04640 1 -- 172.21.15.66:0/4083609472 wait complete. 2024-03-30T13:11:57.994 DEBUG:tasks.cephfs.test_uninlining:getting inode info #4 of 23:1099511628294 2024-03-30T13:11:57.995 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628294 2024-03-30T13:11:57.995 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628294 2024-03-30T13:11:58.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.145+0000 7fdb33688640 1 Processor -- start 2024-03-30T13:11:58.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.145+0000 7fdb33688640 1 -- start start 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb33688640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb2c0ba090 0x7fdb2c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb33688640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb2c0b73d0 0x7fdb2c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb33688640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2c0d2ff0 con 0x7fdb2c0b5dc0 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb33688640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fdb2c0d32d0 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb33688640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fdb2c0d35b0 con 0x7fdb2c0ba090 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb2c0b73d0 0x7fdb2c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb2c0b73d0 0x7fdb2c0b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57086/0 (socket says 172.21.15.66:57086) 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 -- 172.21.15.66:0/2300382873 learned_addr learned my addr 172.21.15.66:0/2300382873 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb30c23640 1 --2- 172.21.15.66:0/2300382873 >> v2:172.21.15.117:3300/0 conn(0x7fdb2c0ba090 0x7fdb2c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 -- 172.21.15.66:0/2300382873 >> v2:172.21.15.117:3300/0 conn(0x7fdb2c0ba090 msgr2=0x7fdb2c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 --2- 172.21.15.66:0/2300382873 >> v2:172.21.15.117:3300/0 conn(0x7fdb2c0ba090 0x7fdb2c0d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 -- 172.21.15.66:0/2300382873 >> v1:172.21.15.117:6789/0 conn(0x7fdb2c0b5dc0 legacy=0x7fdb2c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.146+0000 7fdb31c25640 1 -- 172.21.15.66:0/2300382873 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c0d3890 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.147+0000 7fdb31c25640 1 --2- 172.21.15.66:0/2300382873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c0b7820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009e50 tx=0x7fdb2402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3f26e47b2b9df5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.147+0000 7fdb23fff640 1 -- 172.21.15.66:0/2300382873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2402cb20 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.147+0000 7fdb23fff640 1 -- 172.21.15.66:0/2300382873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2402ccc0 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.147+0000 7fdb23fff640 1 -- 172.21.15.66:0/2300382873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24034610 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 -- 172.21.15.66:0/2300382873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 msgr2=0x7fdb2c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 --2- 172.21.15.66:0/2300382873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c0b7820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009e50 tx=0x7fdb2402bf80 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 -- 172.21.15.66:0/2300382873 shutdown_connections 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 --2- 172.21.15.66:0/2300382873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c0b7820 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 --2- 172.21.15.66:0/2300382873 >> v2:172.21.15.117:3300/0 conn(0x7fdb2c0ba090 0x7fdb2c0d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 -- 172.21.15.66:0/2300382873 >> 172.21.15.66:0/2300382873 conn(0x7fdb2c0678f0 msgr2=0x7fdb2c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 -- 172.21.15.66:0/2300382873 shutdown_connections 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.148+0000 7fdb33688640 1 -- 172.21.15.66:0/2300382873 wait complete. 2024-03-30T13:11:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.149+0000 7fdb33688640 1 Processor -- start 2024-03-30T13:11:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.149+0000 7fdb33688640 1 -- start start 2024-03-30T13:11:58.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb33688640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2c0b5dc0 0x7fdb2c0b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb33688640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb33688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2c0ba090 0x7fdb2c1f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb33688640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c15e880 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb33688640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0d4090 con 0x7fdb2c0ba090 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb33688640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c15ea00 con 0x7fdb2c0b5dc0 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57090/0 (socket says 172.21.15.66:57090) 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 -- 172.21.15.66:0/1374770092 learned_addr learned my addr 172.21.15.66:0/1374770092 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb31c25640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2c0ba090 0x7fdb2c1f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb31424640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2c0b5dc0 0x7fdb2c0b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 -- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2c0b5dc0 msgr2=0x7fdb2c0b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2c0b5dc0 0x7fdb2c0b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 -- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2c0ba090 msgr2=0x7fdb2c1f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2c0ba090 0x7fdb2c1f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.150+0000 7fdb30c23640 1 -- 172.21.15.66:0/1374770092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c1f3f00 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.151+0000 7fdb30c23640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c158a70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e870 tx=0x7fdb1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.151+0000 7fdb21ffb640 1 -- 172.21.15.66:0/1374770092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c017070 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.151+0000 7fdb21ffb640 1 -- 172.21.15.66:0/1374770092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00c940 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.151+0000 7fdb21ffb640 1 -- 172.21.15.66:0/1374770092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00cd30 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.151+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c1594c0 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.151+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c1f4460 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.153+0000 7fdb21ffb640 1 -- 172.21.15.66:0/1374770092 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdb1c01a070 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.154+0000 7fdb21ffb640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdb1006c530 0x7fdb1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.154+0000 7fdb21ffb640 1 -- 172.21.15.66:0/1374770092 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fdb1c08b670 con 0x7fdb2c0b73d0 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.154+0000 7fdb31424640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdb1006c530 0x7fdb1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.154+0000 7fdb36270740 1 Processor -- start 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.154+0000 7fdb36270740 1 -- start start 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.154+0000 7fdb31424640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdb1006c530 0x7fdb1006e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007fd0 tx=0x7fdb28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb36270740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 0x55855f7ec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819950 0x55855f81c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f7eca50 0x55855f81c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb36270740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55855f756590 con 0x55855f819570 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55855f6f4d30 con 0x55855f819950 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55855f752540 con 0x55855f7eca50 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 0x55855f7ec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 0x55855f7ec510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57094/0 (socket says 172.21.15.66:57094) 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 -- 172.21.15.66:0/499403146 learned_addr learned my addr 172.21.15.66:0/499403146 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 -- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f7eca50 msgr2=0x55855f81c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb30c23640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819950 0x55855f81c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31c25640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f7eca50 0x55855f81c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f7eca50 0x55855f81c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 -- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819950 msgr2=0x55855f81c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819950 0x55855f81c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 -- 172.21.15.66:0/499403146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855f536920 con 0x55855f819570 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31c25640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f7eca50 0x55855f81c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb30c23640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819950 0x55855f81c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb31424640 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 0x55855f7ec510 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdb280052d0 tx=0x7fdb28011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18071ae094fa98f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.155+0000 7fdb0effd640 1 -- 172.21.15.66:0/499403146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2800e500 con 0x55855f819570 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb0effd640 1 -- 172.21.15.66:0/499403146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2800e6a0 con 0x55855f819570 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb0effd640 1 -- 172.21.15.66:0/499403146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2800ea50 con 0x55855f819570 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 -- 172.21.15.66:0/499403146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 msgr2=0x55855f7ec510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 0x55855f7ec510 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdb280052d0 tx=0x7fdb28011000 comp rx=0 tx=0).stop 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 -- 172.21.15.66:0/499403146 shutdown_connections 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f7eca50 0x55855f81c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819950 0x55855f81c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 --2- 172.21.15.66:0/499403146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819570 0x55855f7ec510 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 -- 172.21.15.66:0/499403146 >> 172.21.15.66:0/499403146 conn(0x55855f7ee780 msgr2=0x55855f837300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 -- 172.21.15.66:0/499403146 shutdown_connections 2024-03-30T13:11:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.156+0000 7fdb36270740 1 -- 172.21.15.66:0/499403146 wait complete. 2024-03-30T13:11:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.157+0000 7fdb36270740 1 -- start start 2024-03-30T13:11:58.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.157+0000 7fdb36270740 1 Processor -- start 2024-03-30T13:11:58.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f80c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 0x55855f81f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f80c8d0 0x55855f81fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55855f7f0e20 con 0x55855f819950 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55855f6f29f0 con 0x55855f819570 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55855f7f3660 con 0x55855f80c8d0 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb36270740 10 client.0 _ll_register_callbacks cb 0x55855f6f6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 0x55855f81f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 0x55855f81f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57106/0 (socket says 172.21.15.66:57106) 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 -- 172.21.15.66:0/1084288745 learned_addr learned my addr 172.21.15.66:0/1084288745 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb31424640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f80c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb31c25640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f80c8d0 0x55855f81fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 -- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f80c8d0 msgr2=0x55855f81fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f80c8d0 0x55855f81fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 -- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 msgr2=0x55855f80c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f80c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 -- 172.21.15.66:0/1084288745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855f42a210 con 0x55855f819950 2024-03-30T13:11:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb30c23640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 0x55855f81f8d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c059420 tx=0x7fdb1c059490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f40a10a1b25aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.158+0000 7fdb07fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb07fff640 1 -- 172.21.15.66:0/1084288745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c021040 con 0x55855f819950 2024-03-30T13:11:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb07fff640 1 -- 172.21.15.66:0/1084288745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fdb1c058ae0 con 0x55855f819950 2024-03-30T13:11:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2800e6a0 con 0x55855f819950 2024-03-30T13:11:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb07fff640 1 -- 172.21.15.66:0/1084288745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdb1c0551e0 con 0x55855f819950 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 10 client.5739 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55855f5970b0 con 0x55855f819950 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb07fff640 1 -- 172.21.15.66:0/1084288745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2800e6a0 con 0x55855f819950 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb07fff640 1 -- 172.21.15.66:0/1084288745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fdb1c024060 con 0x55855f819950 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 10 client.5739 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 10 client.5739 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 20 client.5739 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f84b040 0x55855f84b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 4 client.5739 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855f763ad0 con 0x55855f84b040 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.159+0000 7fdb31424640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f84b040 0x55855f84b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb07fff640 10 client.5739 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb31424640 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f84b040 0x55855f84b460 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da29faa08b66042a server_cookie=280d1e3dca44e2a8 in_seq=0 out_seq=0 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb07fff640 1 -- 172.21.15.66:0/1084288745 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55855f763ad0 con 0x55855f84b040 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb07fff640 10 client.5739 handle_command_reply: tid=0 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 client.5739 shutdown 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 msgr2=0x55855f81f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 0x55855f81f8d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c059420 tx=0x7fdb1c059490 comp rx=0 tx=0).stop 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 shutdown_connections 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f84b040 0x55855f84b460 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f80c8d0 0x55855f81fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f819950 0x55855f81f8d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb07fff640 0 client.5739 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 --2- 172.21.15.66:0/1084288745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f80c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.160+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 >> 172.21.15.66:0/1084288745 conn(0x55855f7eca50 msgr2=0x55855f7f2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.161+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 shutdown_connections 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.161+0000 7fdb36270740 1 -- 172.21.15.66:0/1084288745 wait complete. 2024-03-30T13:11:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.161+0000 7fdb36270740 20 client.5739 trim_cache size 0 max 16384 2024-03-30T13:11:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.175+0000 7fdb36270740 1 Processor -- start 2024-03-30T13:11:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.175+0000 7fdb36270740 1 -- start start 2024-03-30T13:11:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb36270740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f81c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f81c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb36270740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c201280 con 0x55855f80c8d0 2024-03-30T13:11:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c200260 con 0x55855f819570 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c0cd270 con 0x55855f819950 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f81c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f81c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57108/0 (socket says 172.21.15.66:57108) 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 -- 172.21.15.66:0/2542821202 learned_addr learned my addr 172.21.15.66:0/2542821202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb30c23640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f81c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31c25640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 -- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 msgr2=0x55855f7eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 -- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 msgr2=0x55855f81c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f81c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31424640 1 -- 172.21.15.66:0/2542821202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855f80c680 con 0x55855f80c8d0 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb30c23640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f81c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.176+0000 7fdb31c25640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb31424640 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f81c0d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdb28014000 tx=0x7fdb28005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e29df5f259fb18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb0e7fc640 1 -- 172.21.15.66:0/2542821202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2802d250 con 0x55855f80c8d0 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb0e7fc640 1 -- 172.21.15.66:0/2542821202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28032070 con 0x55855f80c8d0 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb0e7fc640 1 -- 172.21.15.66:0/2542821202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2802d630 con 0x55855f80c8d0 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 -- 172.21.15.66:0/2542821202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 msgr2=0x55855f81c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f81c0d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdb28014000 tx=0x7fdb28005000 comp rx=0 tx=0).stop 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 -- 172.21.15.66:0/2542821202 shutdown_connections 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f81c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 --2- 172.21.15.66:0/2542821202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f81c0d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 -- 172.21.15.66:0/2542821202 >> 172.21.15.66:0/2542821202 conn(0x55855f7eca50 msgr2=0x55855f836e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 -- 172.21.15.66:0/2542821202 shutdown_connections 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.177+0000 7fdb36270740 1 -- 172.21.15.66:0/2542821202 wait complete. 2024-03-30T13:11:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.178+0000 7fdb36270740 1 -- start start 2024-03-30T13:11:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.178+0000 7fdb36270740 1 Processor -- start 2024-03-30T13:11:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f834290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f8347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55855f821150 con 0x55855f80c8d0 2024-03-30T13:11:58.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c1ffc10 con 0x55855f819570 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55855f763ad0 con 0x55855f819950 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb36270740 10 client.0 _ll_register_callbacks cb 0x7fdb1c00ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f834290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f834290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57120/0 (socket says 172.21.15.66:57120) 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 -- 172.21.15.66:0/3639897027 learned_addr learned my addr 172.21.15.66:0/3639897027 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb30c23640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f8347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31c25640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 -- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 msgr2=0x55855f7f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 -- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 msgr2=0x55855f8347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f8347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 -- 172.21.15.66:0/3639897027 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855f4032d0 con 0x55855f80c8d0 2024-03-30T13:11:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb30c23640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f8347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb31424640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f834290 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009010 tx=0x7fdb28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a67b06d5197a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.179+0000 7fdb07fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb07fff640 1 -- 172.21.15.66:0/3639897027 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803a070 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb07fff640 1 -- 172.21.15.66:0/3639897027 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fdb2802d460 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb28032070 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb07fff640 1 -- 172.21.15.66:0/3639897027 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdb28045050 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 10 client.5745 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55855f7f1fe0 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb07fff640 1 -- 172.21.15.66:0/3639897027 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28032070 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb07fff640 1 -- 172.21.15.66:0/3639897027 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fdb28048020 con 0x55855f80c8d0 2024-03-30T13:11:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 10 client.5745 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 10 client.5745 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 20 client.5745 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f81e960 0x55855f7f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 4 client.5745 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628294}] 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628294}) -- 0x55855f821d70 con 0x55855f81e960 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.180+0000 7fdb30c23640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f81e960 0x55855f7f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb07fff640 10 client.5745 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb30c23640 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f81e960 0x55855f7f16d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f548d1f12a7ed066 server_cookie=b061707ad8bc36ab in_seq=0 out_seq=0 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb07fff640 1 -- 172.21.15.66:0/3639897027 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4351 (crc 0 0 0) 0x55855f821d70 con 0x55855f81e960 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb07fff640 10 client.5745 handle_command_reply: tid=0 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 client.5745 shutdown 2024-03-30T13:11:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 msgr2=0x55855f834290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f834290 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009010 tx=0x7fdb28000bc0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 shutdown_connections 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55855f81e960 0x55855f7f16d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55855f819950 0x55855f7f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55855f819570 0x55855f8347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb07fff640 0 client.5745 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 --2- 172.21.15.66:0/3639897027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55855f80c8d0 0x55855f834290 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 >> 172.21.15.66:0/3639897027 conn(0x55855f7eca50 msgr2=0x55855f836e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 shutdown_connections 2024-03-30T13:11:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 1 -- 172.21.15.66:0/3639897027 wait complete. 2024-03-30T13:11:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.181+0000 7fdb36270740 20 client.5745 trim_cache size 0 max 16384 2024-03-30T13:11:58.193 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/io.h", 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628294, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.829293+0000", 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.298203+0000", 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:58.194 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 387, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:58.195 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.298203+0000", 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:58.196 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.197 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 387, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.829293+0000" 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 387, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.198 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.829293+0000" 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 176, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 30, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:58.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:58.200 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:58.201 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:58.202 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyrstat": 0, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:58.203 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:58.204 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.205 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.206 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:58.207 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:58.208 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdb1006c530 msgr2=0x7fdb1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdb1006c530 0x7fdb1006e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007fd0 tx=0x7fdb28009040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 msgr2=0x7fdb2c158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c158a70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e870 tx=0x7fdb1c00ed30 comp rx=0 tx=0).stop 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 shutdown_connections 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdb1006c530 0x7fdb1006e9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2c0ba090 0x7fdb2c1f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb2c0b73d0 0x7fdb2c158a70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 --2- 172.21.15.66:0/1374770092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb2c0b5dc0 0x7fdb2c0b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 >> 172.21.15.66:0/1374770092 conn(0x7fdb2c0678f0 msgr2=0x7fdb2c0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 shutdown_connections 2024-03-30T13:11:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.182+0000 7fdb33688640 1 -- 172.21.15.66:0/1374770092 wait complete. 2024-03-30T13:11:58.211 DEBUG:tasks.cephfs.test_uninlining:getting inode info #5 of 23:1099511628296 2024-03-30T13:11:58.211 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628296 2024-03-30T13:11:58.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628296 2024-03-30T13:11:58.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.317+0000 7fb69991c640 1 Processor -- start 2024-03-30T13:11:58.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.317+0000 7fb69991c640 1 -- start start 2024-03-30T13:11:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb69991c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb694145d70 0x7fb694160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb69991c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6940ce490 0x7fb6940cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb69991c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6940d0290 0x7fb69415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb69991c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb6940ce8f0 con 0x7fb694145d70 2024-03-30T13:11:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb69991c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fb694146150 con 0x7fb6940ce490 2024-03-30T13:11:58.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb69991c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb69415ca20 con 0x7fb6940d0290 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb692ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb694145d70 0x7fb694160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb692ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb694145d70 0x7fb694160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57134/0 (socket says 172.21.15.66:57134) 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb692ffd640 1 -- 172.21.15.66:0/384999937 learned_addr learned my addr 172.21.15.66:0/384999937 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb6927fc640 1 --2- 172.21.15.66:0/384999937 >> v2:172.21.15.117:3300/0 conn(0x7fb6940ce490 0x7fb6940cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.318+0000 7fb6937fe640 1 --2- 172.21.15.66:0/384999937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6940d0290 0x7fb69415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.319+0000 7fb692ffd640 1 -- 172.21.15.66:0/384999937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6940d0290 msgr2=0x7fb69415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.319+0000 7fb692ffd640 1 --2- 172.21.15.66:0/384999937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6940d0290 0x7fb69415ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.319+0000 7fb692ffd640 1 -- 172.21.15.66:0/384999937 >> v2:172.21.15.117:3300/0 conn(0x7fb6940ce490 msgr2=0x7fb6940cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.319+0000 7fb692ffd640 1 --2- 172.21.15.66:0/384999937 >> v2:172.21.15.117:3300/0 conn(0x7fb6940ce490 0x7fb6940cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.319+0000 7fb692ffd640 1 -- 172.21.15.66:0/384999937 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6940cc970 con 0x7fb694145d70 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.319+0000 7fb692ffd640 1 --2- 172.21.15.66:0/384999937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694145d70 0x7fb694160880 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb694072290 tx=0x7fb68802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c8cbd11a573048 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb691ffb640 1 -- 172.21.15.66:0/384999937 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68802e070 con 0x7fb694145d70 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb691ffb640 1 -- 172.21.15.66:0/384999937 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688037070 con 0x7fb694145d70 2024-03-30T13:11:58.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb691ffb640 1 -- 172.21.15.66:0/384999937 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688031a00 con 0x7fb694145d70 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 -- 172.21.15.66:0/384999937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694145d70 msgr2=0x7fb694160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 --2- 172.21.15.66:0/384999937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694145d70 0x7fb694160880 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb694072290 tx=0x7fb68802b880 comp rx=0 tx=0).stop 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 -- 172.21.15.66:0/384999937 shutdown_connections 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 --2- 172.21.15.66:0/384999937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6940d0290 0x7fb69415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 --2- 172.21.15.66:0/384999937 >> v2:172.21.15.117:3300/0 conn(0x7fb6940ce490 0x7fb6940cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 --2- 172.21.15.66:0/384999937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb694145d70 0x7fb694160880 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 -- 172.21.15.66:0/384999937 >> 172.21.15.66:0/384999937 conn(0x7fb6940b3e70 msgr2=0x7fb6940b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 -- 172.21.15.66:0/384999937 shutdown_connections 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.320+0000 7fb69991c640 1 -- 172.21.15.66:0/384999937 wait complete. 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.322+0000 7fb69991c640 1 Processor -- start 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.322+0000 7fb69991c640 1 -- start start 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.322+0000 7fb69991c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 0x7fb6940b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6940d0290 0x7fb694142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb694145d70 0x7fb694142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940ccf30 con 0x7fb6940ce490 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940cc500 con 0x7fb6940d0290 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb69415d3f0 con 0x7fb694145d70 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 0x7fb6940b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 0x7fb6940b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57144/0 (socket says 172.21.15.66:57144) 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 -- 172.21.15.66:0/3527769677 learned_addr learned my addr 172.21.15.66:0/3527769677 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6940d0290 0x7fb694142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb6937fe640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb694145d70 0x7fb694142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 -- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb694145d70 msgr2=0x7fb694142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb694145d70 0x7fb694142ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 -- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6940d0290 msgr2=0x7fb694142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6940d0290 0x7fb694142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 -- 172.21.15.66:0/3527769677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6941f3930 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6940d0290 0x7fb694142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb6937fe640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb694145d70 0x7fb694142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb692ffd640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 0x7fb6940b66f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb688000c00 tx=0x7fb6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb67bfff640 1 -- 172.21.15.66:0/3527769677 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688032670 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb67bfff640 1 -- 172.21.15.66:0/3527769677 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688032810 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6941f3390 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb67bfff640 1 -- 172.21.15.66:0/3527769677 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688032c00 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.323+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6941f3ed0 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.326+0000 7fb67bfff640 1 -- 172.21.15.66:0/3527769677 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb68804c060 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.326+0000 7fb67bfff640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb68006c530 0x7fb68006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.326+0000 7fb67bfff640 1 -- 172.21.15.66:0/3527769677 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fb6880ad180 con 0x7fb6940ce490 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.326+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb68006c530 0x7fb68006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.327+0000 7fb69c5e9740 1 Processor -- start 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.327+0000 7fb69c5e9740 1 -- start start 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.327+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb68006c530 0x7fb68006e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb6940b2890 tx=0x7fb684008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b570 0x55b001f1e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b950 0x55b001f4e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 0x55b001f4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b001e88590 con 0x55b001f1ea50 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b001e26d30 con 0x55b001f4b950 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b001e84540 con 0x55b001f4b570 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 0x55b001f4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 0x55b001f4e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57158/0 (socket says 172.21.15.66:57158) 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 -- 172.21.15.66:0/2262118193 learned_addr learned my addr 172.21.15.66:0/2262118193 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb692ffd640 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b570 0x55b001f1e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6927fc640 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b950 0x55b001f4e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 -- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b570 msgr2=0x55b001f1e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b570 0x55b001f1e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 -- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b950 msgr2=0x55b001f4e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b950 0x55b001f4e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 -- 172.21.15.66:0/2262118193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b001c68920 con 0x55b001f1ea50 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6927fc640 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b950 0x55b001f4e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6937fe640 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 0x55b001f4e840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0074f0 tx=0x7fb68c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992eb99db07fea58 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6797fa640 1 -- 172.21.15.66:0/2262118193 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c014760 con 0x55b001f1ea50 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6797fa640 1 -- 172.21.15.66:0/2262118193 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c01a070 con 0x55b001f1ea50 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.328+0000 7fb6797fa640 1 -- 172.21.15.66:0/2262118193 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c014b00 con 0x55b001f1ea50 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 -- 172.21.15.66:0/2262118193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 msgr2=0x55b001f4e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 0x55b001f4e840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0074f0 tx=0x7fb68c009750 comp rx=0 tx=0).stop 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 -- 172.21.15.66:0/2262118193 shutdown_connections 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f1ea50 0x55b001f4e840 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b950 0x55b001f4e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/2262118193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b570 0x55b001f1e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 -- 172.21.15.66:0/2262118193 >> 172.21.15.66:0/2262118193 conn(0x55b001f20780 msgr2=0x55b001f69300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 -- 172.21.15.66:0/2262118193 shutdown_connections 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.329+0000 7fb69c5e9740 1 -- 172.21.15.66:0/2262118193 wait complete. 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.330+0000 7fb69c5e9740 1 -- start start 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.330+0000 7fb69c5e9740 1 Processor -- start 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f44310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f44850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b001f22e20 con 0x55b001f4b950 2024-03-30T13:11:58.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b001e249f0 con 0x55b001f4b570 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b001f25660 con 0x55b001f35450 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 10 client.0 _ll_register_callbacks cb 0x55b001e28c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f44850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f44850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57166/0 (socket says 172.21.15.66:57166) 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 -- 172.21.15.66:0/3835656036 learned_addr learned my addr 172.21.15.66:0/3835656036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb692ffd640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f44310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6937fe640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 -- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 msgr2=0x55b001f358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 -- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 msgr2=0x55b001f44310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f44310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 -- 172.21.15.66:0/3835656036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b001b5c210 con 0x55b001f4b950 2024-03-30T13:11:58.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6927fc640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f44850 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb684015880 tx=0x7fb684004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdce572ff45f7a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6727fc640 1 -- 172.21.15.66:0/3835656036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68401c070 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb6727fc640 1 -- 172.21.15.66:0/3835656036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fb684015e00 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.331+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb68c01a070 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb6727fc640 1 -- 172.21.15.66:0/3835656036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb68402e050 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 10 client.5757 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b001cc90b0 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb6727fc640 1 -- 172.21.15.66:0/3835656036 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c01a070 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb6727fc640 1 -- 172.21.15.66:0/3835656036 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fb684031020 con 0x55b001f4b950 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 10 client.5757 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 10 client.5757 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 20 client.5757 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 4 client.5757 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b001e95ad0 con 0x55b001f69260 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb692ffd640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb6727fc640 10 client.5757 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.332+0000 7fb692ffd640 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f69710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4322ea72464ad7b server_cookie=ca3cdd7d9af29eb6 in_seq=0 out_seq=0 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb6727fc640 1 -- 172.21.15.66:0/3835656036 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55b001e95ad0 con 0x55b001f69260 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb6727fc640 10 client.5757 handle_command_reply: tid=0 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 client.5757 shutdown 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 msgr2=0x55b001f44850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f44850 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb684015880 tx=0x7fb684004730 comp rx=0 tx=0).stop 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 shutdown_connections 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f69710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f44850 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb6727fc640 0 client.5757 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/3835656036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f44310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 >> 172.21.15.66:0/3835656036 conn(0x55b001f1ea50 msgr2=0x55b001f22420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 shutdown_connections 2024-03-30T13:11:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 1 -- 172.21.15.66:0/3835656036 wait complete. 2024-03-30T13:11:58.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.333+0000 7fb69c5e9740 20 client.5757 trim_cache size 0 max 16384 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.348+0000 7fb69c5e9740 1 Processor -- start 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.348+0000 7fb69c5e9740 1 -- start start 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f35450 0x55b001f6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 0x55b001f49c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b950 0x55b001f4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941ff0a0 con 0x55b001f4b570 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941fe0c0 con 0x55b001f35450 2024-03-30T13:11:58.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb6941f5680 con 0x55b001f4b950 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 0x55b001f49c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 0x55b001f49c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57168/0 (socket says 172.21.15.66:57168) 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 -- 172.21.15.66:0/1239149898 learned_addr learned my addr 172.21.15.66:0/1239149898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb692ffd640 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f35450 0x55b001f6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6937fe640 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b950 0x55b001f4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 -- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b950 msgr2=0x55b001f4a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b950 0x55b001f4a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 -- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f35450 msgr2=0x55b001f6ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f35450 0x55b001f6ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 -- 172.21.15.66:0/1239149898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b001b352d0 con 0x55b001f4b570 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb692ffd640 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f35450 0x55b001f6ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb6927fc640 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 0x55b001f49c70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb684008010 tx=0x7fb684017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3258166fa12e851 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.349+0000 7fb678ff9640 1 -- 172.21.15.66:0/1239149898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68401c040 con 0x55b001f4b570 2024-03-30T13:11:58.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb678ff9640 1 -- 172.21.15.66:0/1239149898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb684038070 con 0x55b001f4b570 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb678ff9640 1 -- 172.21.15.66:0/1239149898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68400f040 con 0x55b001f4b570 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 -- 172.21.15.66:0/1239149898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 msgr2=0x55b001f49c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 0x55b001f49c70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb684008010 tx=0x7fb684017000 comp rx=0 tx=0).stop 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 -- 172.21.15.66:0/1239149898 shutdown_connections 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f4b950 0x55b001f4a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b570 0x55b001f49c70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/1239149898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f35450 0x55b001f6ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 -- 172.21.15.66:0/1239149898 >> 172.21.15.66:0/1239149898 conn(0x55b001f1ea50 msgr2=0x55b001f8e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 -- 172.21.15.66:0/1239149898 shutdown_connections 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.350+0000 7fb69c5e9740 1 -- 172.21.15.66:0/1239149898 wait complete. 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.351+0000 7fb69c5e9740 1 -- start start 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.351+0000 7fb69c5e9740 1 Processor -- start 2024-03-30T13:11:58.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f4cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b001f46810 con 0x55b001f4b950 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941fdaa0 con 0x55b001f4b570 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b001e95ad0 con 0x55b001f35450 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb69c5e9740 10 client.0 _ll_register_callbacks cb 0x7fb684009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f49a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57174/0 (socket says 172.21.15.66:57174) 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 -- 172.21.15.66:0/843103431 learned_addr learned my addr 172.21.15.66:0/843103431 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6927fc640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f4cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb692ffd640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 -- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 msgr2=0x55b001f782f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f782f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 -- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 msgr2=0x55b001f4cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f4cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 -- 172.21.15.66:0/843103431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b001f78830 con 0x55b001f4b950 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6937fe640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f49a60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0074c0 tx=0x7fb68c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe7f9c445e1325c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.352+0000 7fb6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb6727fc640 1 -- 172.21.15.66:0/843103431 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c02a680 con 0x55b001f4b950 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb6727fc640 1 -- 172.21.15.66:0/843103431 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fb68c02a820 con 0x55b001f4b950 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb684038070 con 0x55b001f4b950 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb6727fc640 1 -- 172.21.15.66:0/843103431 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb68c032050 con 0x55b001f4b950 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 10 client.5763 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b001f4d5b0 con 0x55b001f4b950 2024-03-30T13:11:58.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb6727fc640 1 -- 172.21.15.66:0/843103431 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb684038070 con 0x55b001f4b950 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb6727fc640 1 -- 172.21.15.66:0/843103431 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fb68c035020 con 0x55b001f4b950 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 10 client.5763 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 10 client.5763 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 20 client.5763 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 4 client.5763 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628296}] 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628296}) -- 0x55b001f46130 con 0x55b001f69260 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb692ffd640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.353+0000 7fb6727fc640 10 client.5763 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb692ffd640 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f49fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba6ea96284da0ae server_cookie=b384cbd958b5feb4 in_seq=0 out_seq=0 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb6727fc640 1 -- 172.21.15.66:0/843103431 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4357 (crc 0 0 0) 0x55b001f46130 con 0x55b001f69260 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb6727fc640 10 client.5763 handle_command_reply: tid=0 2024-03-30T13:11:58.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 client.5763 shutdown 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 msgr2=0x55b001f49a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f49a60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0074c0 tx=0x7fb68c004ab0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 shutdown_connections 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b001f69260 0x55b001f49fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b001f4b950 0x55b001f49a60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b001f4b570 0x55b001f4cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb6727fc640 0 client.5763 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 --2- 172.21.15.66:0/843103431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b001f35450 0x55b001f782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 >> 172.21.15.66:0/843103431 conn(0x55b001f1ea50 msgr2=0x55b001f51d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 shutdown_connections 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 1 -- 172.21.15.66:0/843103431 wait complete. 2024-03-30T13:11:58.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.354+0000 7fb69c5e9740 20 client.5763 trim_cache size 0 max 16384 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/ioctl.h", 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628296, 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.830843+0000", 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.300203+0000", 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:58.366 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.367 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 3724, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.300203+0000", 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:58.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:58.369 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3724, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.830843+0000" 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3724, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.830843+0000" 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 180, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 34, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:58.371 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:58.372 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.373 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyrstat": 0, 2024-03-30T13:11:58.374 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.375 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.376 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:58.378 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.379 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:58.380 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:58.381 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb68006c530 msgr2=0x7fb68006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb68006c530 0x7fb68006e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb6940b2890 tx=0x7fb684008040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 msgr2=0x7fb6940b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 0x7fb6940b66f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb688000c00 tx=0x7fb6880046c0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 shutdown_connections 2024-03-30T13:11:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb68006c530 0x7fb68006e9f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb694145d70 0x7fb694142ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6940d0290 0x7fb694142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 --2- 172.21.15.66:0/3527769677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6940ce490 0x7fb6940b66f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 >> 172.21.15.66:0/3527769677 conn(0x7fb6940b3e70 msgr2=0x7fb6940b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 shutdown_connections 2024-03-30T13:11:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.355+0000 7fb69991c640 1 -- 172.21.15.66:0/3527769677 wait complete. 2024-03-30T13:11:58.385 DEBUG:tasks.cephfs.test_uninlining:getting inode info #6 of 23:1099511628308 2024-03-30T13:11:58.385 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628308 2024-03-30T13:11:58.385 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628308 2024-03-30T13:11:58.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.492+0000 7fd0f359e640 1 Processor -- start 2024-03-30T13:11:58.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.492+0000 7fd0f359e640 1 -- start start 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f359e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0f40d12c0 0x7fd0f40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f359e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd0f406ce70 0x7fd0f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f359e640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f4068070 con 0x7fd0f415e6c0 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f359e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f40ccd30 con 0x7fd0f40d12c0 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f359e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f415eaa0 con 0x7fd0f406ce70 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0ebfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd0f40d12c0 0x7fd0f40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f259c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd0f415e6c0 0x7fd0f40cc620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37482/0 (socket says 172.21.15.66:37482) 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.493+0000 7fd0f259c640 1 -- 172.21.15.66:0/459073614 learned_addr learned my addr 172.21.15.66:0/459073614 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.494+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/459073614 >> v2:172.21.15.117:3300/0 conn(0x7fd0f406ce70 0x7fd0f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.494+0000 7fd0ebfff640 1 -- 172.21.15.66:0/459073614 >> v2:172.21.15.117:3300/0 conn(0x7fd0f406ce70 msgr2=0x7fd0f406d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.494+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/459073614 >> v2:172.21.15.117:3300/0 conn(0x7fd0f406ce70 0x7fd0f406d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.494+0000 7fd0ebfff640 1 -- 172.21.15.66:0/459073614 >> v1:172.21.15.66:6789/0 conn(0x7fd0f415e6c0 legacy=0x7fd0f40cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.494+0000 7fd0ebfff640 1 -- 172.21.15.66:0/459073614 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f406c9c0 con 0x7fd0f40d12c0 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.494+0000 7fd0f1d9b640 1 -- 172.21.15.66:0/459073614 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340118415 0 0) 0x7fd0e4001110 con 0x7fd0f415e6c0 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/459073614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f40d16e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406c380 tx=0x7fd0dc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9794ff3d0fe1609 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f1d9b640 1 -- 172.21.15.66:0/459073614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc034070 con 0x7fd0f40d12c0 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f1d9b640 1 -- 172.21.15.66:0/459073614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc030610 con 0x7fd0f40d12c0 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f1d9b640 1 -- 172.21.15.66:0/459073614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0309e0 con 0x7fd0f40d12c0 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 -- 172.21.15.66:0/459073614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 msgr2=0x7fd0f40d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 --2- 172.21.15.66:0/459073614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f40d16e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406c380 tx=0x7fd0dc02b960 comp rx=0 tx=0).stop 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 -- 172.21.15.66:0/459073614 shutdown_connections 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 --2- 172.21.15.66:0/459073614 >> v2:172.21.15.117:3300/0 conn(0x7fd0f406ce70 0x7fd0f406d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 --2- 172.21.15.66:0/459073614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f40d16e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 -- 172.21.15.66:0/459073614 >> 172.21.15.66:0/459073614 conn(0x7fd0f40cea50 msgr2=0x7fd0f40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 -- 172.21.15.66:0/459073614 shutdown_connections 2024-03-30T13:11:58.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.495+0000 7fd0f359e640 1 -- 172.21.15.66:0/459073614 wait complete. 2024-03-30T13:11:58.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 Processor -- start 2024-03-30T13:11:58.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 -- start start 2024-03-30T13:11:58.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0f406ce70 0x7fd0f415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f415e6c0 0x7fd0f41600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40d2d40 con 0x7fd0f40d12c0 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40d1c20 con 0x7fd0f415e6c0 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.497+0000 7fd0f359e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f406dbf0 con 0x7fd0f406ce70 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f415fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57186/0 (socket says 172.21.15.66:57186) 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 -- 172.21.15.66:0/1104624763 learned_addr learned my addr 172.21.15.66:0/1104624763 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f415e6c0 0x7fd0f41600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0f259c640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0f406ce70 0x7fd0f415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 -- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0f406ce70 msgr2=0x7fd0f415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0f406ce70 0x7fd0f415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 -- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f415e6c0 msgr2=0x7fd0f41600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f415e6c0 0x7fd0f41600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 -- 172.21.15.66:0/1104624763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f4161d80 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f415e6c0 0x7fd0f41600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0f259c640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0f406ce70 0x7fd0f415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f415fba0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc000c00 tx=0x7fd0dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0eaffd640 1 -- 172.21.15.66:0/1104624763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc034040 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0eaffd640 1 -- 172.21.15.66:0/1104624763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc030410 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f4162060 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0eaffd640 1 -- 172.21.15.66:0/1104624763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc039c50 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.498+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f4162620 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.500+0000 7fd0eaffd640 1 -- 172.21.15.66:0/1104624763 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd0dc037070 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.500+0000 7fd0eaffd640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd0d406c530 0x7fd0d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.500+0000 7fd0eaffd640 1 -- 172.21.15.66:0/1104624763 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd0dc0ad880 con 0x7fd0f40d12c0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.500+0000 7fd0f259c640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd0d406c530 0x7fd0d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.501+0000 7fd0f259c640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd0d406c530 0x7fd0d406e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000f60 tx=0x7fd0e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.501+0000 7fd0fb7ea740 1 Processor -- start 2024-03-30T13:11:58.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.501+0000 7fd0fb7ea740 1 -- start start 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b401186730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b401186c70 0x55b4011b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b4010f0590 con 0x55b4011b3b70 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b40108ed30 con 0x55b401186c70 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b4010ec540 con 0x55b4011b3790 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57198/0 (socket says 172.21.15.66:57198) 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0ebfff640 1 -- 172.21.15.66:0/2051411674 learned_addr learned my addr 172.21.15.66:0/2051411674 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0f259c640 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b401186730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.502+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b401186c70 0x55b4011b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0ebfff640 1 -- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 msgr2=0x55b401186730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b401186730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0ebfff640 1 -- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b401186c70 msgr2=0x55b4011b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b401186c70 0x55b4011b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0ebfff640 1 -- 172.21.15.66:0/2051411674 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b400ed0920 con 0x55b4011b3b70 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b401186c70 0x55b4011b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011b6520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc07b2f0 tx=0x7fd0dc0779c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8aa75d5fbf9d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0d3fff640 1 -- 172.21.15.66:0/2051411674 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc07b5f0 con 0x55b4011b3b70 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0d3fff640 1 -- 172.21.15.66:0/2051411674 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc07f9a0 con 0x55b4011b3b70 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.503+0000 7fd0d3fff640 1 -- 172.21.15.66:0/2051411674 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0841d0 con 0x55b4011b3b70 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/2051411674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 msgr2=0x55b4011b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011b6520 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc07b2f0 tx=0x7fd0dc0779c0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/2051411674 shutdown_connections 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b401186c70 0x55b4011b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011b6520 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/2051411674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b401186730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/2051411674 >> 172.21.15.66:0/2051411674 conn(0x55b4011889a0 msgr2=0x55b4011d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/2051411674 shutdown_connections 2024-03-30T13:11:58.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/2051411674 wait complete. 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 -- start start 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.504+0000 7fd0fb7ea740 1 Processor -- start 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b4011bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011e4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b4a20 0x55b4011e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b4010f0590 con 0x55b4011b3b70 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b40108c9f0 con 0x55b4011b4a20 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b40118b040 con 0x55b4011b3790 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0fb7ea740 10 client.0 _ll_register_callbacks cb 0x55b401090c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57212/0 (socket says 172.21.15.66:57212) 2024-03-30T13:11:58.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0ebfff640 1 -- 172.21.15.66:0/3432076622 learned_addr learned my addr 172.21.15.66:0/3432076622 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b4a20 0x55b4011e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.505+0000 7fd0f259c640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b4011bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0ebfff640 1 -- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 msgr2=0x55b4011bb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b4011bb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0ebfff640 1 -- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b4a20 msgr2=0x55b4011e4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b4a20 0x55b4011e4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0ebfff640 1 -- 172.21.15.66:0/3432076622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b400dc4210 con 0x55b4011b3b70 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b4a20 0x55b4011e4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0f259c640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b4011bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011e4820 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0abbd0 tx=0x7fd0dc0abad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d522d182ed4a03fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.518 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0d17fa640 1 -- 172.21.15.66:0/3432076622 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0841d0 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0d17fa640 1 -- 172.21.15.66:0/3432076622 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd0dc0305c0 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0dc07f9a0 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0d17fa640 1 -- 172.21.15.66:0/3432076622 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc07b5f0 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0fb7ea740 10 client.5778 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b400f310b0 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.506+0000 7fd0d17fa640 1 -- 172.21.15.66:0/3432076622 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc07f9a0 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0d17fa640 1 -- 172.21.15.66:0/3432076622 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fd0dc041070 con 0x55b4011b3b70 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0fb7ea740 10 client.5778 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0fb7ea740 10 client.5778 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0fb7ea740 20 client.5778 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b4011a0160 0x55b4011a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0fb7ea740 4 client.5778 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4010fdad0 con 0x55b4011a0160 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0f259c640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b4011a0160 0x55b4011a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0d17fa640 10 client.5778 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.507+0000 7fd0f259c640 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b4011a0160 0x55b4011a05b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf1544a5be6e990 server_cookie=f78553e7467d8773 in_seq=0 out_seq=0 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0d17fa640 1 -- 172.21.15.66:0/3432076622 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55b4010fdad0 con 0x55b4011a0160 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0d17fa640 10 client.5778 handle_command_reply: tid=0 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 client.5778 shutdown 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 msgr2=0x55b4011e4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011e4820 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0abbd0 tx=0x7fd0dc0abad0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 shutdown_connections 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b4011a0160 0x55b4011a05b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b4a20 0x55b4011e4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b3b70 0x55b4011e4820 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0d17fa640 0 client.5778 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/3432076622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b4011bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 >> 172.21.15.66:0/3432076622 conn(0x55b401186c70 msgr2=0x55b40118a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 shutdown_connections 2024-03-30T13:11:58.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/3432076622 wait complete. 2024-03-30T13:11:58.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.508+0000 7fd0fb7ea740 20 client.5778 trim_cache size 0 max 16384 2024-03-30T13:11:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 Processor -- start 2024-03-30T13:11:58.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 -- start start 2024-03-30T13:11:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b4011d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f42010d0 con 0x55b4011b4a20 2024-03-30T13:11:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4200110 con 0x55b4011b3b70 2024-03-30T13:11:58.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f40c9f00 con 0x55b4011b3790 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.523+0000 7fd0f2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b4011d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b4011d46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57218/0 (socket says 172.21.15.66:57218) 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/924000876 learned_addr learned my addr 172.21.15.66:0/924000876 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f259c640 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 msgr2=0x55b40118a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 msgr2=0x55b40118a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/924000876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b400d9d2d0 con 0x55b4011b4a20 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b4011d46f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007550 tx=0x7fd0ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f533dbac6a8437e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0d37fe640 1 -- 172.21.15.66:0/924000876 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec01b070 con 0x55b4011b4a20 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0d37fe640 1 -- 172.21.15.66:0/924000876 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ec017640 con 0x55b4011b4a20 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.524+0000 7fd0d37fe640 1 -- 172.21.15.66:0/924000876 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec017a50 con 0x55b4011b4a20 2024-03-30T13:11:58.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/924000876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 msgr2=0x55b4011d46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b4011d46f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007550 tx=0x7fd0ec009750 comp rx=0 tx=0).stop 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/924000876 shutdown_connections 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b4011d46f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/924000876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/924000876 >> 172.21.15.66:0/924000876 conn(0x55b401186c70 msgr2=0x55b4011b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/924000876 shutdown_connections 2024-03-30T13:11:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/924000876 wait complete. 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 -- start start 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.525+0000 7fd0fb7ea740 1 Processor -- start 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b40118a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b4010fe630 con 0x55b4011b4a20 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f41ffaf0 con 0x55b4011b3b70 2024-03-30T13:11:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b4011e0f50 con 0x55b4011b3790 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0fb7ea740 10 client.0 _ll_register_callbacks cb 0x7fd0e4001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0f2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b40118a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0f2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b40118a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57228/0 (socket says 172.21.15.66:57228) 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.526+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/174881993 learned_addr learned my addr 172.21.15.66:0/174881993 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f259c640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0ebfff640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 msgr2=0x55b40118d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 msgr2=0x55b40118a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f2d9d640 1 -- 172.21.15.66:0/174881993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40124c4f0 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0f2d9d640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b40118a7d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007520 tx=0x7fd0ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06cc6b50923c7a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0d17fa640 1 -- 172.21.15.66:0/174881993 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec02e060 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0d17fa640 1 -- 172.21.15.66:0/174881993 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd0ec020660 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ec017640 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0d17fa640 1 -- 172.21.15.66:0/174881993 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd0ec034050 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0fb7ea740 10 client.5784 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.527+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b4011a3220 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0d17fa640 1 -- 172.21.15.66:0/174881993 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ec017640 con 0x55b4011b4a20 2024-03-30T13:11:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0d17fa640 1 -- 172.21.15.66:0/174881993 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fd0ec037020 con 0x55b4011b4a20 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 10 client.5784 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 10 client.5784 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 20 client.5784 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b40118ca20 0x55b4011855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 4 client.5784 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628308}] 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628308}) -- 0x55b4010f7e20 con 0x55b40118ca20 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0f259c640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b40118ca20 0x55b4011855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0d17fa640 10 client.5784 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0f259c640 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b40118ca20 0x55b4011855c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623c7526d56e3da3 server_cookie=7dd98db134669325 in_seq=0 out_seq=0 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0d17fa640 1 -- 172.21.15.66:0/174881993 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4330 (crc 0 0 0) 0x55b4010f7e20 con 0x55b40118ca20 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0d17fa640 10 client.5784 handle_command_reply: tid=0 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.528+0000 7fd0fb7ea740 1 client.5784 shutdown 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 msgr2=0x55b40118a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b40118a7d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec007520 tx=0x7fd0ec004ab0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 shutdown_connections 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b40118ca20 0x55b4011855c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b4011b4a20 0x55b40118a7d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4011b3b70 0x55b40118a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 --2- 172.21.15.66:0/174881993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b4011b3790 0x55b40118d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0d17fa640 0 client.5784 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 >> 172.21.15.66:0/174881993 conn(0x55b401186c70 msgr2=0x55b4011b9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 shutdown_connections 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 1 -- 172.21.15.66:0/174881993 wait complete. 2024-03-30T13:11:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0fb7ea740 20 client.5784 trim_cache size 0 max 16384 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/Kconfig", 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628308, 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.838220+0000", 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.331203+0000", 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:58.547 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:58.548 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1240, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.331203+0000", 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:58.549 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:58.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1240, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.838220+0000" 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1240, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.838220+0000" 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 72, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 2, 2024-03-30T13:11:58.552 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:58.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:58.554 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.555 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:58.556 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.557 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.558 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.559 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.560 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:58.561 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.561 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.561 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.561 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.561 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.562 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:58.562 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:58.562 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:58.562 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:58.562 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:58.562 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:58.563 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:58.564 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.564 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.564 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:58.564 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:58.564 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:58.564 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:58.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd0d406c530 msgr2=0x7fd0d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd0d406c530 0x7fd0d406e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000f60 tx=0x7fd0e4009040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 msgr2=0x7fd0f415fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f415fba0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc000c00 tx=0x7fd0dc0046c0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 shutdown_connections 2024-03-30T13:11:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd0d406c530 0x7fd0d406e9f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f415e6c0 0x7fd0f41600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0f40d12c0 0x7fd0f415fba0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.529+0000 7fd0f359e640 1 --2- 172.21.15.66:0/1104624763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0f406ce70 0x7fd0f415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.530+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 >> 172.21.15.66:0/1104624763 conn(0x7fd0f40cea50 msgr2=0x7fd0f40b7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.530+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 shutdown_connections 2024-03-30T13:11:58.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.530+0000 7fd0f359e640 1 -- 172.21.15.66:0/1104624763 wait complete. 2024-03-30T13:11:58.570 DEBUG:tasks.cephfs.test_uninlining:getting inode info #7 of 23:1099511628326 2024-03-30T13:11:58.570 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628326 2024-03-30T13:11:58.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628326 2024-03-30T13:11:58.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.677+0000 7f4813f52640 1 Processor -- start 2024-03-30T13:11:58.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.677+0000 7f4813f52640 1 -- start start 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4813f52640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f480c0b5dc0 0x7f480c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4813f52640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 0x7f480c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4813f52640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480c0b66e0 con 0x7f480c0ba090 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4813f52640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f480c0d2d90 con 0x7f480c0b5dc0 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4813f52640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f480c0d2ff0 con 0x7f480c0b73d0 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f48114ed640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f480c0ba090 0x7f480c0d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37498/0 (socket says 172.21.15.66:37498) 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f48114ed640 1 -- 172.21.15.66:0/3448316689 learned_addr learned my addr 172.21.15.66:0/3448316689 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:58.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4811cee640 1 --2- 172.21.15.66:0/3448316689 >> v2:172.21.15.117:3300/0 conn(0x7f480c0b5dc0 0x7f480c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f48124ef640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 0x7f480c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973861317 0 0) 0x7f480c0b66e0 con 0x7f480c0ba090 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.678+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47f8003520 con 0x7f480c0ba090 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2067329855 0 0) 0x7f47f8003520 con 0x7f480c0ba090 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 msgr2=0x7f480c0b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 0x7f480c0b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 >> v2:172.21.15.117:3300/0 conn(0x7f480c0b5dc0 msgr2=0x7f480c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 --2- 172.21.15.66:0/3448316689 >> v2:172.21.15.117:3300/0 conn(0x7f480c0b5dc0 0x7f480c0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c0d32d0 con 0x7f480c0ba090 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f47fc001670 con 0x7f480c0ba090 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 >> v1:172.21.15.66:6789/0 conn(0x7f480c0ba090 legacy=0x7f480c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47f80039b0 0x7f47f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 0x7f47f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47f8004d40 0x7f47f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d2d90 con 0x7f47f80039b0 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d2ff0 con 0x7f47f8004d40 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f47f800b9c0 con 0x7f480c0b73d0 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4811cee640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47f80039b0 0x7f47f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f48124ef640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47f8004d40 0x7f47f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f48114ed640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 0x7f47f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4811cee640 1 -- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 msgr2=0x7f47f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4811cee640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f480c0b73d0 0x7f47f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4811cee640 1 -- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47f8004d40 msgr2=0x7f47f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4811cee640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47f8004d40 0x7f47f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.679+0000 7f4811cee640 1 -- 172.21.15.66:0/3448316689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f8003520 con 0x7f47f80039b0 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.680+0000 7f4811cee640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47f80039b0 0x7f47f8003dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4808009ac0 tx=0x7f47f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649da81542970485 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.680+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808039070 con 0x7f47f80039b0 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.680+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480802c710 con 0x7f47f80039b0 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.680+0000 7f4810cec640 1 -- 172.21.15.66:0/3448316689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808009690 con 0x7f47f80039b0 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f4813f52640 1 -- 172.21.15.66:0/3448316689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47f80039b0 msgr2=0x7f47f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f4813f52640 1 --2- 172.21.15.66:0/3448316689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47f80039b0 0x7f47f8003dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4808009ac0 tx=0x7f47f8005830 comp rx=0 tx=0).stop 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f48124ef640 1 -- 172.21.15.66:0/3448316689 reap_dead start 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f4813f52640 1 -- 172.21.15.66:0/3448316689 shutdown_connections 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f4813f52640 1 -- 172.21.15.66:0/3448316689 >> 172.21.15.66:0/3448316689 conn(0x7f480c0678f0 msgr2=0x7f480c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f4813f52640 1 -- 172.21.15.66:0/3448316689 shutdown_connections 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.681+0000 7f4813f52640 1 -- 172.21.15.66:0/3448316689 wait complete. 2024-03-30T13:11:58.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.682+0000 7f4813f52640 1 Processor -- start 2024-03-30T13:11:58.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.682+0000 7f4813f52640 1 -- start start 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4813f52640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47f80039b0 0x7f480c161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4813f52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f480c1e6c20 0x7f480c15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4813f52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 0x7f480c0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4813f52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d3b70 con 0x7f480c0c9e10 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4813f52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d3890 con 0x7f480c1e6c20 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4813f52640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f480c0d4090 con 0x7f47f80039b0 2024-03-30T13:11:58.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 0x7f480c0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 0x7f480c0cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57234/0 (socket says 172.21.15.66:57234) 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 -- 172.21.15.66:0/3865328623 learned_addr learned my addr 172.21.15.66:0/3865328623 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48114ed640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f480c1e6c20 0x7f480c15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4811cee640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47f80039b0 0x7f480c161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 -- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47f80039b0 msgr2=0x7f480c161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47f80039b0 0x7f480c161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 -- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f480c1e6c20 msgr2=0x7f480c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f480c1e6c20 0x7f480c15fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 -- 172.21.15.66:0/3865328623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1604a0 con 0x7f480c0c9e10 2024-03-30T13:11:58.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48114ed640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f480c1e6c20 0x7f480c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f4811cee640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47f80039b0 0x7f480c161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48124ef640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 0x7f480c0cc460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f480c0ba470 tx=0x7f480400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48027fc640 1 -- 172.21.15.66:0/3865328623 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400fde0 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.683+0000 7f48027fc640 1 -- 172.21.15.66:0/3865328623 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4804015430 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.684+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c160780 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.684+0000 7f48027fc640 1 -- 172.21.15.66:0/3865328623 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804015820 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.684+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c160d40 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.686+0000 7f48027fc640 1 -- 172.21.15.66:0/3865328623 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f48040159c0 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.686+0000 7f48027fc640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47f006c530 0x7f47f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.686+0000 7f48027fc640 1 -- 172.21.15.66:0/3865328623 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f480408c6e0 con 0x7f480c0c9e10 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.686+0000 7f4811cee640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47f006c530 0x7f47f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.686+0000 7f4816c1a740 1 Processor -- start 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 -- start start 2024-03-30T13:11:58.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4811cee640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47f006c530 0x7f47f006e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4808036000 tx=0x7f4808030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e910510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e940300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e910a50 0x55ff3e940840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ff3e87a590 con 0x55ff3e93d570 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ff3e818d30 con 0x55ff3e910a50 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55ff3e876540 con 0x55ff3e93d950 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4811cee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e910510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4811cee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e910510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57242/0 (socket says 172.21.15.66:57242) 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.687+0000 7f4811cee640 1 -- 172.21.15.66:0/1705951498 learned_addr learned my addr 172.21.15.66:0/1705951498 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f4811cee640 1 -- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 msgr2=0x55ff3e940300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f4811cee640 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e940300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f48124ef640 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e910a50 0x55ff3e940840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f4811cee640 1 -- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e910a50 msgr2=0x55ff3e940840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f4811cee640 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e910a50 0x55ff3e940840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f4811cee640 1 -- 172.21.15.66:0/1705951498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3e65a920 con 0x55ff3e93d570 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f4811cee640 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e910510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4808004210 tx=0x7f480802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca63449b4cd085e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f47ef7fe640 1 -- 172.21.15.66:0/1705951498 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808050070 con 0x55ff3e93d570 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f47ef7fe640 1 -- 172.21.15.66:0/1705951498 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480802cb90 con 0x55ff3e93d570 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.688+0000 7f47ef7fe640 1 -- 172.21.15.66:0/1705951498 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808039040 con 0x55ff3e93d570 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 -- 172.21.15.66:0/1705951498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 msgr2=0x55ff3e910510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e910510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4808004210 tx=0x7f480802e000 comp rx=0 tx=0).stop 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 -- 172.21.15.66:0/1705951498 shutdown_connections 2024-03-30T13:11:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e910a50 0x55ff3e940840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e940300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 --2- 172.21.15.66:0/1705951498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e910510 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 -- 172.21.15.66:0/1705951498 >> 172.21.15.66:0/1705951498 conn(0x55ff3e912780 msgr2=0x55ff3e95b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 -- 172.21.15.66:0/1705951498 shutdown_connections 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.689+0000 7f4816c1a740 1 -- 172.21.15.66:0/1705951498 wait complete. 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.690+0000 7f4816c1a740 1 -- start start 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.690+0000 7f4816c1a740 1 Processor -- start 2024-03-30T13:11:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e96e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e938650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e936250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ff3e87a590 con 0x55ff3e93d570 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ff3e8169f0 con 0x55ff3e938b90 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55ff3e914e20 con 0x55ff3e93d950 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4811cee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e96e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.691+0000 7f4816c1a740 10 client.0 _ll_register_callbacks cb 0x55ff3e81ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e96e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57248/0 (socket says 172.21.15.66:57248) 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 -- 172.21.15.66:0/2769975601 learned_addr learned my addr 172.21.15.66:0/2769975601 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f48114ed640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e938650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f48124ef640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e936250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 -- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 msgr2=0x55ff3e938650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e938650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 -- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 msgr2=0x55ff3e936250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e936250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 -- 172.21.15.66:0/2769975601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3e54e210 con 0x55ff3e93d570 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f48124ef640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e936250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4811cee640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e96e390 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4808009be0 tx=0x7f480802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8099c3e6dbb3e395 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f47ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f47ecff9640 1 -- 172.21.15.66:0/2769975601 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808050040 con 0x55ff3e93d570 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f47ecff9640 1 -- 172.21.15.66:0/2769975601 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f4808003d90 con 0x55ff3e93d570 2024-03-30T13:11:58.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480802cb90 con 0x55ff3e93d570 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f47ecff9640 1 -- 172.21.15.66:0/2769975601 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4808063050 con 0x55ff3e93d570 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4816c1a740 10 client.5796 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.692+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55ff3e6bb0b0 con 0x55ff3e93d570 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f47ecff9640 1 -- 172.21.15.66:0/2769975601 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480802cb90 con 0x55ff3e93d570 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f47ecff9640 1 -- 172.21.15.66:0/2769975601 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f4808066020 con 0x55ff3e93d570 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f4816c1a740 10 client.5796 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f4816c1a740 10 client.5796 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f4816c1a740 20 client.5796 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e96c750 0x55ff3e95b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f4816c1a740 4 client.5796 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff3e887ad0 con 0x55ff3e96c750 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f48114ed640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e96c750 0x55ff3e95b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f47ecff9640 10 client.5796 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.693+0000 7f48114ed640 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e96c750 0x55ff3e95b260 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baed150cda5089b0 server_cookie=361ced5871d4f56 in_seq=0 out_seq=0 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f47ecff9640 1 -- 172.21.15.66:0/2769975601 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55ff3e887ad0 con 0x55ff3e96c750 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f47ecff9640 10 client.5796 handle_command_reply: tid=0 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 client.5796 shutdown 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 msgr2=0x55ff3e96e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e96e390 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4808009be0 tx=0x7f480802e070 comp rx=0 tx=0).stop 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 shutdown_connections 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e96c750 0x55ff3e95b260 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e936250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e938650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f47ecff9640 0 client.5796 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2769975601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e96e390 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 >> 172.21.15.66:0/2769975601 conn(0x55ff3e910a50 msgr2=0x55ff3e9144f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 shutdown_connections 2024-03-30T13:11:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 1 -- 172.21.15.66:0/2769975601 wait complete. 2024-03-30T13:11:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.694+0000 7f4816c1a740 20 client.5796 trim_cache size 0 max 16384 2024-03-30T13:11:58.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 Processor -- start 2024-03-30T13:11:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 -- start start 2024-03-30T13:11:58.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e938b90 0x55ff3e937880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e93d570 0x55ff3e95ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 0x55ff3e90f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f480c201920 con 0x55ff3e93d950 2024-03-30T13:11:58.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f480c200940 con 0x55ff3e93d570 2024-03-30T13:11:58.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f480c1ffd70 con 0x55ff3e938b90 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.709+0000 7f48124ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 0x55ff3e90f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 0x55ff3e90f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57258/0 (socket says 172.21.15.66:57258) 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 -- 172.21.15.66:0/2527048276 learned_addr learned my addr 172.21.15.66:0/2527048276 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48114ed640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e93d570 0x55ff3e95ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f4811cee640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e938b90 0x55ff3e937880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 -- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e938b90 msgr2=0x55ff3e937880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e938b90 0x55ff3e937880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 -- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e93d570 msgr2=0x55ff3e95ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e93d570 0x55ff3e95ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 -- 172.21.15.66:0/2527048276 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3e5272d0 con 0x55ff3e93d950 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48114ed640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e93d570 0x55ff3e95ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f4811cee640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e938b90 0x55ff3e937880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f48124ef640 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 0x55ff3e90f780 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4804010010 tx=0x7f4804009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d5821fa77fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f47eeffd640 1 -- 172.21.15.66:0/2527048276 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480401bde0 con 0x55ff3e93d950 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f47eeffd640 1 -- 172.21.15.66:0/2527048276 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48040587e0 con 0x55ff3e93d950 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.710+0000 7f47eeffd640 1 -- 172.21.15.66:0/2527048276 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804059930 con 0x55ff3e93d950 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 -- 172.21.15.66:0/2527048276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 msgr2=0x55ff3e90f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 0x55ff3e90f780 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4804010010 tx=0x7f4804009400 comp rx=0 tx=0).stop 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 -- 172.21.15.66:0/2527048276 shutdown_connections 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d950 0x55ff3e90f780 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e93d570 0x55ff3e95ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2527048276 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e938b90 0x55ff3e937880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 -- 172.21.15.66:0/2527048276 >> 172.21.15.66:0/2527048276 conn(0x55ff3e910a50 msgr2=0x55ff3e9172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 -- 172.21.15.66:0/2527048276 shutdown_connections 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 -- 172.21.15.66:0/2527048276 wait complete. 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 -- start start 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.711+0000 7f4816c1a740 1 Processor -- start 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e97f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e90f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e90fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ff3e944250 con 0x55ff3e93d570 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f480c2002f0 con 0x55ff3e938b90 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55ff3e8850f0 con 0x55ff3e93d950 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4816c1a740 10 client.0 _ll_register_callbacks cb 0x7f480804e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f48114ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e90f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f48114ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e90f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57260/0 (socket says 172.21.15.66:57260) 2024-03-30T13:11:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f48114ed640 1 -- 172.21.15.66:0/2905467234 learned_addr learned my addr 172.21.15.66:0/2905467234 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.712+0000 7f4811cee640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e97f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48124ef640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e90fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48114ed640 1 -- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 msgr2=0x55ff3e90fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48114ed640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e90fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48114ed640 1 -- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 msgr2=0x55ff3e97f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48114ed640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e97f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48114ed640 1 -- 172.21.15.66:0/2905467234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3e93ef80 con 0x55ff3e93d570 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f48114ed640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e90f660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55ff3e941020 tx=0x7f47fc001e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1344c2e5c30c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f47ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f47ecff9640 1 -- 172.21.15.66:0/2905467234 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc01d070 con 0x55ff3e93d570 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f47ecff9640 1 -- 172.21.15.66:0/2905467234 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f47fc019450 con 0x55ff3e93d570 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48040587e0 con 0x55ff3e93d570 2024-03-30T13:11:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f47ecff9640 1 -- 172.21.15.66:0/2905467234 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f47fc028050 con 0x55ff3e93d570 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f4816c1a740 10 client.5802 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55ff3e9da550 con 0x55ff3e93d570 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f47ecff9640 1 -- 172.21.15.66:0/2905467234 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48040587e0 con 0x55ff3e93d570 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f47ecff9640 1 -- 172.21.15.66:0/2905467234 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f47fc02b020 con 0x55ff3e93d570 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f4816c1a740 10 client.5802 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.713+0000 7f4816c1a740 10 client.5802 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 20 client.5802 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e9381f0 0x55ff3e9385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 4 client.5802 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628326}] 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628326}) -- 0x55ff3e937330 con 0x55ff3e9381f0 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4811cee640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e9381f0 0x55ff3e9385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f47ecff9640 10 client.5802 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4811cee640 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e9381f0 0x55ff3e9385d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49ad8a9f1a46596 server_cookie=b26850e7f390df41 in_seq=0 out_seq=0 2024-03-30T13:11:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f47ecff9640 1 -- 172.21.15.66:0/2905467234 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4336 (crc 0 0 0) 0x55ff3e937330 con 0x55ff3e9381f0 2024-03-30T13:11:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f47ecff9640 10 client.5802 handle_command_reply: tid=0 2024-03-30T13:11:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 client.5802 shutdown 2024-03-30T13:11:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 msgr2=0x55ff3e90f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e90f660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55ff3e941020 tx=0x7f47fc001e40 comp rx=0 tx=0).stop 2024-03-30T13:11:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 shutdown_connections 2024-03-30T13:11:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ff3e9381f0 0x55ff3e9385d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ff3e93d950 0x55ff3e90fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff3e93d570 0x55ff3e90f660 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f47ecff9640 0 client.5802 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 --2- 172.21.15.66:0/2905467234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff3e938b90 0x55ff3e97f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.714+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 >> 172.21.15.66:0/2905467234 conn(0x55ff3e910a50 msgr2=0x55ff3e935f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 shutdown_connections 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4816c1a740 1 -- 172.21.15.66:0/2905467234 wait complete. 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4816c1a740 20 client.5802 trim_cache size 0 max 16384 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr_user.c", 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628326, 2024-03-30T13:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.843040+0000", 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.349203+0000", 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1232, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.349203+0000", 2024-03-30T13:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1232, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.843040+0000" 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1232, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.843040+0000" 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 74, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 38, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:58.747 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47f006c530 msgr2=0x7f47f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47f006c530 0x7f47f006e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4808036000 tx=0x7f4808030000 comp rx=0 tx=0).stop 2024-03-30T13:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 msgr2=0x7f480c0cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 0x7f480c0cc460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f480c0ba470 tx=0x7f480400f580 comp rx=0 tx=0).stop 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 shutdown_connections 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f47f006c530 0x7f47f006e9f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480c0c9e10 0x7f480c0cc460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f480c1e6c20 0x7f480c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 --2- 172.21.15.66:0/3865328623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47f80039b0 0x7f480c161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 >> 172.21.15.66:0/3865328623 conn(0x7f480c0678f0 msgr2=0x7f480c0b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 shutdown_connections 2024-03-30T13:11:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.715+0000 7f4813f52640 1 -- 172.21.15.66:0/3865328623 wait complete. 2024-03-30T13:11:58.752 DEBUG:tasks.cephfs.test_uninlining:getting inode info #8 of 23:1099511628324 2024-03-30T13:11:58.752 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628324 2024-03-30T13:11:58.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628324 2024-03-30T13:11:58.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.861+0000 7f9386ab3640 1 Processor -- start 2024-03-30T13:11:58.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.861+0000 7f9386ab3640 1 -- start start 2024-03-30T13:11:58.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f9386ab3640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93800b5dc0 0x7f93800b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f9386ab3640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f93800ba090 0x7f93800d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f9386ab3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93800b66e0 con 0x7f938006c620 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f9386ab3640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f93800d2ff0 con 0x7f93800b5dc0 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f9386ab3640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f93800d32d0 con 0x7f93800ba090 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f938484f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93800b5dc0 0x7f93800b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f938484f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93800b5dc0 0x7f93800b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57270/0 (socket says 172.21.15.66:57270) 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f938484f640 1 -- 172.21.15.66:0/4190285078 learned_addr learned my addr 172.21.15.66:0/4190285078 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.862+0000 7f937ffff640 1 --2- 172.21.15.66:0/4190285078 >> v2:172.21.15.117:3300/0 conn(0x7f93800ba090 0x7f93800d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f938484f640 1 -- 172.21.15.66:0/4190285078 >> v2:172.21.15.117:3300/0 conn(0x7f93800ba090 msgr2=0x7f93800d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f938484f640 1 --2- 172.21.15.66:0/4190285078 >> v2:172.21.15.117:3300/0 conn(0x7f93800ba090 0x7f93800d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f938484f640 1 -- 172.21.15.66:0/4190285078 >> v1:172.21.15.117:6789/0 conn(0x7f938006c620 legacy=0x7f938006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f938484f640 1 -- 172.21.15.66:0/4190285078 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93800d35b0 con 0x7f93800b5dc0 2024-03-30T13:11:58.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f937ffff640 1 --2- 172.21.15.66:0/4190285078 >> v2:172.21.15.117:3300/0 conn(0x7f93800ba090 0x7f93800d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f938484f640 1 --2- 172.21.15.66:0/4190285078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800b5dc0 0x7f93800b61a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f93800b7240 tx=0x7f937402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807d0e0c225882cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f937f7fe640 1 -- 172.21.15.66:0/4190285078 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374034070 con 0x7f93800b5dc0 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.863+0000 7f937f7fe640 1 -- 172.21.15.66:0/4190285078 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9374030860 con 0x7f93800b5dc0 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f937f7fe640 1 -- 172.21.15.66:0/4190285078 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374030c30 con 0x7f93800b5dc0 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 -- 172.21.15.66:0/4190285078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800b5dc0 msgr2=0x7f93800b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 --2- 172.21.15.66:0/4190285078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800b5dc0 0x7f93800b61a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f93800b7240 tx=0x7f937402b9a0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 -- 172.21.15.66:0/4190285078 shutdown_connections 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 --2- 172.21.15.66:0/4190285078 >> v2:172.21.15.117:3300/0 conn(0x7f93800ba090 0x7f93800d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 --2- 172.21.15.66:0/4190285078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800b5dc0 0x7f93800b61a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 -- 172.21.15.66:0/4190285078 >> 172.21.15.66:0/4190285078 conn(0x7f93800678f0 msgr2=0x7f93800d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 -- 172.21.15.66:0/4190285078 shutdown_connections 2024-03-30T13:11:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.864+0000 7f9386ab3640 1 -- 172.21.15.66:0/4190285078 wait complete. 2024-03-30T13:11:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.865+0000 7f9386ab3640 1 Processor -- start 2024-03-30T13:11:58.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.865+0000 7f9386ab3640 1 -- start start 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9386ab3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938006c620 0x7f93800cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9386ab3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93800b5dc0 0x7f9380155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9386ab3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 0x7f93801566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9386ab3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93800d4090 con 0x7f93800ba090 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9386ab3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93800d3b70 con 0x7f93800b5dc0 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9386ab3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f93800b83e0 con 0x7f938006c620 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9385050640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 0x7f93801566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9385050640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 0x7f93801566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57276/0 (socket says 172.21.15.66:57276) 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f9385050640 1 -- 172.21.15.66:0/2923895169 learned_addr learned my addr 172.21.15.66:0/2923895169 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.866+0000 7f938484f640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938006c620 0x7f93800cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f937ffff640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93800b5dc0 0x7f9380155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9385050640 1 -- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938006c620 msgr2=0x7f93800cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9385050640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938006c620 0x7f93800cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9385050640 1 -- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93800b5dc0 msgr2=0x7f9380155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9385050640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93800b5dc0 0x7f9380155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9385050640 1 -- 172.21.15.66:0/2923895169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93801f78b0 con 0x7f93800ba090 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f938484f640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938006c620 0x7f93800cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9385050640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 0x7f93801566a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f937800c7d0 tx=0x7f937800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f937d7fa640 1 -- 172.21.15.66:0/2923895169 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378003b70 con 0x7f93800ba090 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f937d7fa640 1 -- 172.21.15.66:0/2923895169 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9378003d30 con 0x7f93800ba090 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801f7b90 con 0x7f93800ba090 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f937d7fa640 1 -- 172.21.15.66:0/2923895169 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937800aa30 con 0x7f93800ba090 2024-03-30T13:11:58.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.867+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801f8150 con 0x7f93800ba090 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.869+0000 7f937d7fa640 1 -- 172.21.15.66:0/2923895169 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f937800abd0 con 0x7f93800ba090 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.869+0000 7f937d7fa640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f936406c530 0x7f936406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.869+0000 7f937d7fa640 1 -- 172.21.15.66:0/2923895169 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f937808b690 con 0x7f93800ba090 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.869+0000 7f938484f640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f936406c530 0x7f936406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.870+0000 7f938484f640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f936406c530 0x7f936406e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9374036000 tx=0x7f937403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.870+0000 7f938977a740 1 Processor -- start 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.870+0000 7f938977a740 1 -- start start 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x555687602510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938977a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687632300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555687602a50 0x555687632840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938977a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55568756c590 con 0x55568762f950 2024-03-30T13:11:58.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55568750ad30 con 0x55568762f570 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555687568540 con 0x555687602a50 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687632300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687632300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57280/0 (socket says 172.21.15.66:57280) 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 -- 172.21.15.66:0/1900139584 learned_addr learned my addr 172.21.15.66:0/1900139584 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938484f640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x555687602510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f9385050640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555687602a50 0x555687632840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 -- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555687602a50 msgr2=0x555687632840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555687602a50 0x555687632840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 -- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 msgr2=0x555687602510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x555687602510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 -- 172.21.15.66:0/1900139584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568734c920 con 0x55568762f950 2024-03-30T13:11:58.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f938484f640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x555687602510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f9385050640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555687602a50 0x555687632840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.871+0000 7f937ffff640 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687632300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b8c0 tx=0x7f936c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f6ac8d119dc8db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f936a7fc640 1 -- 172.21.15.66:0/1900139584 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c017020 con 0x55568762f950 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f936a7fc640 1 -- 172.21.15.66:0/1900139584 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c004230 con 0x55568762f950 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f936a7fc640 1 -- 172.21.15.66:0/1900139584 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c00f760 con 0x55568762f950 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 -- 172.21.15.66:0/1900139584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 msgr2=0x555687632300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687632300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b8c0 tx=0x7f936c00bd80 comp rx=0 tx=0).stop 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 -- 172.21.15.66:0/1900139584 shutdown_connections 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555687602a50 0x555687632840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687632300 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 --2- 172.21.15.66:0/1900139584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x555687602510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 -- 172.21.15.66:0/1900139584 >> 172.21.15.66:0/1900139584 conn(0x555687604780 msgr2=0x55568764d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 -- 172.21.15.66:0/1900139584 shutdown_connections 2024-03-30T13:11:58.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.872+0000 7f938977a740 1 -- 172.21.15.66:0/1900139584 wait complete. 2024-03-30T13:11:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.873+0000 7f938977a740 1 -- start start 2024-03-30T13:11:58.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.873+0000 7f938977a740 1 Processor -- start 2024-03-30T13:11:58.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687635d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f950 0x555687627a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 0x555687633d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555687606e20 con 0x55568764db80 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5556875089f0 con 0x55568762f950 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555687609660 con 0x55568762f570 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938977a740 10 client.0 _ll_register_callbacks cb 0x55568750cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 0x555687633d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 0x555687633d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57290/0 (socket says 172.21.15.66:57290) 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 -- 172.21.15.66:0/2983164227 learned_addr learned my addr 172.21.15.66:0/2983164227 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f937ffff640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f950 0x555687627a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938484f640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687635d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 -- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 msgr2=0x555687635d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687635d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 -- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f950 msgr2=0x555687627a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f950 0x555687627a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f9385050640 1 -- 172.21.15.66:0/2983164227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555687240210 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.874+0000 7f938484f640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687635d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f9385050640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 0x555687633d20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9378059420 tx=0x7f9378059100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4850d318471b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f935f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f935f7fe640 1 -- 172.21.15.66:0/2983164227 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937805d840 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c004230 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f935f7fe640 1 -- 172.21.15.66:0/2983164227 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f93780588e0 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f935f7fe640 1 -- 172.21.15.66:0/2983164227 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9378062070 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 10 client.5814 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5556873ad0b0 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f935f7fe640 1 -- 172.21.15.66:0/2983164227 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c004230 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f935f7fe640 1 -- 172.21.15.66:0/2983164227 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f93780551e0 con 0x55568764db80 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 10 client.5814 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 10 client.5814 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 20 client.5814 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5556876614b0 0x555687661890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 4 client.5814 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555687579ad0 con 0x5556876614b0 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.875+0000 7f938484f640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5556876614b0 0x555687661890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.876+0000 7f935f7fe640 10 client.5814 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.876+0000 7f938484f640 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5556876614b0 0x555687661890 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d9dfff889a9e004 server_cookie=704d66c911fa4c8a in_seq=0 out_seq=0 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.876+0000 7f935f7fe640 1 -- 172.21.15.66:0/2983164227 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x555687579ad0 con 0x5556876614b0 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.876+0000 7f935f7fe640 10 client.5814 handle_command_reply: tid=0 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.876+0000 7f938977a740 1 client.5814 shutdown 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 msgr2=0x555687633d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 0x555687633d20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9378059420 tx=0x7f9378059100 comp rx=0 tx=0).stop 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 shutdown_connections 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5556876614b0 0x555687661890 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568764db80 0x555687633d20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f950 0x555687627a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f935f7fe640 0 client.5814 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 --2- 172.21.15.66:0/2983164227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687635d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 >> 172.21.15.66:0/2983164227 conn(0x555687602a50 msgr2=0x555687650780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 shutdown_connections 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 1 -- 172.21.15.66:0/2983164227 wait complete. 2024-03-30T13:11:58.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.877+0000 7f938977a740 20 client.5814 trim_cache size 0 max 16384 2024-03-30T13:11:58.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.891+0000 7f938977a740 1 Processor -- start 2024-03-30T13:11:58.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.891+0000 7f938977a740 1 -- start start 2024-03-30T13:11:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.892+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x5556876506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.892+0000 7f938977a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x55568762a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568764db80 0x55568762a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f938977a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9380201280 con 0x55568762f950 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93802002a0 con 0x55568762f570 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f93800cc270 con 0x55568764db80 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x55568762a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x55568762a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57300/0 (socket says 172.21.15.66:57300) 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 -- 172.21.15.66:0/225828668 learned_addr learned my addr 172.21.15.66:0/225828668 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f938484f640 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x5556876506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f9385050640 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568764db80 0x55568762a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 -- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568764db80 msgr2=0x55568762a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568764db80 0x55568762a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 -- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 msgr2=0x5556876506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x5556876506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 -- 172.21.15.66:0/225828668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556872192d0 con 0x55568762f950 2024-03-30T13:11:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f937ffff640 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x55568762a1b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f936c016040 tx=0x7f936c009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e1ddf7e7e17061 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f9369ffb640 1 -- 172.21.15.66:0/225828668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c02b070 con 0x55568762f950 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f9369ffb640 1 -- 172.21.15.66:0/225828668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c004300 con 0x55568762f950 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.893+0000 7f9369ffb640 1 -- 172.21.15.66:0/225828668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c017070 con 0x55568762f950 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 -- 172.21.15.66:0/225828668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 msgr2=0x55568762a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x55568762a1b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f936c016040 tx=0x7f936c009000 comp rx=0 tx=0).stop 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 -- 172.21.15.66:0/225828668 shutdown_connections 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568764db80 0x55568762a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x55568762a1b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 --2- 172.21.15.66:0/225828668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568762f570 0x5556876506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 -- 172.21.15.66:0/225828668 >> 172.21.15.66:0/225828668 conn(0x555687602a50 msgr2=0x555687635b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 -- 172.21.15.66:0/225828668 shutdown_connections 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.894+0000 7f938977a740 1 -- 172.21.15.66:0/225828668 wait complete. 2024-03-30T13:11:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.895+0000 7f938977a740 1 -- start start 2024-03-30T13:11:58.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.895+0000 7f938977a740 1 Processor -- start 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.895+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687625e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687626370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568764db80 0x555687630d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555687633300 con 0x55568762f950 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93801ffc50 con 0x55568764db80 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555687631c00 con 0x55568762f570 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 10 client.0 _ll_register_callbacks cb 0x7f9378001830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687626370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687626370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57310/0 (socket says 172.21.15.66:57310) 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 -- 172.21.15.66:0/257888350 learned_addr learned my addr 172.21.15.66:0/257888350 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938484f640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687625e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f9385050640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568764db80 0x555687630d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 -- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 msgr2=0x555687625e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687625e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 -- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568764db80 msgr2=0x555687630d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568764db80 0x555687630d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 -- 172.21.15.66:0/257888350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555687650860 con 0x55568762f950 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f937ffff640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687626370 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f936c008010 tx=0x7f936c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e26513bb2a74b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f935f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f935f7fe640 1 -- 172.21.15.66:0/257888350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c02b070 con 0x55568762f950 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f935f7fe640 1 -- 172.21.15.66:0/257888350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f936c023040 con 0x55568762f950 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c004300 con 0x55568762f950 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.896+0000 7f935f7fe640 1 -- 172.21.15.66:0/257888350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f936c017050 con 0x55568762f950 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 10 client.5820 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55568762ab90 con 0x55568762f950 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f935f7fe640 1 -- 172.21.15.66:0/257888350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c004300 con 0x55568762f950 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f935f7fe640 1 -- 172.21.15.66:0/257888350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f936c003610 con 0x55568762f950 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 10 client.5820 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 10 client.5820 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 20 client.5820 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55568765c750 0x555687631240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 4 client.5820 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628324}] 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628324}) -- 0x555687573e20 con 0x55568765c750 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938484f640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55568765c750 0x555687631240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f935f7fe640 10 client.5820 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.897+0000 7f938484f640 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55568765c750 0x555687631240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b51b40714c8f33 server_cookie=1291e915087373c5 in_seq=0 out_seq=0 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f935f7fe640 1 -- 172.21.15.66:0/257888350 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4340 (crc 0 0 0) 0x555687573e20 con 0x55568765c750 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f935f7fe640 10 client.5820 handle_command_reply: tid=0 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 client.5820 shutdown 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 msgr2=0x555687626370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687626370 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f936c008010 tx=0x7f936c004730 comp rx=0 tx=0).stop 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 shutdown_connections 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55568765c750 0x555687631240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55568764db80 0x555687630d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55568762f950 0x555687626370 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f935f7fe640 0 client.5820 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 --2- 172.21.15.66:0/257888350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55568762f570 0x555687625e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 >> 172.21.15.66:0/257888350 conn(0x555687602a50 msgr2=0x555687671690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 shutdown_connections 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 1 -- 172.21.15.66:0/257888350 wait complete. 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f938977a740 20 client.5820 trim_cache size 0 max 16384 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr_security.c", 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628324, 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.848308+0000", 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.347203+0000", 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:58.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1473, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.347203+0000", 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1473, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.848308+0000" 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:58.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1473, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.848308+0000" 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 76, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 34, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:58.917 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:58.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:58.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:58.921 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f936406c530 msgr2=0x7f936406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f9386ab3640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f936406c530 0x7f936406e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9374036000 tx=0x7f937403b040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 msgr2=0x7f93801566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.898+0000 7f9386ab3640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 0x7f93801566a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f937800c7d0 tx=0x7f937800cc90 comp rx=0 tx=0).stop 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 shutdown_connections 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f936406c530 0x7f936406e9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93800ba090 0x7f93801566a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93800b5dc0 0x7f9380155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 --2- 172.21.15.66:0/2923895169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f938006c620 0x7f93800cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 >> 172.21.15.66:0/2923895169 conn(0x7f93800678f0 msgr2=0x7f93800b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 shutdown_connections 2024-03-30T13:11:58.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:58.899+0000 7f9386ab3640 1 -- 172.21.15.66:0/2923895169 wait complete. 2024-03-30T13:11:58.924 DEBUG:tasks.cephfs.test_uninlining:getting inode info #9 of 23:1099511628309 2024-03-30T13:11:58.924 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628309 2024-03-30T13:11:58.924 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628309 2024-03-30T13:11:59.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.031+0000 7f0dc4d52640 1 Processor -- start 2024-03-30T13:11:59.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.031+0000 7f0dc4d52640 1 -- start start 2024-03-30T13:11:59.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dc4d52640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0dc0167120 0x7f0dc0169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dc4d52640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc00d2ff0 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dc4d52640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc00d32d0 con 0x7f0dc00ba090 2024-03-30T13:11:59.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dc4d52640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc00d2d90 con 0x7f0dc0167120 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dbed9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0dc0167120 0x7f0dc0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dbed9d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0dc0167120 0x7f0dc0169510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57314/0 (socket says 172.21.15.66:57314) 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dbe59c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0dc00b5dc0 0x7f0dc00b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37512/0 (socket says 172.21.15.66:37512) 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dbed9d640 1 -- 172.21.15.66:0/249499737 learned_addr learned my addr 172.21.15.66:0/249499737 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dbed9d640 1 -- 172.21.15.66:0/249499737 >> v1:172.21.15.117:6789/0 conn(0x7f0dc00ba090 legacy=0x7f0dc00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.032+0000 7f0dbed9d640 1 -- 172.21.15.66:0/249499737 >> v1:172.21.15.66:6789/0 conn(0x7f0dc00b5dc0 legacy=0x7f0dc00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.033+0000 7f0dbdd9b640 1 -- 172.21.15.66:0/249499737 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992577893 0 0) 0x7f0dc00d2ff0 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.033+0000 7f0dbed9d640 1 -- 172.21.15.66:0/249499737 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc00d35b0 con 0x7f0dc0167120 2024-03-30T13:11:59.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.033+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/249499737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc0167120 0x7f0dc0169510 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009e80 tx=0x7f0db802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa692b9993b8e2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.033+0000 7f0dbdd9b640 1 -- 172.21.15.66:0/249499737 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db802caf0 con 0x7f0dc0167120 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.033+0000 7f0dbdd9b640 1 -- 172.21.15.66:0/249499737 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db802cc90 con 0x7f0dc0167120 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.033+0000 7f0dbdd9b640 1 -- 172.21.15.66:0/249499737 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8034610 con 0x7f0dc0167120 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 -- 172.21.15.66:0/249499737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc0167120 msgr2=0x7f0dc0169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/249499737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc0167120 0x7f0dc0169510 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009e80 tx=0x7f0db802bf50 comp rx=0 tx=0).stop 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 -- 172.21.15.66:0/249499737 shutdown_connections 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/249499737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc0167120 0x7f0dc0169510 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 -- 172.21.15.66:0/249499737 >> 172.21.15.66:0/249499737 conn(0x7f0dc00678f0 msgr2=0x7f0dc00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 -- 172.21.15.66:0/249499737 shutdown_connections 2024-03-30T13:11:59.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.034+0000 7f0dc4d52640 1 -- 172.21.15.66:0/249499737 wait complete. 2024-03-30T13:11:59.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.035+0000 7f0dc4d52640 1 Processor -- start 2024-03-30T13:11:59.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.035+0000 7f0dc4d52640 1 -- start start 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dc4d52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 0x7f0dc0157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dc4d52640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dc00ba090 0x7f0dc00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dc4d52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc0167120 0x7f0dc00b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dc4d52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc00d4090 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dc4d52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc00d3b70 con 0x7f0dc0167120 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dc4d52640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc00b83e0 con 0x7f0dc00ba090 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dbe59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 0x7f0dc0157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dbe59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 0x7f0dc0157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57316/0 (socket says 172.21.15.66:57316) 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dbe59c640 1 -- 172.21.15.66:0/3771224329 learned_addr learned my addr 172.21.15.66:0/3771224329 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc0167120 0x7f0dc00b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.036+0000 7f0db7fff640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dc00ba090 0x7f0dc00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dbe59c640 1 -- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dc00ba090 msgr2=0x7f0dc00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dc00ba090 0x7f0dc00cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dbe59c640 1 -- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc0167120 msgr2=0x7f0dc00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc0167120 0x7f0dc00b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dbe59c640 1 -- 172.21.15.66:0/3771224329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc0157dc0 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 0x7f0dc0157880 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c430 tx=0x7f0db000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0db6ffd640 1 -- 172.21.15.66:0/3771224329 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00074e0 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0db6ffd640 1 -- 172.21.15.66:0/3771224329 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db00047e0 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc00ccb80 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0db6ffd640 1 -- 172.21.15.66:0/3771224329 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0004bd0 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.037+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc00cd140 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.039+0000 7f0db6ffd640 1 -- 172.21.15.66:0/3771224329 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0db0011070 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.039+0000 7f0db6ffd640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0da006c530 0x7f0da006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.039+0000 7f0db7fff640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0da006c530 0x7f0da006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.039+0000 7f0db6ffd640 1 -- 172.21.15.66:0/3771224329 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0db008b370 con 0x7f0dc00b5dc0 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.040+0000 7f0db7fff640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0da006c530 0x7f0da006e9f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00b9050 tx=0x7f0da8007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.040+0000 7f0dc7a1c740 1 Processor -- start 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.040+0000 7f0dc7a1c740 1 -- start start 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bde510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 0x562303c0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303bdea50 0x562303c0e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562303b48590 con 0x562303c0b950 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562303ae6d30 con 0x562303bdea50 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x562303b44540 con 0x562303c0b570 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 0x562303c0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 0x562303c0e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57328/0 (socket says 172.21.15.66:57328) 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 -- 172.21.15.66:0/2579706860 learned_addr learned my addr 172.21.15.66:0/2579706860 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bde510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303bdea50 0x562303c0e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 -- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 msgr2=0x562303bde510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bde510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 -- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303bdea50 msgr2=0x562303c0e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303bdea50 0x562303c0e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0db7fff640 1 -- 172.21.15.66:0/2579706860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562303928920 con 0x562303c0b950 2024-03-30T13:11:59.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bde510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.041+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303bdea50 0x562303c0e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0db7fff640 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 0x562303c0e300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0da8003cc0 tx=0x7f0da8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bae74c34636e12 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0d9ffff640 1 -- 172.21.15.66:0/2579706860 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8025070 con 0x562303c0b950 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0d9ffff640 1 -- 172.21.15.66:0/2579706860 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da801b050 con 0x562303c0b950 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0d9ffff640 1 -- 172.21.15.66:0/2579706860 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80045d0 con 0x562303c0b950 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2579706860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 msgr2=0x562303c0e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 0x562303c0e300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0da8003cc0 tx=0x7f0da8012000 comp rx=0 tx=0).stop 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2579706860 shutdown_connections 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303bdea50 0x562303c0e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b950 0x562303c0e300 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2579706860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bde510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.042+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2579706860 >> 172.21.15.66:0/2579706860 conn(0x562303be0780 msgr2=0x562303c29300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.043+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2579706860 shutdown_connections 2024-03-30T13:11:59.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.043+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2579706860 wait complete. 2024-03-30T13:11:59.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.043+0000 7f0dc7a1c740 1 -- start start 2024-03-30T13:11:59.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.043+0000 7f0dc7a1c740 1 Processor -- start 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303c299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bfb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303c39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562303be2e20 con 0x562303c10c50 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562303ae49f0 con 0x562303c0b950 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x562303be5660 con 0x562303c0b570 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dc7a1c740 10 client.0 _ll_register_callbacks cb 0x562303ae8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303c39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303c39640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57342/0 (socket says 172.21.15.66:57342) 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dbed9d640 1 -- 172.21.15.66:0/2210517274 learned_addr learned my addr 172.21.15.66:0/2210517274 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303c299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.044+0000 7f0db7fff640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bfb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dbed9d640 1 -- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 msgr2=0x562303c299d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303c299d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dbed9d640 1 -- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 msgr2=0x562303bfb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bfb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dbed9d640 1 -- 172.21.15.66:0/2210517274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230381c210 con 0x562303c10c50 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303c39640 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0db8036040 tx=0x7f0db8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcb4094aad06c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0d9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/2210517274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8044070 con 0x562303c10c50 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/2210517274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0db804acc0 con 0x562303c10c50 2024-03-30T13:11:59.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da801b050 con 0x562303c10c50 2024-03-30T13:11:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/2210517274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0db8009af0 con 0x562303c10c50 2024-03-30T13:11:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dc7a1c740 10 client.5835 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5623039890b0 con 0x562303c10c50 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/2210517274 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da801b050 con 0x562303c10c50 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/2210517274 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f0db8004040 con 0x562303c10c50 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.045+0000 7f0dc7a1c740 10 client.5835 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dc7a1c740 10 client.5835 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dc7a1c740 20 client.5835 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bf8950 0x562303bf8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dc7a1c740 4 client.5835 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562303b55ad0 con 0x562303bf8950 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bf8950 0x562303bf8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bf8950 0x562303bf8d60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1d7274de16ba53 server_cookie=58b8f5b64b34cbdc in_seq=0 out_seq=0 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.046+0000 7f0d9d7fa640 10 client.5835 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/2210517274 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x562303b55ad0 con 0x562303bf8950 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0d9d7fa640 10 client.5835 handle_command_reply: tid=0 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 client.5835 shutdown 2024-03-30T13:11:59.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 msgr2=0x562303c39640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303c39640 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0db8036040 tx=0x7f0db8004ab0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 shutdown_connections 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bf8950 0x562303bf8d60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303c39640 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bfb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0d9d7fa640 0 client.5835 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/2210517274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303c299d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 >> 172.21.15.66:0/2210517274 conn(0x562303bdea50 msgr2=0x562303be2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 shutdown_connections 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/2210517274 wait complete. 2024-03-30T13:11:59.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.047+0000 7f0dc7a1c740 20 client.5835 trim_cache size 0 max 16384 2024-03-30T13:11:59.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.061+0000 7f0dc7a1c740 1 Processor -- start 2024-03-30T13:11:59.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 -- start start 2024-03-30T13:11:59.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bfb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bf7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303bf8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0200fd0 con 0x562303c10c50 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01ffff0 con 0x562303c0b950 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc015f790 con 0x562303c0b570 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303bf8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303bf8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57356/0 (socket says 172.21.15.66:57356) 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.062+0000 7f0dbed9d640 1 -- 172.21.15.66:0/3279735612 learned_addr learned my addr 172.21.15.66:0/3279735612 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0db7fff640 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bf7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bfb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbed9d640 1 -- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 msgr2=0x562303bfb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bfb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbed9d640 1 -- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 msgr2=0x562303bf7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bf7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbed9d640 1 -- 172.21.15.66:0/3279735612 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623037f52d0 con 0x562303c10c50 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303bf8320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0db80072a0 tx=0x7f0db8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0ab2fecab0981a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0d9f7fe640 1 -- 172.21.15.66:0/3279735612 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8044070 con 0x562303c10c50 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0d9f7fe640 1 -- 172.21.15.66:0/3279735612 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db8003fd0 con 0x562303c10c50 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.063+0000 7f0d9f7fe640 1 -- 172.21.15.66:0/3279735612 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80047b0 con 0x562303c10c50 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/3279735612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 msgr2=0x562303bf8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303bf8320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0db80072a0 tx=0x7f0db8041000 comp rx=0 tx=0).stop 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/3279735612 shutdown_connections 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c10c50 0x562303bf8320 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c0b950 0x562303bf7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/3279735612 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b570 0x562303bfb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/3279735612 >> 172.21.15.66:0/3279735612 conn(0x562303bdea50 msgr2=0x562303be52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/3279735612 shutdown_connections 2024-03-30T13:11:59.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/3279735612 wait complete. 2024-03-30T13:11:59.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 -- start start 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.064+0000 7f0dc7a1c740 1 Processor -- start 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 0x562303c9ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b950 0x562303bf98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c10c50 0x562303bf9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562303be3190 con 0x562303c0b570 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01ffa50 con 0x562303c10c50 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x562303ca5dc0 con 0x562303c0b950 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dc7a1c740 10 client.0 _ll_register_callbacks cb 0x7f0db802cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dbe59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 0x562303c9ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dbe59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 0x562303c9ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57360/0 (socket says 172.21.15.66:57360) 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.065+0000 7f0dbe59c640 1 -- 172.21.15.66:0/4270934360 learned_addr learned my addr 172.21.15.66:0/4270934360 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0db7fff640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b950 0x562303bf98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c10c50 0x562303bf9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbe59c640 1 -- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b950 msgr2=0x562303bf98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b950 0x562303bf98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbe59c640 1 -- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c10c50 msgr2=0x562303bf9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c10c50 0x562303bf9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbe59c640 1 -- 172.21.15.66:0/4270934360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562303c09320 con 0x562303c0b570 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0db7fff640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b950 0x562303bf98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbed9d640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c10c50 0x562303bf9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dbe59c640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 0x562303c9ef10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0db0059350 tx=0x7f0db0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf09912abe101aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0d9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/4270934360 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0089330 con 0x562303c0b570 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/4270934360 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0db008b0c0 con 0x562303c0b570 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db8003fd0 con 0x562303c0b570 2024-03-30T13:11:59.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/4270934360 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0db005d120 con 0x562303c0b570 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 10 client.5841 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562303c4ca80 con 0x562303c0b570 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/4270934360 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db8003fd0 con 0x562303c0b570 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/4270934360 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f0db0055800 con 0x562303c0b570 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 10 client.5841 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 10 client.5841 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 20 client.5841 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bfa740 0x562303ca4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 4 client.5841 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628309}] 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.066+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628309}) -- 0x562303b4fe20 con 0x562303bfa740 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0db7fff640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bfa740 0x562303ca4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0d9d7fa640 10 client.5841 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0db7fff640 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bfa740 0x562303ca4e50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ed64d25d39500a server_cookie=d3cdc7e706ca6785 in_seq=0 out_seq=0 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0d9d7fa640 1 -- 172.21.15.66:0/4270934360 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4329 (crc 0 0 0) 0x562303b4fe20 con 0x562303bfa740 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0d9d7fa640 10 client.5841 handle_command_reply: tid=0 2024-03-30T13:11:59.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 client.5841 shutdown 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 msgr2=0x562303c9ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 0x562303c9ef10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0db0059350 tx=0x7f0db0009040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 shutdown_connections 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562303bfa740 0x562303ca4e50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562303c10c50 0x562303bf9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562303c0b950 0x562303bf98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 --2- 172.21.15.66:0/4270934360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562303c0b570 0x562303c9ef10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 >> 172.21.15.66:0/4270934360 conn(0x562303bdea50 msgr2=0x562303c0e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.067+0000 7f0d9d7fa640 0 client.5841 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 shutdown_connections 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc7a1c740 1 -- 172.21.15.66:0/4270934360 wait complete. 2024-03-30T13:11:59.078 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc7a1c740 20 client.5841 trim_cache size 0 max 16384 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/Makefile", 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628309, 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.875324+0000", 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.332203+0000", 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:59.079 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 378, 2024-03-30T13:11:59.080 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.332203+0000", 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:59.081 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:59.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 378, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.875324+0000" 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 378, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.875324+0000" 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 100, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 4, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:59.084 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:59.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:59.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:59.087 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.089 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:59.092 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:59.093 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:59.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0da006c530 msgr2=0x7f0da006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0da006c530 0x7f0da006e9f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00b9050 tx=0x7f0da8007e30 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 msgr2=0x7f0dc0157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 0x7f0dc0157880 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c430 tx=0x7f0db000c8f0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 shutdown_connections 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0da006c530 0x7f0da006e9f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc0167120 0x7f0dc00b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dc00ba090 0x7f0dc00cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 --2- 172.21.15.66:0/3771224329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc00b5dc0 0x7f0dc0157880 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 >> 172.21.15.66:0/3771224329 conn(0x7f0dc00678f0 msgr2=0x7f0dc00ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 shutdown_connections 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.068+0000 7f0dc4d52640 1 -- 172.21.15.66:0/3771224329 wait complete. 2024-03-30T13:11:59.096 DEBUG:tasks.cephfs.test_uninlining:getting inode info #10 of 23:1099511628321 2024-03-30T13:11:59.096 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628321 2024-03-30T13:11:59.096 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628321 2024-03-30T13:11:59.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 Processor -- start 2024-03-30T13:11:59.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 -- start start 2024-03-30T13:11:59.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ac00b5dc0 0x7f9ac00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00ba090 0x7f9ac00d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ac006c620 0x7f9ac006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac00b66e0 con 0x7f9ac006c620 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac00d2ff0 con 0x7f9ac00b5dc0 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.204+0000 7f9ac50d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac00d32d0 con 0x7f9ac00ba090 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ac006c620 0x7f9ac006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ac006c620 0x7f9ac006cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57372/0 (socket says 172.21.15.66:57372) 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 -- 172.21.15.66:0/2608515107 learned_addr learned my addr 172.21.15.66:0/2608515107 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abed9d640 1 --2- 172.21.15.66:0/2608515107 >> v2:172.21.15.117:3300/0 conn(0x7f9ac00b5dc0 0x7f9ac00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abe59c640 1 --2- 172.21.15.66:0/2608515107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00ba090 0x7f9ac00d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 -- 172.21.15.66:0/2608515107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00ba090 msgr2=0x7f9ac00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 --2- 172.21.15.66:0/2608515107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00ba090 0x7f9ac00d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 -- 172.21.15.66:0/2608515107 >> v2:172.21.15.117:3300/0 conn(0x7f9ac00b5dc0 msgr2=0x7f9ac00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 --2- 172.21.15.66:0/2608515107 >> v2:172.21.15.117:3300/0 conn(0x7f9ac00b5dc0 0x7f9ac00b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.205+0000 7f9abf59e640 1 -- 172.21.15.66:0/2608515107 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac00d35b0 con 0x7f9ac006c620 2024-03-30T13:11:59.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9abf59e640 1 --2- 172.21.15.66:0/2608515107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac006c620 0x7f9ac006cad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9ac006d770 tx=0x7f9ab802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5811470d4c5bd356 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9abdd9b640 1 -- 172.21.15.66:0/2608515107 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab802ca70 con 0x7f9ac006c620 2024-03-30T13:11:59.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9abdd9b640 1 -- 172.21.15.66:0/2608515107 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab802cc10 con 0x7f9ac006c620 2024-03-30T13:11:59.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9abdd9b640 1 -- 172.21.15.66:0/2608515107 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80343f0 con 0x7f9ac006c620 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 -- 172.21.15.66:0/2608515107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac006c620 msgr2=0x7f9ac006cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/2608515107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac006c620 0x7f9ac006cad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9ac006d770 tx=0x7f9ab802bed0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 -- 172.21.15.66:0/2608515107 shutdown_connections 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/2608515107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac006c620 0x7f9ac006cad0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/2608515107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00ba090 0x7f9ac00d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/2608515107 >> v2:172.21.15.117:3300/0 conn(0x7f9ac00b5dc0 0x7f9ac00b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 -- 172.21.15.66:0/2608515107 >> 172.21.15.66:0/2608515107 conn(0x7f9ac00678f0 msgr2=0x7f9ac00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 -- 172.21.15.66:0/2608515107 shutdown_connections 2024-03-30T13:11:59.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.206+0000 7f9ac50d9640 1 -- 172.21.15.66:0/2608515107 wait complete. 2024-03-30T13:11:59.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.208+0000 7f9ac50d9640 1 Processor -- start 2024-03-30T13:11:59.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.208+0000 7f9ac50d9640 1 -- start start 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9ac50d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac006c620 0x7f9ac01f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9ac50d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00b5dc0 0x7f9ac00cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9ac50d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 0x7f9ac015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9ac50d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac00d4090 con 0x7f9ac00ba090 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9ac50d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac00d3b70 con 0x7f9ac006c620 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9ac50d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac015e880 con 0x7f9ac00b5dc0 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 0x7f9ac015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 0x7f9ac015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57382/0 (socket says 172.21.15.66:57382) 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 -- 172.21.15.66:0/733826615 learned_addr learned my addr 172.21.15.66:0/733826615 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abed9d640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac006c620 0x7f9ac01f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abe59c640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00b5dc0 0x7f9ac00cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 -- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00b5dc0 msgr2=0x7f9ac00cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00b5dc0 0x7f9ac00cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 -- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac006c620 msgr2=0x7f9ac01f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac006c620 0x7f9ac01f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 -- 172.21.15.66:0/733826615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac01e90b0 con 0x7f9ac00ba090 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.209+0000 7f9abf59e640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 0x7f9ac015fc00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8036040 tx=0x7f9ab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.210+0000 7f9aa77fe640 1 -- 172.21.15.66:0/733826615 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8043020 con 0x7f9ac00ba090 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.210+0000 7f9aa77fe640 1 -- 172.21.15.66:0/733826615 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab803e450 con 0x7f9ac00ba090 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.210+0000 7f9aa77fe640 1 -- 172.21.15.66:0/733826615 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8009510 con 0x7f9ac00ba090 2024-03-30T13:11:59.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.210+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac01e9390 con 0x7f9ac00ba090 2024-03-30T13:11:59.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.210+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac01e9950 con 0x7f9ac00ba090 2024-03-30T13:11:59.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.211+0000 7f9aa77fe640 1 -- 172.21.15.66:0/733826615 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ab8046020 con 0x7f9ac00ba090 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.212+0000 7f9aa77fe640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9aac06c530 0x7f9aac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.212+0000 7f9aa77fe640 1 -- 172.21.15.66:0/733826615 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9ab80adf90 con 0x7f9ac00ba090 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.212+0000 7f9abed9d640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9aac06c530 0x7f9aac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.212+0000 7f9abed9d640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9aac06c530 0x7f9aac06e9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0006fd0 tx=0x7f9ab0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.213+0000 7f9ac7d9f740 1 Processor -- start 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.213+0000 7f9ac7d9f740 1 -- start start 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89d9950 0x5579e89dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89aca50 0x5579e89dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5579e8916590 con 0x5579e89d9570 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5579e88b4d30 con 0x5579e89aca50 2024-03-30T13:11:59.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5579e8912540 con 0x5579e89d9950 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57384/0 (socket says 172.21.15.66:57384) 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 -- 172.21.15.66:0/3221261974 learned_addr learned my addr 172.21.15.66:0/3221261974 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abf59e640 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89aca50 0x5579e89dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abe59c640 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89d9950 0x5579e89dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 -- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89d9950 msgr2=0x5579e89dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89d9950 0x5579e89dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 -- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89aca50 msgr2=0x5579e89dc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89aca50 0x5579e89dc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 -- 172.21.15.66:0/3221261974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e86f6920 con 0x5579e89d9570 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abf59e640 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89aca50 0x5579e89dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9abed9d640 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89ac510 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b970 tx=0x7f9ab0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e100f34886cf539 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9aa4ff9640 1 -- 172.21.15.66:0/3221261974 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab001c050 con 0x5579e89d9570 2024-03-30T13:11:59.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9aa4ff9640 1 -- 172.21.15.66:0/3221261974 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0026070 con 0x5579e89d9570 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.214+0000 7f9aa4ff9640 1 -- 172.21.15.66:0/3221261974 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0016e10 con 0x5579e89d9570 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3221261974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 msgr2=0x5579e89ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89ac510 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b970 tx=0x7f9ab0004610 comp rx=0 tx=0).stop 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3221261974 shutdown_connections 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89aca50 0x5579e89dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89d9950 0x5579e89dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3221261974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89ac510 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3221261974 >> 172.21.15.66:0/3221261974 conn(0x5579e89ae780 msgr2=0x5579e89f7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3221261974 shutdown_connections 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.215+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3221261974 wait complete. 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.216+0000 7f9ac7d9f740 1 -- start start 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.216+0000 7f9ac7d9f740 1 Processor -- start 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.226 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e89e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5579e89b0e20 con 0x5579e89d9570 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5579e88b29f0 con 0x5579e89d9950 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5579e89b3660 con 0x5579e89cd0f0 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 10 client.0 _ll_register_callbacks cb 0x5579e88b6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57398/0 (socket says 172.21.15.66:57398) 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 -- 172.21.15.66:0/3016782709 learned_addr learned my addr 172.21.15.66:0/3016782709 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abe59c640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e89e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abf59e640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 -- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 msgr2=0x5579e89e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89e2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 -- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 msgr2=0x5579e89e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e89e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 -- 172.21.15.66:0/3016782709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e85ea210 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9abed9d640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89cc950 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b910 tx=0x7f9ab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d66b98870bd75d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9a9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9a9dffb640 1 -- 172.21.15.66:0/3016782709 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0021ac0 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9a9dffb640 1 -- 172.21.15.66:0/3016782709 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9ab0036c70 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.217+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab0026070 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9a9dffb640 1 -- 172.21.15.66:0/3016782709 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9ab001c020 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 10 client.5853 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5579e87570b0 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9a9dffb640 1 -- 172.21.15.66:0/3016782709 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0026070 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9a9dffb640 1 -- 172.21.15.66:0/3016782709 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f9ab0021c60 con 0x5579e89d9570 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 10 client.5853 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 10 client.5853 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 20 client.5853 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e89de390 0x5579e89de810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 4 client.5853 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e8923ad0 con 0x5579e89de390 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.218+0000 7f9abe59c640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e89de390 0x5579e89de810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.219+0000 7f9abe59c640 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e89de390 0x5579e89de810 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506be26bf40e104f server_cookie=4d6cd0799023c5a3 in_seq=0 out_seq=0 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.219+0000 7f9a9dffb640 10 client.5853 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.219+0000 7f9a9dffb640 1 -- 172.21.15.66:0/3016782709 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5579e8923ad0 con 0x5579e89de390 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.219+0000 7f9a9dffb640 10 client.5853 handle_command_reply: tid=0 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 client.5853 shutdown 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 msgr2=0x5579e89cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89cc950 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b910 tx=0x7f9ab0004730 comp rx=0 tx=0).stop 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 shutdown_connections 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e89de390 0x5579e89de810 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e89e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/3016782709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e89cc950 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9a9dffb640 0 client.5853 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 >> 172.21.15.66:0/3016782709 conn(0x5579e89aca50 msgr2=0x5579e89b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 shutdown_connections 2024-03-30T13:11:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/3016782709 wait complete. 2024-03-30T13:11:59.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.220+0000 7f9ac7d9f740 20 client.5853 trim_cache size 0 max 16384 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.234+0000 7f9ac7d9f740 1 Processor -- start 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.234+0000 7f9ac7d9f740 1 -- start start 2024-03-30T13:11:59.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e8a6d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9570 0x5579e8a09a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 0x5579e8a09f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5579e89df6e0 con 0x5579e89d9950 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0201230 con 0x5579e89d9570 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac0200230 con 0x5579e89cd0f0 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 0x5579e8a09f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 0x5579e8a09f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57402/0 (socket says 172.21.15.66:57402) 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 -- 172.21.15.66:0/1057142500 learned_addr learned my addr 172.21.15.66:0/1057142500 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abed9d640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e8a6d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abe59c640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9570 0x5579e8a09a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 -- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 msgr2=0x5579e8a6d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e8a6d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 -- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9570 msgr2=0x5579e8a09a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9570 0x5579e8a09a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abf59e640 1 -- 172.21.15.66:0/1057142500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e85c32d0 con 0x5579e89d9950 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abed9d640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e8a6d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.235+0000 7f9abe59c640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9570 0x5579e8a09a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9abf59e640 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 0x5579e8a09f80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80073c0 tx=0x7f9ab803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a06e405accf008 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9a9ffff640 1 -- 172.21.15.66:0/1057142500 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80800b0 con 0x5579e89d9950 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9a9ffff640 1 -- 172.21.15.66:0/1057142500 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab80848e0 con 0x5579e89d9950 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9a9ffff640 1 -- 172.21.15.66:0/1057142500 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8078420 con 0x5579e89d9950 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/1057142500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 msgr2=0x5579e8a09f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 0x5579e8a09f80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80073c0 tx=0x7f9ab803c000 comp rx=0 tx=0).stop 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/1057142500 shutdown_connections 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9950 0x5579e8a09f80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9570 0x5579e8a09a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/1057142500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e8a6d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/1057142500 >> 172.21.15.66:0/1057142500 conn(0x5579e89aca50 msgr2=0x5579e89cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/1057142500 shutdown_connections 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.236+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/1057142500 wait complete. 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.237+0000 7f9ac7d9f740 1 -- start start 2024-03-30T13:11:59.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.237+0000 7f9ac7d9f740 1 Processor -- start 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e8a09e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e8a569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac015c8c0 con 0x5579e89d9570 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac015c270 con 0x5579e89d9950 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5579e8a1a690 con 0x5579e89cd0f0 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 10 client.0 _ll_register_callbacks cb 0x7f9ab000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e8a09e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e8a09e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57416/0 (socket says 172.21.15.66:57416) 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 -- 172.21.15.66:0/2162310673 learned_addr learned my addr 172.21.15.66:0/2162310673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abf59e640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e8a569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abed9d640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 -- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 msgr2=0x5579e89b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 -- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 msgr2=0x5579e8a569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e8a569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 -- 172.21.15.66:0/2162310673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e8a1a1c0 con 0x5579e89d9570 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9abe59c640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e8a09e40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00ba4b0 tx=0x7f9ab4007fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4a79f57ff352e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9a9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9a9dffb640 1 -- 172.21.15.66:0/2162310673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4005990 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9a9dffb640 1 -- 172.21.15.66:0/2162310673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9ab4005b30 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.238+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab80848e0 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9a9dffb640 1 -- 172.21.15.66:0/2162310673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f9ab402d050 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 10 client.5859 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5579e8a1a3d0 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9a9dffb640 1 -- 172.21.15.66:0/2162310673 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab80848e0 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9a9dffb640 1 -- 172.21.15.66:0/2162310673 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f9ab4030020 con 0x5579e89d9570 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 10 client.5859 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 10 client.5859 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 20 client.5859 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e8a56f10 0x5579e89e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 4 client.5859 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628321}] 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628321}) -- 0x5579e89b2130 con 0x5579e8a56f10 2024-03-30T13:11:59.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9abed9d640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e8a56f10 0x5579e89e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9a9dffb640 10 client.5859 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.239+0000 7f9abed9d640 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e8a56f10 0x5579e89e0de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8054133e9e28dabe server_cookie=5018e9296684fb41 in_seq=0 out_seq=0 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9a9dffb640 1 -- 172.21.15.66:0/2162310673 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4331 (crc 0 0 0) 0x5579e89b2130 con 0x5579e8a56f10 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9a9dffb640 10 client.5859 handle_command_reply: tid=0 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 client.5859 shutdown 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 msgr2=0x5579e8a09e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e8a09e40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00ba4b0 tx=0x7f9ab4007fd0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 shutdown_connections 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5579e8a56f10 0x5579e89e0de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579e89d9950 0x5579e8a569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5579e89d9570 0x5579e8a09e40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 --2- 172.21.15.66:0/2162310673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5579e89cd0f0 0x5579e89b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 >> 172.21.15.66:0/2162310673 conn(0x5579e89aca50 msgr2=0x5579e89dbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.240+0000 7f9a9dffb640 0 client.5859 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 shutdown_connections 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac7d9f740 1 -- 172.21.15.66:0/2162310673 wait complete. 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac7d9f740 20 client.5859 trim_cache size 0 max 16384 2024-03-30T13:11:59.252 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/symlink.c", 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628321, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.879128+0000", 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.345203+0000", 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:59.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 916, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:59.254 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.345203+0000", 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 916, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.256 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.879128+0000" 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 916, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.879128+0000" 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 102, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 28, 2024-03-30T13:11:59.257 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:59.258 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:59.259 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:59.260 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.262 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.263 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:59.264 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:59.265 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9aac06c530 msgr2=0x7f9aac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9aac06c530 0x7f9aac06e9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0006fd0 tx=0x7f9ab0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 msgr2=0x7f9ac015fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 0x7f9ac015fc00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8036040 tx=0x7f9ab8004a40 comp rx=0 tx=0).stop 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 shutdown_connections 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9aac06c530 0x7f9aac06e9f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ac00ba090 0x7f9ac015fc00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac00b5dc0 0x7f9ac00cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 --2- 172.21.15.66:0/733826615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ac006c620 0x7f9ac01f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 >> 172.21.15.66:0/733826615 conn(0x7f9ac00678f0 msgr2=0x7f9ac00cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 shutdown_connections 2024-03-30T13:11:59.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.241+0000 7f9ac50d9640 1 -- 172.21.15.66:0/733826615 wait complete. 2024-03-30T13:11:59.268 DEBUG:tasks.cephfs.test_uninlining:getting inode info #11 of 23:1099511628311 2024-03-30T13:11:59.268 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628311 2024-03-30T13:11:59.268 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628311 2024-03-30T13:11:59.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.374+0000 7f7a48138640 1 Processor -- start 2024-03-30T13:11:59.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.374+0000 7f7a48138640 1 -- start start 2024-03-30T13:11:59.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a48138640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7a400b5dc0 0x7f7a400b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a48138640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7a400ba090 0x7f7a400d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a48138640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a400b66e0 con 0x7f7a4006c620 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a48138640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7a400d2ff0 con 0x7f7a400b5dc0 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a48138640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7a400d32d0 con 0x7f7a400ba090 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a45ed4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7a400b5dc0 0x7f7a400b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a466d5640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7a4006c620 0x7f7a4006ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37516/0 (socket says 172.21.15.66:37516) 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a466d5640 1 -- 172.21.15.66:0/1433833560 learned_addr learned my addr 172.21.15.66:0/1433833560 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.375+0000 7f7a456d3640 1 --2- 172.21.15.66:0/1433833560 >> v2:172.21.15.117:3300/0 conn(0x7f7a400ba090 0x7f7a400d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.376+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1433833560 >> v2:172.21.15.117:3300/0 conn(0x7f7a400ba090 msgr2=0x7f7a400d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.376+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1433833560 >> v2:172.21.15.117:3300/0 conn(0x7f7a400ba090 0x7f7a400d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.376+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1433833560 >> v1:172.21.15.66:6789/0 conn(0x7f7a4006c620 legacy=0x7f7a4006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.376+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1433833560 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a400d35b0 con 0x7f7a400b5dc0 2024-03-30T13:11:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.376+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1433833560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400b5dc0 0x7f7a400b61a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7a400b7240 tx=0x7f7a3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5429fd19d0dc1e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a44ed2640 1 -- 172.21.15.66:0/1433833560 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30035020 con 0x7f7a400b5dc0 2024-03-30T13:11:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a44ed2640 1 -- 172.21.15.66:0/1433833560 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30031830 con 0x7f7a400b5dc0 2024-03-30T13:11:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a44ed2640 1 -- 172.21.15.66:0/1433833560 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30031c00 con 0x7f7a400b5dc0 2024-03-30T13:11:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 -- 172.21.15.66:0/1433833560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400b5dc0 msgr2=0x7f7a400b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 --2- 172.21.15.66:0/1433833560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400b5dc0 0x7f7a400b61a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7a400b7240 tx=0x7f7a3002b8d0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 -- 172.21.15.66:0/1433833560 shutdown_connections 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 --2- 172.21.15.66:0/1433833560 >> v2:172.21.15.117:3300/0 conn(0x7f7a400ba090 0x7f7a400d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 --2- 172.21.15.66:0/1433833560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400b5dc0 0x7f7a400b61a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 -- 172.21.15.66:0/1433833560 >> 172.21.15.66:0/1433833560 conn(0x7f7a400678f0 msgr2=0x7f7a400d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 -- 172.21.15.66:0/1433833560 shutdown_connections 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.377+0000 7f7a48138640 1 -- 172.21.15.66:0/1433833560 wait complete. 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.378+0000 7f7a48138640 1 Processor -- start 2024-03-30T13:11:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.378+0000 7f7a48138640 1 -- start start 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a48138640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a4006c620 0x7f7a401549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a48138640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a400b5dc0 0x7f7a401470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a48138640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 0x7f7a401475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a48138640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400d4090 con 0x7f7a400ba090 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a48138640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400d3b70 con 0x7f7a400b5dc0 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a48138640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7a400b83e0 con 0x7f7a4006c620 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a466d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 0x7f7a401475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a466d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 0x7f7a401475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57426/0 (socket says 172.21.15.66:57426) 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a466d5640 1 -- 172.21.15.66:0/1741685562 learned_addr learned my addr 172.21.15.66:0/1741685562 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a456d3640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a400b5dc0 0x7f7a401470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.379+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a4006c620 0x7f7a401549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a466d5640 1 -- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a4006c620 msgr2=0x7f7a401549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a4006c620 0x7f7a401549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a466d5640 1 -- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a400b5dc0 msgr2=0x7f7a401470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a400b5dc0 0x7f7a401470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a466d5640 1 -- 172.21.15.66:0/1741685562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a40157540 con 0x7f7a400ba090 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a4006c620 0x7f7a401549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 0x7f7a401475e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800cbf0 tx=0x7f7a3800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a367fc640 1 -- 172.21.15.66:0/1741685562 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38017070 con 0x7f7a400ba090 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a367fc640 1 -- 172.21.15.66:0/1741685562 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3800ad40 con 0x7f7a400ba090 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a40157820 con 0x7f7a400ba090 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a367fc640 1 -- 172.21.15.66:0/1741685562 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38012600 con 0x7f7a400ba090 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.380+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a400cc8e0 con 0x7f7a400ba090 2024-03-30T13:11:59.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.382+0000 7f7a367fc640 1 -- 172.21.15.66:0/1741685562 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a3801a070 con 0x7f7a400ba090 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.383+0000 7f7a367fc640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7a2406c530 0x7f7a2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.383+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7a2406c530 0x7f7a2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.383+0000 7f7a367fc640 1 -- 172.21.15.66:0/1741685562 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7a3808b510 con 0x7f7a400ba090 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.383+0000 7f7a4adf4740 1 Processor -- start 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 -- start start 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7a2406c530 0x7f7a2406e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a30036000 tx=0x7f7a30008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b900667510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 0x55b900697300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900667a50 0x55b900697840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9005d1590 con 0x55b900694950 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b90056fd30 con 0x55b900694570 2024-03-30T13:11:59.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b9005cd540 con 0x55b900667a50 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a456d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 0x55b900697300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a456d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 0x55b900697300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57430/0 (socket says 172.21.15.66:57430) 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a456d3640 1 -- 172.21.15.66:0/1637583987 learned_addr learned my addr 172.21.15.66:0/1637583987 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.384+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b900667510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900667a50 0x55b900697840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a456d3640 1 -- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900667a50 msgr2=0x55b900697840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a456d3640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900667a50 0x55b900697840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a456d3640 1 -- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 msgr2=0x55b900667510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a456d3640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b900667510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a456d3640 1 -- 172.21.15.66:0/1637583987 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9003b1920 con 0x55b900694950 2024-03-30T13:11:59.392 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b900667510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900667a50 0x55b900697840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a456d3640 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 0x55b900697300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0078f0 tx=0x7f7a3c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7ec2961a39af13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a237fe640 1 -- 172.21.15.66:0/1637583987 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c010aa0 con 0x55b900694950 2024-03-30T13:11:59.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a237fe640 1 -- 172.21.15.66:0/1637583987 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c010c40 con 0x55b900694950 2024-03-30T13:11:59.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.385+0000 7f7a237fe640 1 -- 172.21.15.66:0/1637583987 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004030 con 0x55b900694950 2024-03-30T13:11:59.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1637583987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 msgr2=0x55b900697300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 0x55b900697300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0078f0 tx=0x7f7a3c007db0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1637583987 shutdown_connections 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900667a50 0x55b900697840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694950 0x55b900697300 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1637583987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b900667510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1637583987 >> 172.21.15.66:0/1637583987 conn(0x55b900669780 msgr2=0x55b9006b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1637583987 shutdown_connections 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.386+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1637583987 wait complete. 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.387+0000 7f7a4adf4740 1 -- start start 2024-03-30T13:11:59.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.387+0000 7f7a4adf4740 1 Processor -- start 2024-03-30T13:11:59.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b9006c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694950 0x55b90069bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b90066be20 con 0x55b90069ad50 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b90056d9f0 con 0x55b900694570 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b90066e660 con 0x55b900694950 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a4adf4740 10 client.0 _ll_register_callbacks cb 0x55b900571c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57432/0 (socket says 172.21.15.66:57432) 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 -- 172.21.15.66:0/3928445201 learned_addr learned my addr 172.21.15.66:0/3928445201 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b9006c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a456d3640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694950 0x55b90069bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 -- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694950 msgr2=0x55b90069bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694950 0x55b90069bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 -- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 msgr2=0x55b9006c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b9006c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 -- 172.21.15.66:0/3928445201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9002a5210 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b9006c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a466d5640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b01c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a38061930 tx=0x7f7a3805d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b6a9d617b6f446 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.389+0000 7f7a20ff9640 1 -- 172.21.15.66:0/3928445201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38061bd0 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a20ff9640 1 -- 172.21.15.66:0/3928445201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7a3808b0d0 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3c010c40 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a20ff9640 1 -- 172.21.15.66:0/3928445201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7a3808b2d0 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 10 client.5874 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b9004120b0 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a20ff9640 1 -- 172.21.15.66:0/3928445201 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c010c40 con 0x55b90069ad50 2024-03-30T13:11:59.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a20ff9640 1 -- 172.21.15.66:0/3928445201 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7a3805d170 con 0x55b90069ad50 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 10 client.5874 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 10 client.5874 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 20 client.5874 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b900686810 0x55b900686c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 4 client.5874 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9005dead0 con 0x55b900686810 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b900686810 0x55b900686c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b900686810 0x55b900686c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c040224ee41b6be1 server_cookie=d55458cdd3d680dc in_seq=0 out_seq=0 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.390+0000 7f7a20ff9640 10 client.5874 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a20ff9640 1 -- 172.21.15.66:0/3928445201 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55b9005dead0 con 0x55b900686810 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a20ff9640 10 client.5874 handle_command_reply: tid=0 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 client.5874 shutdown 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 msgr2=0x55b9006b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b01c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7a38061930 tx=0x7f7a3805d630 comp rx=0 tx=0).stop 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 shutdown_connections 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b900686810 0x55b900686c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b01c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694950 0x55b90069bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a20ff9640 0 client.5874 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/3928445201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694570 0x55b9006c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 >> 172.21.15.66:0/3928445201 conn(0x55b900667a50 msgr2=0x55b9006b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 shutdown_connections 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 1 -- 172.21.15.66:0/3928445201 wait complete. 2024-03-30T13:11:59.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.391+0000 7f7a4adf4740 20 client.5874 trim_cache size 0 max 16384 2024-03-30T13:11:59.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.406+0000 7f7a4adf4740 1 Processor -- start 2024-03-30T13:11:59.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.406+0000 7f7a4adf4740 1 -- start start 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 0x55b9006c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9006b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b90069ad50 0x55b9006b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40201230 con 0x55b900694570 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40200270 con 0x55b900694950 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7a400cd760 con 0x55b90069ad50 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 0x55b9006c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 0x55b9006c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57446/0 (socket says 172.21.15.66:57446) 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1728113982 learned_addr learned my addr 172.21.15.66:0/1728113982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1728113982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b90069ad50 msgr2=0x55b9006b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b90069ad50 0x55b9006b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1728113982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 msgr2=0x55b9006b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9006b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 -- 172.21.15.66:0/1728113982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90027e2d0 con 0x55b900694570 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 0x55b9006c5470 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009890 tx=0x7f7a30030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9633f54c01b46dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a22ffd640 1 -- 172.21.15.66:0/1728113982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30057070 con 0x55b900694570 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a22ffd640 1 -- 172.21.15.66:0/1728113982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30046c00 con 0x55b900694570 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.407+0000 7f7a22ffd640 1 -- 172.21.15.66:0/1728113982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30035040 con 0x55b900694570 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1728113982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 msgr2=0x55b9006c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 0x55b9006c5470 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009890 tx=0x7f7a30030000 comp rx=0 tx=0).stop 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1728113982 shutdown_connections 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b90069ad50 0x55b9006b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9006b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1728113982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b900694570 0x55b9006c5470 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1728113982 >> 172.21.15.66:0/1728113982 conn(0x55b900667a50 msgr2=0x55b900687450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1728113982 shutdown_connections 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1728113982 wait complete. 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.408+0000 7f7a4adf4740 1 -- start start 2024-03-30T13:11:59.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.409+0000 7f7a4adf4740 1 Processor -- start 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694570 0x55b9006d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9007144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9006b4090 con 0x55b90069ad50 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400cddb0 con 0x55b900694950 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b90066da60 con 0x55b900694570 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a4adf4740 10 client.0 _ll_register_callbacks cb 0x7f7a3c007120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57452/0 (socket says 172.21.15.66:57452) 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 -- 172.21.15.66:0/1867483137 learned_addr learned my addr 172.21.15.66:0/1867483137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694570 0x55b9006d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a456d3640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9007144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 -- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694570 msgr2=0x55b9006d5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694570 0x55b9006d5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 -- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 msgr2=0x55b9007144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.410+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9007144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a466d5640 1 -- 172.21.15.66:0/1867483137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b900667230 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a466d5640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b34c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7a380592c0 tx=0x7f7a38001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904af61d919e9c94 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a20ff9640 1 -- 172.21.15.66:0/1867483137 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38075070 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a20ff9640 1 -- 172.21.15.66:0/1867483137 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7a38058a50 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a30046c00 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a20ff9640 1 -- 172.21.15.66:0/1867483137 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7a38080050 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 10 client.5880 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b900715d60 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a20ff9640 1 -- 172.21.15.66:0/1867483137 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30046c00 con 0x55b90069ad50 2024-03-30T13:11:59.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a20ff9640 1 -- 172.21.15.66:0/1867483137 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7a380619d0 con 0x55b90069ad50 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 10 client.5880 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 10 client.5880 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 20 client.5880 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9006b3a00 0x55b9006c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 4 client.5880 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628311}] 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628311}) -- 0x55b9005d8e20 con 0x55b9006b3a00 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.411+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9006b3a00 0x55b9006c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a45ed4640 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9006b3a00 0x55b9006c0a20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a4a224fe938e77 server_cookie=de5e9b7130e777e5 in_seq=0 out_seq=0 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a20ff9640 10 client.5880 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a20ff9640 1 -- 172.21.15.66:0/1867483137 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4329 (crc 0 0 0) 0x55b9005d8e20 con 0x55b9006b3a00 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a20ff9640 10 client.5880 handle_command_reply: tid=0 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 client.5880 shutdown 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 msgr2=0x55b9006b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b34c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7a380592c0 tx=0x7f7a38001250 comp rx=0 tx=0).stop 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 shutdown_connections 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9006b3a00 0x55b9006c0a20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b90069ad50 0x55b9006b34c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b900694950 0x55b9007144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a20ff9640 0 client.5880 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 --2- 172.21.15.66:0/1867483137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b900694570 0x55b9006d5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 >> 172.21.15.66:0/1867483137 conn(0x55b900667a50 msgr2=0x55b900662740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.412+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 shutdown_connections 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a4adf4740 1 -- 172.21.15.66:0/1867483137 wait complete. 2024-03-30T13:11:59.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a4adf4740 20 client.5880 trim_cache size 0 max 16384 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/acl.h", 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628311, 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.885066+0000", 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.334203+0000", 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:59.423 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.424 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1502, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.334203+0000", 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:59.425 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:59.426 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1502, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.885066+0000" 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:59.427 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1502, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.885066+0000" 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 106, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 8, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:59.428 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.429 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:59.430 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:59.431 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.433 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:59.434 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:59.435 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.436 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.437 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:59.437 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:59.437 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:59.437 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:59.437 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7a2406c530 msgr2=0x7f7a2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7a2406c530 0x7f7a2406e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a30036000 tx=0x7f7a30008030 comp rx=0 tx=0).stop 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 msgr2=0x7f7a401475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 0x7f7a401475e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800cbf0 tx=0x7f7a3800a430 comp rx=0 tx=0).stop 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 shutdown_connections 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7a2406c530 0x7f7a2406e9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a400ba090 0x7f7a401475e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a400b5dc0 0x7f7a401470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 --2- 172.21.15.66:0/1741685562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a4006c620 0x7f7a401549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 >> 172.21.15.66:0/1741685562 conn(0x7f7a400678f0 msgr2=0x7f7a400b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 shutdown_connections 2024-03-30T13:11:59.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.413+0000 7f7a48138640 1 -- 172.21.15.66:0/1741685562 wait complete. 2024-03-30T13:11:59.440 DEBUG:tasks.cephfs.test_uninlining:getting inode info #12 of 23:1099511628325 2024-03-30T13:11:59.440 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628325 2024-03-30T13:11:59.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628325 2024-03-30T13:11:59.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.547+0000 7efd2aca0640 1 Processor -- start 2024-03-30T13:11:59.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.547+0000 7efd2aca0640 1 -- start start 2024-03-30T13:11:59.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd2aca0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd240b5dc0 0x7efd240b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd2aca0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efd240ba090 0x7efd240d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd2aca0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd240b66e0 con 0x7efd2406c620 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd2aca0640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efd240d2ff0 con 0x7efd240ba090 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd2aca0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efd240d32d0 con 0x7efd240b5dc0 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efd240ba090 0x7efd240d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efd240ba090 0x7efd240d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57464/0 (socket says 172.21.15.66:57464) 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 -- 172.21.15.66:0/3536287133 learned_addr learned my addr 172.21.15.66:0/3536287133 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd28a3c640 1 --2- 172.21.15.66:0/3536287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd240b5dc0 0x7efd240b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 -- 172.21.15.66:0/3536287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd240b5dc0 msgr2=0x7efd240b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 --2- 172.21.15.66:0/3536287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd240b5dc0 0x7efd240b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 -- 172.21.15.66:0/3536287133 >> v1:172.21.15.117:6789/0 conn(0x7efd2406c620 legacy=0x7efd2406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.548+0000 7efd23fff640 1 -- 172.21.15.66:0/3536287133 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd240d35b0 con 0x7efd240ba090 2024-03-30T13:11:59.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.549+0000 7efd23fff640 1 --2- 172.21.15.66:0/3536287133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240ba090 0x7efd240d2680 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efd10009870 tx=0x7efd1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940b05caef28f114 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.549+0000 7efd237fe640 1 -- 172.21.15.66:0/3536287133 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10034070 con 0x7efd240ba090 2024-03-30T13:11:59.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.549+0000 7efd237fe640 1 -- 172.21.15.66:0/3536287133 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd10030840 con 0x7efd240ba090 2024-03-30T13:11:59.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.549+0000 7efd237fe640 1 -- 172.21.15.66:0/3536287133 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10030c10 con 0x7efd240ba090 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 -- 172.21.15.66:0/3536287133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240ba090 msgr2=0x7efd240d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3536287133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240ba090 0x7efd240d2680 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efd10009870 tx=0x7efd1002b980 comp rx=0 tx=0).stop 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 -- 172.21.15.66:0/3536287133 shutdown_connections 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3536287133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240ba090 0x7efd240d2680 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3536287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd240b5dc0 0x7efd240b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 -- 172.21.15.66:0/3536287133 >> 172.21.15.66:0/3536287133 conn(0x7efd240678f0 msgr2=0x7efd240d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 -- 172.21.15.66:0/3536287133 shutdown_connections 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.550+0000 7efd2aca0640 1 -- 172.21.15.66:0/3536287133 wait complete. 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.551+0000 7efd2aca0640 1 Processor -- start 2024-03-30T13:11:59.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.551+0000 7efd2aca0640 1 -- start start 2024-03-30T13:11:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2aca0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2406c620 0x7efd2414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2aca0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 0x7efd240c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2aca0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd240ba090 0x7efd241e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2aca0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd240d4090 con 0x7efd240b5dc0 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2aca0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd240d3b70 con 0x7efd240ba090 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2aca0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efd240b83e0 con 0x7efd2406c620 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 0x7efd240c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 0x7efd240c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57478/0 (socket says 172.21.15.66:57478) 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 -- 172.21.15.66:0/3035556797 learned_addr learned my addr 172.21.15.66:0/3035556797 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd28a3c640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2406c620 0x7efd2414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd2923d640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd240ba090 0x7efd241e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 -- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2406c620 msgr2=0x7efd2414b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2406c620 0x7efd2414b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 -- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd240ba090 msgr2=0x7efd241e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd240ba090 0x7efd241e7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 -- 172.21.15.66:0/3035556797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd24147fa0 con 0x7efd240b5dc0 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.552+0000 7efd23fff640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 0x7efd240c9f50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efd10000c00 tx=0x7efd100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.553+0000 7efd217fa640 1 -- 172.21.15.66:0/3035556797 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10034040 con 0x7efd240b5dc0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.553+0000 7efd217fa640 1 -- 172.21.15.66:0/3035556797 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1003ada0 con 0x7efd240b5dc0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.553+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd24148280 con 0x7efd240b5dc0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.553+0000 7efd217fa640 1 -- 172.21.15.66:0/3035556797 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10044630 con 0x7efd240b5dc0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.553+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd24148840 con 0x7efd240b5dc0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.554+0000 7efd217fa640 1 -- 172.21.15.66:0/3035556797 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd1004b060 con 0x7efd240b5dc0 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.555+0000 7efd217fa640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd0806c560 0x7efd0806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.555+0000 7efd217fa640 1 -- 172.21.15.66:0/3035556797 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7efd100ad790 con 0x7efd240b5dc0 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.555+0000 7efd28a3c640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd0806c560 0x7efd0806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.555+0000 7efd28a3c640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd0806c560 0x7efd0806ea20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd18006fd0 tx=0x7efd18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.556+0000 7efd2d885740 1 Processor -- start 2024-03-30T13:11:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.556+0000 7efd2d885740 1 -- start start 2024-03-30T13:11:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd2d885740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474d6f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474d9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d6fa50 0x562474d9f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd2d885740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562474cd9590 con 0x562474d9c570 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562474c77d30 con 0x562474d6fa50 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x562474cd5540 con 0x562474d9c950 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd28a3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474d6f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd28a3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474d6f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57482/0 (socket says 172.21.15.66:57482) 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.557+0000 7efd28a3c640 1 -- 172.21.15.66:0/2369485023 learned_addr learned my addr 172.21.15.66:0/2369485023 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd28a3c640 1 -- 172.21.15.66:0/2369485023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 msgr2=0x562474d9f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd28a3c640 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474d9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd28a3c640 1 -- 172.21.15.66:0/2369485023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d6fa50 msgr2=0x562474d9f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd28a3c640 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d6fa50 0x562474d9f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd28a3c640 1 -- 172.21.15.66:0/2369485023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562474ab9920 con 0x562474d9c570 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd28a3c640 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474d6f510 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efd18004220 tx=0x7efd18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575a345468dc832 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd0e7fc640 1 -- 172.21.15.66:0/2369485023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18027070 con 0x562474d9c570 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd0e7fc640 1 -- 172.21.15.66:0/2369485023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd180038e0 con 0x562474d9c570 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.558+0000 7efd0e7fc640 1 -- 172.21.15.66:0/2369485023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18003c90 con 0x562474d9c570 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 -- 172.21.15.66:0/2369485023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 msgr2=0x562474d6f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474d6f510 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efd18004220 tx=0x7efd18013000 comp rx=0 tx=0).stop 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 -- 172.21.15.66:0/2369485023 shutdown_connections 2024-03-30T13:11:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d6fa50 0x562474d9f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474d9f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 --2- 172.21.15.66:0/2369485023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474d6f510 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 -- 172.21.15.66:0/2369485023 >> 172.21.15.66:0/2369485023 conn(0x562474d71780 msgr2=0x562474dba300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 -- 172.21.15.66:0/2369485023 shutdown_connections 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 -- 172.21.15.66:0/2369485023 wait complete. 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.559+0000 7efd2d885740 1 -- start start 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.560+0000 7efd2d885740 1 Processor -- start 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.560+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.560+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9c950 0x562474dcb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.560+0000 7efd2d885740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 0x562474d9ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.560+0000 7efd2d885740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562474d73e20 con 0x562474d9e870 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562474c759f0 con 0x562474d9c950 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x562474d76660 con 0x562474d9c570 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2d885740 10 client.0 _ll_register_callbacks cb 0x562474c79c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 0x562474d9ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 0x562474d9ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57494/0 (socket says 172.21.15.66:57494) 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 -- 172.21.15.66:0/4189705091 learned_addr learned my addr 172.21.15.66:0/4189705091 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd23fff640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9c950 0x562474dcb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd28a3c640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 -- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 msgr2=0x562474dba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 -- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9c950 msgr2=0x562474dcb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9c950 0x562474dcb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 -- 172.21.15.66:0/4189705091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624749ad210 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd23fff640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9c950 0x562474dcb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2923d640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 0x562474d9ecf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00f570 tx=0x7efd1c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5eb6ac46db32847 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd037fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd037fe640 1 -- 172.21.15.66:0/4189705091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c003b80 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd037fe640 1 -- 172.21.15.66:0/4189705091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7efd1c003d20 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd180038e0 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.561+0000 7efd037fe640 1 -- 172.21.15.66:0/4189705091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7efd1c020050 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 10 client.5892 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562474b1a0b0 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd037fe640 1 -- 172.21.15.66:0/4189705091 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd180038e0 con 0x562474d9e870 2024-03-30T13:11:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd037fe640 1 -- 172.21.15.66:0/4189705091 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7efd1c023020 con 0x562474d9e870 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 10 client.5892 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 10 client.5892 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 20 client.5892 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474db1af0 0x562474db1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 4 client.5892 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562474ce6ad0 con 0x562474db1af0 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd28a3c640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474db1af0 0x562474db1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd28a3c640 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474db1af0 0x562474db1ed0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b538a7e6e04e84da server_cookie=860f19687a62340 in_seq=0 out_seq=0 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.562+0000 7efd037fe640 10 client.5892 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd037fe640 1 -- 172.21.15.66:0/4189705091 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x562474ce6ad0 con 0x562474db1af0 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd037fe640 10 client.5892 handle_command_reply: tid=0 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 client.5892 shutdown 2024-03-30T13:11:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 msgr2=0x562474d9ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 0x562474d9ecf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00f570 tx=0x7efd1c00fa30 comp rx=0 tx=0).stop 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 shutdown_connections 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474db1af0 0x562474db1ed0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9e870 0x562474d9ecf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9c950 0x562474dcb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd037fe640 0 client.5892 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 --2- 172.21.15.66:0/4189705091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 >> 172.21.15.66:0/4189705091 conn(0x562474d6fa50 msgr2=0x562474d73630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 shutdown_connections 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 1 -- 172.21.15.66:0/4189705091 wait complete. 2024-03-30T13:11:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.563+0000 7efd2d885740 20 client.5892 trim_cache size 0 max 16384 2024-03-30T13:11:59.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 Processor -- start 2024-03-30T13:11:59.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 -- start start 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dc9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 0x562474d9dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d9e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd24201130 con 0x562474d9c950 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd24200130 con 0x562474d9e870 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.578+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efd241f8160 con 0x562474d9c570 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 0x562474d9dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 0x562474d9dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57500/0 (socket says 172.21.15.66:57500) 2024-03-30T13:11:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 -- 172.21.15.66:0/3243911933 learned_addr learned my addr 172.21.15.66:0/3243911933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd2923d640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d9e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd28a3c640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dc9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 -- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 msgr2=0x562474dc9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dc9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 -- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 msgr2=0x562474d9e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d9e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 -- 172.21.15.66:0/3243911933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624749862d0 con 0x562474d9c950 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd2923d640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d9e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd28a3c640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dc9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd23fff640 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 0x562474d9dd60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd10077a10 tx=0x7efd100094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5478f830d1fa2e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd0dffb640 1 -- 172.21.15.66:0/3243911933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1007f8b0 con 0x562474d9c950 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd0dffb640 1 -- 172.21.15.66:0/3243911933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd100840e0 con 0x562474d9c950 2024-03-30T13:11:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.579+0000 7efd0dffb640 1 -- 172.21.15.66:0/3243911933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10077c20 con 0x562474d9c950 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 -- 172.21.15.66:0/3243911933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 msgr2=0x562474d9dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 0x562474d9dd60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd10077a10 tx=0x7efd100094d0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 -- 172.21.15.66:0/3243911933 shutdown_connections 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d9e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c950 0x562474d9dd60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 --2- 172.21.15.66:0/3243911933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c570 0x562474dc9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 -- 172.21.15.66:0/3243911933 >> 172.21.15.66:0/3243911933 conn(0x562474d6fa50 msgr2=0x562474db2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 -- 172.21.15.66:0/3243911933 shutdown_connections 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 -- 172.21.15.66:0/3243911933 wait complete. 2024-03-30T13:11:59.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.580+0000 7efd2d885740 1 -- start start 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 Processor -- start 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474dafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474dc8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d6e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562474ce40f0 con 0x562474d9c570 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd241ffb80 con 0x562474d9e870 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x562474dbb040 con 0x562474d9c950 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd2d885740 10 client.0 _ll_register_callbacks cb 0x7efd1c001920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd28a3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474dafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd28a3c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474dafed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57510/0 (socket says 172.21.15.66:57510) 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.581+0000 7efd28a3c640 1 -- 172.21.15.66:0/2613687068 learned_addr learned my addr 172.21.15.66:0/2613687068 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd23fff640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474dc8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2923d640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d6e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd28a3c640 1 -- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 msgr2=0x562474dc8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd28a3c640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474dc8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd28a3c640 1 -- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 msgr2=0x562474d6e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd28a3c640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d6e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd28a3c640 1 -- 172.21.15.66:0/2613687068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562474da0d90 con 0x562474d9c570 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd23fff640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474dc8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2923d640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d6e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd28a3c640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474dafed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd18007860 tx=0x7efd18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5b3477923a17ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd037fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd037fe640 1 -- 172.21.15.66:0/2613687068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18027040 con 0x562474d9c570 2024-03-30T13:11:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd037fe640 1 -- 172.21.15.66:0/2613687068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7efd18033220 con 0x562474d9c570 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd100840e0 con 0x562474d9c570 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd037fe640 1 -- 172.21.15.66:0/2613687068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7efd18041050 con 0x562474d9c570 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2d885740 10 client.5898 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x562474d74b00 con 0x562474d9c570 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd037fe640 1 -- 172.21.15.66:0/2613687068 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd100840e0 con 0x562474d9c570 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd037fe640 1 -- 172.21.15.66:0/2613687068 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7efd18044020 con 0x562474d9c570 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2d885740 10 client.5898 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2d885740 10 client.5898 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.582+0000 7efd2d885740 20 client.5898 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd2d885740 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474dba240 0x562474d6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd2d885740 4 client.5898 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628325}] 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628325}) -- 0x562474ce0e20 con 0x562474dba240 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd23fff640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474dba240 0x562474d6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd037fe640 10 client.5898 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd23fff640 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474dba240 0x562474d6ebc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad9598613a3a6e48 server_cookie=1a2581dd404c3f00 in_seq=0 out_seq=0 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd037fe640 1 -- 172.21.15.66:0/2613687068 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4340 (crc 0 0 0) 0x562474ce0e20 con 0x562474dba240 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd037fe640 10 client.5898 handle_command_reply: tid=0 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd2d885740 1 client.5898 shutdown 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 msgr2=0x562474dafed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.583+0000 7efd2d885740 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474dafed0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd18007860 tx=0x7efd18004730 comp rx=0 tx=0).stop 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 shutdown_connections 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x562474dba240 0x562474d6ebc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562474d9e870 0x562474d6e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562474d9c950 0x562474dc8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 --2- 172.21.15.66:0/2613687068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562474d9c570 0x562474dafed0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd037fe640 0 client.5898 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 >> 172.21.15.66:0/2613687068 conn(0x562474d6fa50 msgr2=0x562474da1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 shutdown_connections 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 1 -- 172.21.15.66:0/2613687068 wait complete. 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2d885740 20 client.5898 trim_cache size 0 max 16384 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr_trusted.c", 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628325, 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.891352+0000", 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.348203+0000", 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:59.594 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:59.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1094, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.348203+0000", 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:59.596 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:59.597 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1094, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.891352+0000" 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:59.598 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1094, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.891352+0000" 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 110, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 36, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:59.599 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:59.600 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:59.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:59.604 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.605 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:59.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:59.607 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd0806c560 msgr2=0x7efd0806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd0806c560 0x7efd0806ea20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd18006fd0 tx=0x7efd18008040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 msgr2=0x7efd240c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 0x7efd240c9f50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efd10000c00 tx=0x7efd100046c0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 shutdown_connections 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efd0806c560 0x7efd0806ea20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd240ba090 0x7efd241e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd240b5dc0 0x7efd240c9f50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 --2- 172.21.15.66:0/3035556797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd2406c620 0x7efd2414b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.584+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 >> 172.21.15.66:0/3035556797 conn(0x7efd240678f0 msgr2=0x7efd2415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.585+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 shutdown_connections 2024-03-30T13:11:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.585+0000 7efd2aca0640 1 -- 172.21.15.66:0/3035556797 wait complete. 2024-03-30T13:11:59.610 DEBUG:tasks.cephfs.test_uninlining:getting inode info #13 of 23:1099511628323 2024-03-30T13:11:59.610 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628323 2024-03-30T13:11:59.610 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628323 2024-03-30T13:11:59.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.718+0000 7f0b675cd640 1 Processor -- start 2024-03-30T13:11:59.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.718+0000 7f0b675cd640 1 -- start start 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b675cd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b600b5dc0 0x7f0b600b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b675cd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b600ba090 0x7f0b600d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b675cd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6006c620 0x7f0b6006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b675cd640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0b600b66e0 con 0x7f0b600b5dc0 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b675cd640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f0b600d2ff0 con 0x7f0b600ba090 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b675cd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0b600d32d0 con 0x7f0b6006c620 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b600b5dc0 0x7f0b600b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0b600b5dc0 0x7f0b600b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57512/0 (socket says 172.21.15.66:57512) 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 -- 172.21.15.66:0/3414888077 learned_addr learned my addr 172.21.15.66:0/3414888077 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b64b68640 1 --2- 172.21.15.66:0/3414888077 >> v2:172.21.15.117:3300/0 conn(0x7f0b600ba090 0x7f0b600d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/3414888077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6006c620 0x7f0b6006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 -- 172.21.15.66:0/3414888077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6006c620 msgr2=0x7f0b6006ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 --2- 172.21.15.66:0/3414888077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6006c620 0x7f0b6006ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 -- 172.21.15.66:0/3414888077 >> v2:172.21.15.117:3300/0 conn(0x7f0b600ba090 msgr2=0x7f0b600d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 --2- 172.21.15.66:0/3414888077 >> v2:172.21.15.117:3300/0 conn(0x7f0b600ba090 0x7f0b600d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.719+0000 7f0b65369640 1 -- 172.21.15.66:0/3414888077 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b600d35b0 con 0x7f0b600b5dc0 2024-03-30T13:11:59.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.720+0000 7f0b65369640 1 --2- 172.21.15.66:0/3414888077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b600b61a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0b600b7240 tx=0x7f0b5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24b030a0c4056ad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b57fff640 1 -- 172.21.15.66:0/3414888077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c035020 con 0x7f0b600b5dc0 2024-03-30T13:11:59.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b57fff640 1 -- 172.21.15.66:0/3414888077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c031870 con 0x7f0b600b5dc0 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b57fff640 1 -- 172.21.15.66:0/3414888077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c031c40 con 0x7f0b600b5dc0 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 -- 172.21.15.66:0/3414888077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 msgr2=0x7f0b600b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 --2- 172.21.15.66:0/3414888077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b600b61a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0b600b7240 tx=0x7f0b5c02b910 comp rx=0 tx=0).stop 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 -- 172.21.15.66:0/3414888077 shutdown_connections 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 --2- 172.21.15.66:0/3414888077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6006c620 0x7f0b6006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 --2- 172.21.15.66:0/3414888077 >> v2:172.21.15.117:3300/0 conn(0x7f0b600ba090 0x7f0b600d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 --2- 172.21.15.66:0/3414888077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b600b61a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 -- 172.21.15.66:0/3414888077 >> 172.21.15.66:0/3414888077 conn(0x7f0b600678f0 msgr2=0x7f0b600d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 -- 172.21.15.66:0/3414888077 shutdown_connections 2024-03-30T13:11:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.721+0000 7f0b675cd640 1 -- 172.21.15.66:0/3414888077 wait complete. 2024-03-30T13:11:59.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.722+0000 7f0b675cd640 1 Processor -- start 2024-03-30T13:11:59.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 -- start start 2024-03-30T13:11:59.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6006c620 0x7f0b601e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b601f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600ba090 0x7f0b60160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b600d4090 con 0x7f0b600b5dc0 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b600d3b70 con 0x7f0b6006c620 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b675cd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6015e880 con 0x7f0b600ba090 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b64b68640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b601f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b64b68640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b601f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57526/0 (socket says 172.21.15.66:57526) 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b64b68640 1 -- 172.21.15.66:0/890661267 learned_addr learned my addr 172.21.15.66:0/890661267 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.723+0000 7f0b65369640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6006c620 0x7f0b601e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600ba090 0x7f0b60160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b64b68640 1 -- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600ba090 msgr2=0x7f0b60160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b64b68640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600ba090 0x7f0b60160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b64b68640 1 -- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6006c620 msgr2=0x7f0b601e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b64b68640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6006c620 0x7f0b601e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b64b68640 1 -- 172.21.15.66:0/890661267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6015fd30 con 0x7f0b600b5dc0 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600ba090 0x7f0b60160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b64b68640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b601f7bd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000d5f0 tx=0x7f0b5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b55ffb640 1 -- 172.21.15.66:0/890661267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b500125f0 con 0x7f0b600b5dc0 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b55ffb640 1 -- 172.21.15.66:0/890661267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b50019070 con 0x7f0b600b5dc0 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b60160010 con 0x7f0b600b5dc0 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b55ffb640 1 -- 172.21.15.66:0/890661267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b500129d0 con 0x7f0b600b5dc0 2024-03-30T13:11:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.724+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601608b0 con 0x7f0b600b5dc0 2024-03-30T13:11:59.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.726+0000 7f0b55ffb640 1 -- 172.21.15.66:0/890661267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b50012b70 con 0x7f0b600b5dc0 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.726+0000 7f0b55ffb640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b4406c530 0x7f0b4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.726+0000 7f0b55ffb640 1 -- 172.21.15.66:0/890661267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0b5008bbb0 con 0x7f0b600b5dc0 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.727+0000 7f0b65369640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b4406c530 0x7f0b4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.727+0000 7f0b65369640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b4406c530 0x7f0b4406e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c036000 tx=0x7f0b5c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.727+0000 7f0b6a1bb740 1 Processor -- start 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.727+0000 7f0b6a1bb740 1 -- start start 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb616510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb646300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 0x55fddb646840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fddb580590 con 0x55fddb616a50 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fddb51ed30 con 0x55fddb643570 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fddb57c540 con 0x55fddb643950 2024-03-30T13:11:59.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 0x55fddb646840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 0x55fddb646840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57534/0 (socket says 172.21.15.66:57534) 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 -- 172.21.15.66:0/2738901125 learned_addr learned my addr 172.21.15.66:0/2738901125 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65369640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb616510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b64b68640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb646300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 -- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 msgr2=0x55fddb646300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb646300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 -- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 msgr2=0x55fddb616510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb616510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65b6a640 1 -- 172.21.15.66:0/2738901125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fddb360920 con 0x55fddb616a50 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b64b68640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb646300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.728+0000 7f0b65369640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb616510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 0x55fddb646840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b58007410 tx=0x7f0b580078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef77de238f8d095 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b4affd640 1 -- 172.21.15.66:0/2738901125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580147e0 con 0x55fddb616a50 2024-03-30T13:11:59.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b4affd640 1 -- 172.21.15.66:0/2738901125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5801a070 con 0x55fddb616a50 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b4affd640 1 -- 172.21.15.66:0/2738901125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58014bc0 con 0x55fddb616a50 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/2738901125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 msgr2=0x55fddb646840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 0x55fddb646840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b58007410 tx=0x7f0b580078d0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/2738901125 shutdown_connections 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb616a50 0x55fddb646840 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb646300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/2738901125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb616510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/2738901125 >> 172.21.15.66:0/2738901125 conn(0x55fddb618780 msgr2=0x55fddb661300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/2738901125 shutdown_connections 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.729+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/2738901125 wait complete. 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.730+0000 7f0b6a1bb740 1 -- start start 2024-03-30T13:11:59.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.730+0000 7f0b6a1bb740 1 Processor -- start 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb674410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643950 0x55fddb63e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fddb61ae20 con 0x55fddb63ebc0 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fddb51c9f0 con 0x55fddb643950 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fddb61d660 con 0x55fddb643570 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b6a1bb740 10 client.0 _ll_register_callbacks cb 0x55fddb520c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57546/0 (socket says 172.21.15.66:57546) 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 -- 172.21.15.66:0/1608783101 learned_addr learned my addr 172.21.15.66:0/1608783101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65369640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb674410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b64b68640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643950 0x55fddb63e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 -- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 msgr2=0x55fddb674410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb674410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 -- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643950 msgr2=0x55fddb63e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643950 0x55fddb63e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.731+0000 7f0b65b6a640 1 -- 172.21.15.66:0/1608783101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fddb254210 con 0x55fddb63ebc0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c310 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0b58016000 tx=0x7f0b58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac516f6e7b927f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b3ffff640 1 -- 172.21.15.66:0/1608783101 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5802a680 con 0x55fddb63ebc0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b3ffff640 1 -- 172.21.15.66:0/1608783101 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0b5802a820 con 0x55fddb63ebc0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5801a070 con 0x55fddb63ebc0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b3ffff640 1 -- 172.21.15.66:0/1608783101 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0b58014e40 con 0x55fddb63ebc0 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 10 client.5910 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55fddb3c10b0 con 0x55fddb63ebc0 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b3ffff640 1 -- 172.21.15.66:0/1608783101 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5801a070 con 0x55fddb63ebc0 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b3ffff640 1 -- 172.21.15.66:0/1608783101 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f0b58015470 con 0x55fddb63ebc0 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 10 client.5910 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 10 client.5910 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 20 client.5910 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb645830 0x55fddb645c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 4 client.5910 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.732+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fddb58dad0 con 0x55fddb645830 2024-03-30T13:11:59.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.733+0000 7f0b65369640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb645830 0x55fddb645c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.733+0000 7f0b3ffff640 10 client.5910 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.733+0000 7f0b65369640 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb645830 0x55fddb645c10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b231f619137bd3c server_cookie=467fe380d263d38e in_seq=0 out_seq=0 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b3ffff640 1 -- 172.21.15.66:0/1608783101 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55fddb58dad0 con 0x55fddb645830 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b3ffff640 10 client.5910 handle_command_reply: tid=0 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 client.5910 shutdown 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 msgr2=0x55fddb63c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c310 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0b58016000 tx=0x7f0b58004ab0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 shutdown_connections 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb645830 0x55fddb645c10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c310 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643950 0x55fddb63e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b3ffff640 0 client.5910 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/1608783101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb674410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 >> 172.21.15.66:0/1608783101 conn(0x55fddb616a50 msgr2=0x55fddb61a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 shutdown_connections 2024-03-30T13:11:59.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/1608783101 wait complete. 2024-03-30T13:11:59.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.734+0000 7f0b6a1bb740 20 client.5910 trim_cache size 0 max 16384 2024-03-30T13:11:59.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.749+0000 7f0b6a1bb740 1 Processor -- start 2024-03-30T13:11:59.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.749+0000 7f0b6a1bb740 1 -- start start 2024-03-30T13:11:59.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.749+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb63cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb649d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60201190 con 0x55fddb63ebc0 2024-03-30T13:11:59.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b602001d0 con 0x55fddb643570 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6014ad80 con 0x55fddb643950 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57548/0 (socket says 172.21.15.66:57548) 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 -- 172.21.15.66:0/3686909673 learned_addr learned my addr 172.21.15.66:0/3686909673 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b64b68640 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb63cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb649d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 -- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 msgr2=0x55fddb649d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb649d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 -- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 msgr2=0x55fddb63cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb63cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 -- 172.21.15.66:0/3686909673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fddb22d2d0 con 0x55fddb63ebc0 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b64b68640 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb63cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b65369640 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009960 tx=0x7f0b5c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ff98bda50dd9d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b4a7fc640 1 -- 172.21.15.66:0/3686909673 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c035040 con 0x55fddb63ebc0 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b4a7fc640 1 -- 172.21.15.66:0/3686909673 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c059070 con 0x55fddb63ebc0 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.750+0000 7f0b4a7fc640 1 -- 172.21.15.66:0/3686909673 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c054380 con 0x55fddb63ebc0 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/3686909673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 msgr2=0x55fddb63c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009960 tx=0x7f0b5c030000 comp rx=0 tx=0).stop 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/3686909673 shutdown_connections 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643950 0x55fddb649d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb643570 0x55fddb63cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/3686909673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb63ebc0 0x55fddb63c2f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/3686909673 >> 172.21.15.66:0/3686909673 conn(0x55fddb616a50 msgr2=0x55fddb6710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/3686909673 shutdown_connections 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/3686909673 wait complete. 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.751+0000 7f0b6a1bb740 1 -- start start 2024-03-30T13:11:59.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 Processor -- start 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb63ebc0 0x55fddb649d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb64a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 0x55fddb6644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fddb63b720 con 0x55fddb643950 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ffb80 con 0x55fddb63ebc0 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.752+0000 7f0b6a1bb740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fddb58b0f0 con 0x55fddb643570 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b6a1bb740 10 client.0 _ll_register_callbacks cb 0x7f0b58009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 0x55fddb6644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 0x55fddb6644d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57550/0 (socket says 172.21.15.66:57550) 2024-03-30T13:11:59.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 -- 172.21.15.66:0/700738997 learned_addr learned my addr 172.21.15.66:0/700738997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65369640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb63ebc0 0x55fddb649d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 -- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 msgr2=0x55fddb64a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b64b68640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb64a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb64a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 -- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb63ebc0 msgr2=0x55fddb649d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb63ebc0 0x55fddb649d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 -- 172.21.15.66:0/700738997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fddb641ef0 con 0x55fddb643950 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65369640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb63ebc0 0x55fddb649d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b65b6a640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 0x55fddb6644d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55fddb61ca00 tx=0x7f0b5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f09d2c65fc3bed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b3ffff640 1 -- 172.21.15.66:0/700738997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5803a030 con 0x55fddb643950 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b3ffff640 1 -- 172.21.15.66:0/700738997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0b5802a480 con 0x55fddb643950 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5c059070 con 0x55fddb643950 2024-03-30T13:11:59.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b3ffff640 1 -- 172.21.15.66:0/700738997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0b58043050 con 0x55fddb643950 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.753+0000 7f0b6a1bb740 10 client.5916 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55fddb6ddf90 con 0x55fddb643950 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b3ffff640 1 -- 172.21.15.66:0/700738997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c059070 con 0x55fddb643950 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b3ffff640 1 -- 172.21.15.66:0/700738997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f0b58046020 con 0x55fddb643950 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 10 client.5916 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 10 client.5916 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 20 client.5916 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb6443d0 0x55fddb664a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 4 client.5916 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628323}] 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628323}) -- 0x55fddb587e20 con 0x55fddb6443d0 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b65369640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb6443d0 0x55fddb664a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b3ffff640 10 client.5916 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.754+0000 7f0b65369640 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb6443d0 0x55fddb664a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4f32343f98300d server_cookie=941da4a57bd4abd0 in_seq=0 out_seq=0 2024-03-30T13:11:59.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b3ffff640 1 -- 172.21.15.66:0/700738997 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4332 (crc 0 0 0) 0x55fddb587e20 con 0x55fddb6443d0 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b3ffff640 10 client.5916 handle_command_reply: tid=0 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 client.5916 shutdown 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 msgr2=0x55fddb6644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 0x55fddb6644d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55fddb61ca00 tx=0x7f0b5800b040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 shutdown_connections 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fddb6443d0 0x55fddb664a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fddb643950 0x55fddb6644d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fddb643570 0x55fddb64a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b3ffff640 0 client.5916 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 --2- 172.21.15.66:0/700738997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddb63ebc0 0x55fddb649d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 >> 172.21.15.66:0/700738997 conn(0x55fddb616a50 msgr2=0x55fddb6153a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 shutdown_connections 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 1 -- 172.21.15.66:0/700738997 wait complete. 2024-03-30T13:11:59.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.755+0000 7f0b6a1bb740 20 client.5916 trim_cache size 0 max 16384 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr.h", 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628323, 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.897168+0000", 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.346203+0000", 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.766 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 3340, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.346203+0000", 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:59.767 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:59.768 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3340, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.897168+0000" 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3340, 2024-03-30T13:11:59.769 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.897168+0000" 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 112, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 32, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:59.770 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:59.771 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:59.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:59.773 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.774 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:59.775 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:59.778 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b4406c530 msgr2=0x7f0b4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b4406c530 0x7f0b4406e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c036000 tx=0x7f0b5c03c040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 msgr2=0x7f0b601f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b601f7bd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000d5f0 tx=0x7f0b5000dab0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 shutdown_connections 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0b4406c530 0x7f0b4406e9f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600ba090 0x7f0b60160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b600b5dc0 0x7f0b601f7bd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 --2- 172.21.15.66:0/890661267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6006c620 0x7f0b601e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 >> 172.21.15.66:0/890661267 conn(0x7f0b600678f0 msgr2=0x7f0b600b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 shutdown_connections 2024-03-30T13:11:59.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.756+0000 7f0b675cd640 1 -- 172.21.15.66:0/890661267 wait complete. 2024-03-30T13:11:59.781 DEBUG:tasks.cephfs.test_uninlining:getting inode info #14 of 23:1099511628328 2024-03-30T13:11:59.781 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628328 2024-03-30T13:11:59.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628328 2024-03-30T13:11:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.887+0000 7f76ffe4a640 1 Processor -- start 2024-03-30T13:11:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.888+0000 7f76ffe4a640 1 -- start start 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.888+0000 7f76ffe4a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76f8155b10 0x7f76f80b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.888+0000 7f76ffe4a640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f76f815fae0 0x7f76f80b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76ffe4a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f80b5ff0 con 0x7f76f80d03c0 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76ffe4a640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f76f8155ef0 con 0x7f76f8155b10 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76ffe4a640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80b41b0 con 0x7f76f815fae0 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76f8155b10 0x7f76f80b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76f8155b10 0x7f76f80b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57554/0 (socket says 172.21.15.66:57554) 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 -- 172.21.15.66:0/1854336898 learned_addr learned my addr 172.21.15.66:0/1854336898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/1854336898 >> v2:172.21.15.117:3300/0 conn(0x7f76f815fae0 0x7f76f80b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 -- 172.21.15.66:0/1854336898 >> v2:172.21.15.117:3300/0 conn(0x7f76f815fae0 msgr2=0x7f76f80b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/1854336898 >> v2:172.21.15.117:3300/0 conn(0x7f76f815fae0 0x7f76f80b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 -- 172.21.15.66:0/1854336898 >> v1:172.21.15.117:6789/0 conn(0x7f76f80d03c0 legacy=0x7f76f815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.889+0000 7f76fdbe6640 1 -- 172.21.15.66:0/1854336898 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80b4490 con 0x7f76f8155b10 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.890+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/1854336898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f8155b10 0x7f76f80b4c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f76f806d650 tx=0x7f76f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18d593d27c402c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.890+0000 7f76fcbe4640 1 -- 172.21.15.66:0/1854336898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4034070 con 0x7f76f8155b10 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.890+0000 7f76fcbe4640 1 -- 172.21.15.66:0/1854336898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f4030860 con 0x7f76f8155b10 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.890+0000 7f76fcbe4640 1 -- 172.21.15.66:0/1854336898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4030c30 con 0x7f76f8155b10 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 -- 172.21.15.66:0/1854336898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f8155b10 msgr2=0x7f76f80b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/1854336898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f8155b10 0x7f76f80b4c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f76f806d650 tx=0x7f76f402b9a0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 -- 172.21.15.66:0/1854336898 shutdown_connections 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/1854336898 >> v2:172.21.15.117:3300/0 conn(0x7f76f815fae0 0x7f76f80b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/1854336898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f8155b10 0x7f76f80b4c90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 -- 172.21.15.66:0/1854336898 >> 172.21.15.66:0/1854336898 conn(0x7f76f80d1160 msgr2=0x7f76f80d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 -- 172.21.15.66:0/1854336898 shutdown_connections 2024-03-30T13:11:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.891+0000 7f76ffe4a640 1 -- 172.21.15.66:0/1854336898 wait complete. 2024-03-30T13:11:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.892+0000 7f76ffe4a640 1 Processor -- start 2024-03-30T13:11:59.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.892+0000 7f76ffe4a640 1 -- start start 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76ffe4a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f80d03c0 0x7f76f814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76ffe4a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76f8155b10 0x7f76f806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76ffe4a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 0x7f76f80b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76ffe4a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b62d0 con 0x7f76f815fae0 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76ffe4a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b5ce0 con 0x7f76f80d03c0 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76ffe4a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80d0850 con 0x7f76f8155b10 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76fe3e7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 0x7f76f80b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76fe3e7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 0x7f76f80b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57556/0 (socket says 172.21.15.66:57556) 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76fe3e7640 1 -- 172.21.15.66:0/3780039001 learned_addr learned my addr 172.21.15.66:0/3780039001 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76f8155b10 0x7f76f806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.893+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f80d03c0 0x7f76f814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76fe3e7640 1 -- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76f8155b10 msgr2=0x7f76f806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76f8155b10 0x7f76f806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76fe3e7640 1 -- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f80d03c0 msgr2=0x7f76f814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f80d03c0 0x7f76f814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76fe3e7640 1 -- 172.21.15.66:0/3780039001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80b28c0 con 0x7f76f815fae0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 0x7f76f80b1e70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f76f000c7d0 tx=0x7f76f000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76ee7fc640 1 -- 172.21.15.66:0/3780039001 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0003b70 con 0x7f76f815fae0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76ee7fc640 1 -- 172.21.15.66:0/3780039001 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f0003d30 con 0x7f76f815fae0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76ee7fc640 1 -- 172.21.15.66:0/3780039001 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f000aa90 con 0x7f76f815fae0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f80b2ba0 con 0x7f76f815fae0 2024-03-30T13:11:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.894+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f8149950 con 0x7f76f815fae0 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.896+0000 7f76ee7fc640 1 -- 172.21.15.66:0/3780039001 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f76f000ac30 con 0x7f76f815fae0 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.896+0000 7f76ee7fc640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f76dc06c530 0x7f76dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.896+0000 7f76ee7fc640 1 -- 172.21.15.66:0/3780039001 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f76f008b6a0 con 0x7f76f815fae0 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.896+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f76dc06c530 0x7f76dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.897+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f76dc06c530 0x7f76dc06e9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.897+0000 7f7702a34740 1 Processor -- start 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.897+0000 7f7702a34740 1 -- start start 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f7702a34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566cf930 0x558e566a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f7702a34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558e5660c590 con 0x558e566f03d0 2024-03-30T13:11:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e565aad30 con 0x558e566efff0 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558e56608540 con 0x558e566cf930 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57570/0 (socket says 172.21.15.66:57570) 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 -- 172.21.15.66:0/470544949 learned_addr learned my addr 172.21.15.66:0/470544949 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566cf930 0x558e566a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 -- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566cf930 msgr2=0x558e566a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566cf930 0x558e566a7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 -- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 msgr2=0x558e566edcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566edcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 -- 172.21.15.66:0/470544949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e563ec920 con 0x558e566f03d0 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566cf930 0x558e566a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566a7620 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f76e80078c0 tx=0x7f76e8007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a10438bbb453b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.898+0000 7f76db7fe640 1 -- 172.21.15.66:0/470544949 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8018070 con 0x558e566f03d0 2024-03-30T13:11:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f76db7fe640 1 -- 172.21.15.66:0/470544949 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e8010aa0 con 0x558e566f03d0 2024-03-30T13:11:59.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f76db7fe640 1 -- 172.21.15.66:0/470544949 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8010e50 con 0x558e566f03d0 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 -- 172.21.15.66:0/470544949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 msgr2=0x558e566a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566a7620 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f76e80078c0 tx=0x7f76e8007d80 comp rx=0 tx=0).stop 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 -- 172.21.15.66:0/470544949 shutdown_connections 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566cf930 0x558e566a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566a7620 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 --2- 172.21.15.66:0/470544949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566edcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.899+0000 7f7702a34740 1 -- 172.21.15.66:0/470544949 >> 172.21.15.66:0/470544949 conn(0x558e566a4780 msgr2=0x558e566ed300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.900+0000 7f7702a34740 1 -- 172.21.15.66:0/470544949 shutdown_connections 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.900+0000 7f7702a34740 1 -- 172.21.15.66:0/470544949 wait complete. 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.900+0000 7f7702a34740 1 -- start start 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.901+0000 7f7702a34740 1 Processor -- start 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e566c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558e566a2db0 con 0x558e566f03d0 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e565a89f0 con 0x558e566efff0 2024-03-30T13:11:59.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558e566a6bb0 con 0x558e566c5c50 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f7702a34740 10 client.0 _ll_register_callbacks cb 0x558e565acc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57578/0 (socket says 172.21.15.66:57578) 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f76fd3e5640 1 -- 172.21.15.66:0/516120337 learned_addr learned my addr 172.21.15.66:0/516120337 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e566c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.902+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76fd3e5640 1 -- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 msgr2=0x558e566c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e566c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76fd3e5640 1 -- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 msgr2=0x558e566e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566e7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76fd3e5640 1 -- 172.21.15.66:0/516120337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e562e0210 con 0x558e566f03d0 2024-03-30T13:11:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566c4b30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f76e80077b0 tx=0x7f76e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c3cba6fbd2bf08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76d8ff9640 1 -- 172.21.15.66:0/516120337 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8018040 con 0x558e566f03d0 2024-03-30T13:11:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76d8ff9640 1 -- 172.21.15.66:0/516120337 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f76e8003610 con 0x558e566f03d0 2024-03-30T13:11:59.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e8010aa0 con 0x558e566f03d0 2024-03-30T13:11:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76d8ff9640 1 -- 172.21.15.66:0/516120337 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f76e802f050 con 0x558e566f03d0 2024-03-30T13:11:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f7702a34740 10 client.5928 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558e5644d0b0 con 0x558e566f03d0 2024-03-30T13:11:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76d8ff9640 1 -- 172.21.15.66:0/516120337 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e8010aa0 con 0x558e566f03d0 2024-03-30T13:11:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f76d8ff9640 1 -- 172.21.15.66:0/516120337 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f76e8032020 con 0x558e566f03d0 2024-03-30T13:11:59.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f7702a34740 10 client.5928 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.903+0000 7f7702a34740 10 client.5928 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f7702a34740 20 client.5928 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f7702a34740 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566cd740 0x558e566cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f7702a34740 4 client.5928 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e56619ad0 con 0x558e566cd740 2024-03-30T13:11:59.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566cd740 0x558e566cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f76d8ff9640 10 client.5928 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.904+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566cd740 0x558e566cdb60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df4ae333de0d0e5 server_cookie=fc7be51d62122826 in_seq=0 out_seq=0 2024-03-30T13:11:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f76d8ff9640 1 -- 172.21.15.66:0/516120337 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x558e56619ad0 con 0x558e566cd740 2024-03-30T13:11:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f76d8ff9640 10 client.5928 handle_command_reply: tid=0 2024-03-30T13:11:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 client.5928 shutdown 2024-03-30T13:11:59.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 msgr2=0x558e566c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566c4b30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f76e80077b0 tx=0x7f76e8004730 comp rx=0 tx=0).stop 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 shutdown_connections 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566cd740 0x558e566cdb60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e566c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566f03d0 0x558e566c4b30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 --2- 172.21.15.66:0/516120337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566efff0 0x558e566e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f76d8ff9640 0 client.5928 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 >> 172.21.15.66:0/516120337 conn(0x558e566cf930 msgr2=0x558e566ef130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 shutdown_connections 2024-03-30T13:11:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 1 -- 172.21.15.66:0/516120337 wait complete. 2024-03-30T13:11:59.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.905+0000 7f7702a34740 20 client.5928 trim_cache size 0 max 16384 2024-03-30T13:11:59.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.919+0000 7f7702a34740 1 Processor -- start 2024-03-30T13:11:59.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 -- start start 2024-03-30T13:11:59.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566c5c50 0x558e566cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566f03d0 0x558e566a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76f81fcdb0 con 0x558e566efff0 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b1990 con 0x558e566c5c50 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80b0cf0 con 0x558e566f03d0 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57590/0 (socket says 172.21.15.66:57590) 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.920+0000 7f76fd3e5640 1 -- 172.21.15.66:0/3839819772 learned_addr learned my addr 172.21.15.66:0/3839819772 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76fd3e5640 1 -- 172.21.15.66:0/3839819772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566f03d0 msgr2=0x558e566a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566f03d0 0x558e566a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76fd3e5640 1 -- 172.21.15.66:0/3839819772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566c5c50 msgr2=0x558e566cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566c5c50 0x558e566cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76fd3e5640 1 -- 172.21.15.66:0/3839819772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e562b92d0 con 0x558e566efff0 2024-03-30T13:11:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566a7ce0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f76e80073e0 tx=0x7f76e8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4672e7a929c6514 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76daffd640 1 -- 172.21.15.66:0/3839819772 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8038030 con 0x558e566efff0 2024-03-30T13:11:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76daffd640 1 -- 172.21.15.66:0/3839819772 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e80106d0 con 0x558e566efff0 2024-03-30T13:11:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f76daffd640 1 -- 172.21.15.66:0/3839819772 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8018040 con 0x558e566efff0 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 -- 172.21.15.66:0/3839819772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 msgr2=0x558e566a7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566a7ce0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f76e80073e0 tx=0x7f76e8020040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 -- 172.21.15.66:0/3839819772 shutdown_connections 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566f03d0 0x558e566a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566a7ce0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 --2- 172.21.15.66:0/3839819772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566c5c50 0x558e566cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 -- 172.21.15.66:0/3839819772 >> 172.21.15.66:0/3839819772 conn(0x558e566cf930 msgr2=0x558e5669d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 -- 172.21.15.66:0/3839819772 shutdown_connections 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.921+0000 7f7702a34740 1 -- 172.21.15.66:0/3839819772 wait complete. 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.922+0000 7f7702a34740 1 -- start start 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.922+0000 7f7702a34740 1 Processor -- start 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e567706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566f03d0 0x558e566c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b1340 con 0x558e566efff0 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e5669e2f0 con 0x558e566f03d0 2024-03-30T13:11:59.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558e566d67a0 con 0x558e566c5c50 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f7702a34740 10 client.0 _ll_register_callbacks cb 0x7f76e80076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57606/0 (socket says 172.21.15.66:57606) 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 -- 172.21.15.66:0/209501908 learned_addr learned my addr 172.21.15.66:0/209501908 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566f03d0 0x558e566c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e567706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 -- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 msgr2=0x558e567706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e567706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 -- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566f03d0 msgr2=0x558e566c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566f03d0 0x558e566c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fd3e5640 1 -- 172.21.15.66:0/209501908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e566ccdc0 con 0x558e566efff0 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e567706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.923+0000 7f76fe3e7640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566f03d0 0x558e566c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:59.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76fd3e5640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566c1e10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f76e8010620 tx=0x7f76e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a09dc9d12bf7f56 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76d8ff9640 1 -- 172.21.15.66:0/209501908 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8038050 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76d8ff9640 1 -- 172.21.15.66:0/209501908 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f76e803d670 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e56613e20 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76d8ff9640 1 -- 172.21.15.66:0/209501908 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f76e8047050 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 10 client.5934 fetch_fsmap learned FSMap version 18 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558e56771b40 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76d8ff9640 1 -- 172.21.15.66:0/209501908 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558e56613e20 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76d8ff9640 1 -- 172.21.15.66:0/209501908 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f76e804a020 con 0x558e566efff0 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 10 client.5934 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 10 client.5934 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 20 client.5934 populate_metadata read hostname 'smithi066' 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566c5010 0x558e56770dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 4 client.5934 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628328}] 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628328}) -- 0x558e56591a90 con 0x558e566c5010 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.924+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566c5010 0x558e56770dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f76d8ff9640 10 client.5934 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f76fdbe6640 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566c5010 0x558e56770dd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddad00579b329550 server_cookie=adc4f89586271b26 in_seq=0 out_seq=0 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f76d8ff9640 1 -- 172.21.15.66:0/209501908 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4331 (crc 0 0 0) 0x558e56591a90 con 0x558e566c5010 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f76d8ff9640 10 client.5934 handle_command_reply: tid=0 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 client.5934 shutdown 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 msgr2=0x558e566c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566c1e10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f76e8010620 tx=0x7f76e8008040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 shutdown_connections 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558e566c5010 0x558e56770dd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e566f03d0 0x558e566c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e566efff0 0x558e566c1e10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 --2- 172.21.15.66:0/209501908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558e566c5c50 0x558e567706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 >> 172.21.15.66:0/209501908 conn(0x558e566cf930 msgr2=0x558e566c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f76d8ff9640 0 client.5934 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 shutdown_connections 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.925+0000 7f7702a34740 1 -- 172.21.15.66:0/209501908 wait complete. 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f7702a34740 20 client.5934 trim_cache size 0 max 16384 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/Kconfig", 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628328, 2024-03-30T13:11:59.935 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.900532+0000", 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.446202+0000", 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.936 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 228, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:11:59.937 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.446202+0000", 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:11:59.938 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 228, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.939 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.900532+0000" 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 228, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.900532+0000" 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 84, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 2, 2024-03-30T13:11:59.940 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:11:59.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.942 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:11:59.943 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:11:59.944 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:11:59.945 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:11:59.947 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:11:59.948 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:11:59.949 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:11:59.950 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:11:59.950 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:11:59.950 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:11:59.950 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:11:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f76dc06c530 msgr2=0x7f76dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f76dc06c530 0x7f76dc06e9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f4008030 comp rx=0 tx=0).stop 2024-03-30T13:11:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 msgr2=0x7f76f80b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 0x7f76f80b1e70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f76f000c7d0 tx=0x7f76f000cc90 comp rx=0 tx=0).stop 2024-03-30T13:11:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 shutdown_connections 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f76dc06c530 0x7f76dc06e9f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f815fae0 0x7f76f80b1e70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76f8155b10 0x7f76f806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 --2- 172.21.15.66:0/3780039001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76f80d03c0 0x7f76f814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 >> 172.21.15.66:0/3780039001 conn(0x7f76f80d1160 msgr2=0x7f76f80706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 shutdown_connections 2024-03-30T13:11:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:11:59.926+0000 7f76ffe4a640 1 -- 172.21.15.66:0/3780039001 wait complete. 2024-03-30T13:11:59.954 DEBUG:tasks.cephfs.test_uninlining:getting inode info #15 of 23:1099511628341 2024-03-30T13:11:59.954 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628341 2024-03-30T13:11:59.954 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628341 2024-03-30T13:12:00.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.062+0000 7f7f7eb38640 1 Processor -- start 2024-03-30T13:12:00.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.062+0000 7f7f7eb38640 1 -- start start 2024-03-30T13:12:00.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7eb38640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7f780ba090 0x7f7f780d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7eb38640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7f780b73d0 0x7f7f780b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7eb38640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f780d2ff0 con 0x7f7f780b5dc0 2024-03-30T13:12:00.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7eb38640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7f780d32d0 con 0x7f7f780b73d0 2024-03-30T13:12:00.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7eb38640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7f780d35b0 con 0x7f7f780ba090 2024-03-30T13:12:00.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7f780b73d0 0x7f7f780b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7f780b73d0 0x7f7f780b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57614/0 (socket says 172.21.15.66:57614) 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/21112207 learned_addr learned my addr 172.21.15.66:0/21112207 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/21112207 >> v2:172.21.15.117:3300/0 conn(0x7f7f780ba090 0x7f7f780d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/21112207 >> v2:172.21.15.117:3300/0 conn(0x7f7f780ba090 msgr2=0x7f7f780d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/21112207 >> v2:172.21.15.117:3300/0 conn(0x7f7f780ba090 0x7f7f780d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/21112207 >> v1:172.21.15.117:6789/0 conn(0x7f7f780b5dc0 legacy=0x7f7f780b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.063+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/21112207 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f780d3890 con 0x7f7f780b73d0 2024-03-30T13:12:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.064+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/21112207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780b73d0 0x7f7f780b7820 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009e50 tx=0x7f7f7002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531566ac77bb4f2f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.064+0000 7f7f6f7fe640 1 -- 172.21.15.66:0/21112207 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7002cb20 con 0x7f7f780b73d0 2024-03-30T13:12:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.064+0000 7f7f6f7fe640 1 -- 172.21.15.66:0/21112207 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7002ccc0 con 0x7f7f780b73d0 2024-03-30T13:12:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.064+0000 7f7f6f7fe640 1 -- 172.21.15.66:0/21112207 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70034610 con 0x7f7f780b73d0 2024-03-30T13:12:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 -- 172.21.15.66:0/21112207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780b73d0 msgr2=0x7f7f780b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/21112207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780b73d0 0x7f7f780b7820 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009e50 tx=0x7f7f7002bf80 comp rx=0 tx=0).stop 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 -- 172.21.15.66:0/21112207 shutdown_connections 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/21112207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780b73d0 0x7f7f780b7820 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/21112207 >> v2:172.21.15.117:3300/0 conn(0x7f7f780ba090 0x7f7f780d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 -- 172.21.15.66:0/21112207 >> 172.21.15.66:0/21112207 conn(0x7f7f780678f0 msgr2=0x7f7f780d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 -- 172.21.15.66:0/21112207 shutdown_connections 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.065+0000 7f7f7eb38640 1 -- 172.21.15.66:0/21112207 wait complete. 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.066+0000 7f7f7eb38640 1 Processor -- start 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.066+0000 7f7f7eb38640 1 -- start start 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7eb38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f780b5dc0 0x7f7f780b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7eb38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f780b73d0 0x7f7f78158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7eb38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 0x7f7f781f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7eb38640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7815e880 con 0x7f7f780ba090 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7eb38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f780d4090 con 0x7f7f780b5dc0 2024-03-30T13:12:00.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7eb38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7815ea00 con 0x7f7f780b73d0 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 0x7f7f781f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 0x7f7f781f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57618/0 (socket says 172.21.15.66:57618) 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/3089909646 learned_addr learned my addr 172.21.15.66:0/3089909646 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f780b5dc0 0x7f7f780b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f780b73d0 0x7f7f78158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f780b73d0 msgr2=0x7f7f78158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f780b73d0 0x7f7f78158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f780b5dc0 msgr2=0x7f7f780b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f780b5dc0 0x7f7f780b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.067+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/3089909646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f781f3f00 con 0x7f7f780ba090 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.068+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 0x7f7f781f0ff0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7f70036040 tx=0x7f7f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.068+0000 7f7f6d7fa640 1 -- 172.21.15.66:0/3089909646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70043020 con 0x7f7f780ba090 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.068+0000 7f7f6d7fa640 1 -- 172.21.15.66:0/3089909646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7003e450 con 0x7f7f780ba090 2024-03-30T13:12:00.076 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.068+0000 7f7f6d7fa640 1 -- 172.21.15.66:0/3089909646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70009510 con 0x7f7f780ba090 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.068+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f781594c0 con 0x7f7f780ba090 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.068+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f781f4460 con 0x7f7f780ba090 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.069+0000 7f7f6d7fa640 1 -- 172.21.15.66:0/3089909646 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f70046020 con 0x7f7f780ba090 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.070+0000 7f7f6d7fa640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7f6406c530 0x7f7f6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.070+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7f6406c530 0x7f7f6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.070+0000 7f7f6d7fa640 1 -- 172.21.15.66:0/3089909646 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7f700ade90 con 0x7f7f780ba090 2024-03-30T13:12:00.077 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.070+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7f6406c530 0x7f7f6406e9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f78160270 tx=0x7f7f74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.071+0000 7f7f817f3740 1 Processor -- start 2024-03-30T13:12:00.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.071+0000 7f7f817f3740 1 -- start start 2024-03-30T13:12:00.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.071+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 0x556c597f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.071+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59826300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c597f6a50 0x556c59826840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f817f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c59760590 con 0x556c59823570 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c596fed30 con 0x556c59823950 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x556c5975c540 con 0x556c597f6a50 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 0x556c597f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 0x556c597f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57630/0 (socket says 172.21.15.66:57630) 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 -- 172.21.15.66:0/2401697425 learned_addr learned my addr 172.21.15.66:0/2401697425 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59826300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 -- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c597f6a50 msgr2=0x556c59826840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c597f6a50 0x556c59826840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c597f6a50 0x556c59826840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 -- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 msgr2=0x556c59826300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59826300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 -- 172.21.15.66:0/2401697425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c59540920 con 0x556c59823570 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59826300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c597f6a50 0x556c59826840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 0x556c597f6510 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400fa40 tx=0x7f7f74012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1352dbaafd9b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f5e7fc640 1 -- 172.21.15.66:0/2401697425 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74025070 con 0x556c59823570 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f5e7fc640 1 -- 172.21.15.66:0/2401697425 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7401b050 con 0x556c59823570 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.072+0000 7f7f5e7fc640 1 -- 172.21.15.66:0/2401697425 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74014de0 con 0x556c59823570 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 -- 172.21.15.66:0/2401697425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 msgr2=0x556c597f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 0x556c597f6510 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400fa40 tx=0x7f7f74012000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 -- 172.21.15.66:0/2401697425 shutdown_connections 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c597f6a50 0x556c59826840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59826300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 --2- 172.21.15.66:0/2401697425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59823570 0x556c597f6510 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 -- 172.21.15.66:0/2401697425 >> 172.21.15.66:0/2401697425 conn(0x556c597f8780 msgr2=0x556c59841300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 -- 172.21.15.66:0/2401697425 shutdown_connections 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.073+0000 7f7f817f3740 1 -- 172.21.15.66:0/2401697425 wait complete. 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.074+0000 7f7f817f3740 1 -- start start 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.074+0000 7f7f817f3740 1 Processor -- start 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c598522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59829ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 0x556c59829ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c597fae20 con 0x556c59854990 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c596fc9f0 con 0x556c59823950 2024-03-30T13:12:00.087 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x556c597fd660 con 0x556c59823570 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 10 client.0 _ll_register_callbacks cb 0x556c59700c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 0x556c59829ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 0x556c59829ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57644/0 (socket says 172.21.15.66:57644) 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/1261140005 learned_addr learned my addr 172.21.15.66:0/1261140005 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c598522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59829ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 msgr2=0x556c598522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c598522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 msgr2=0x556c59829ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59829ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/1261140005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c59434210 con 0x556c59854990 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c598522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59829ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 0x556c59829ff0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7f700094a0 tx=0x7f7f7007b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb87a3719ddb6281 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f577fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f577fe640 1 -- 172.21.15.66:0/1261140005 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f700847e0 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f577fe640 1 -- 172.21.15.66:0/1261140005 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7f70084350 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.076+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7401b050 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f577fe640 1 -- 172.21.15.66:0/1261140005 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7f70084550 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 10 client.5946 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x556c595a10b0 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f577fe640 1 -- 172.21.15.66:0/1261140005 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7401b050 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f577fe640 1 -- 172.21.15.66:0/1261140005 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7f700a5b70 con 0x556c59854990 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 10 client.5946 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 10 client.5946 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 20 client.5946 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c59825db0 0x556c59854df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 4 client.5946 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5976dad0 con 0x556c59825db0 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c59825db0 0x556c59854df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f577fe640 10 client.5946 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.077+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c59825db0 0x556c59854df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a1a79a8d7fe2a5 server_cookie=43b19b09f3ae3aed in_seq=0 out_seq=0 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f577fe640 1 -- 172.21.15.66:0/1261140005 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x556c5976dad0 con 0x556c59825db0 2024-03-30T13:12:00.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f577fe640 10 client.5946 handle_command_reply: tid=0 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 client.5946 shutdown 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 msgr2=0x556c59829ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 0x556c59829ff0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7f700094a0 tx=0x7f7f7007b6d0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 shutdown_connections 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c59825db0 0x556c59854df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f577fe640 0 client.5946 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59854990 0x556c59829ff0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59829ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 --2- 172.21.15.66:0/1261140005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c598522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 >> 172.21.15.66:0/1261140005 conn(0x556c597f6a50 msgr2=0x556c59844810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 shutdown_connections 2024-03-30T13:12:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.078+0000 7f7f817f3740 1 -- 172.21.15.66:0/1261140005 wait complete. 2024-03-30T13:12:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.079+0000 7f7f817f3740 20 client.5946 trim_cache size 0 max 16384 2024-03-30T13:12:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.093+0000 7f7f817f3740 1 Processor -- start 2024-03-30T13:12:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.093+0000 7f7f817f3740 1 -- start start 2024-03-30T13:12:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823570 0x556c59850e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823950 0x556c59844810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c59844d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f817f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f782012c0 con 0x556c59825db0 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f782002a0 con 0x556c59823570 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7f780cd2c0 con 0x556c59823950 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c59844d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c59844d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57652/0 (socket says 172.21.15.66:57652) 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/4014367400 learned_addr learned my addr 172.21.15.66:0/4014367400 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823570 0x556c59850e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823950 0x556c59844810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823950 msgr2=0x556c59844810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823950 0x556c59844810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823570 msgr2=0x556c59850e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823570 0x556c59850e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/4014367400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5940d2d0 con 0x556c59825db0 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823950 0x556c59844810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c59844d50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7f7007b9b0 tx=0x7f7f70040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a688cfa62a8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f5dffb640 1 -- 172.21.15.66:0/4014367400 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f700847e0 con 0x556c59825db0 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f5dffb640 1 -- 172.21.15.66:0/4014367400 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7002cb30 con 0x556c59825db0 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.094+0000 7f7f5dffb640 1 -- 172.21.15.66:0/4014367400 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7007b100 con 0x556c59825db0 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 -- 172.21.15.66:0/4014367400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 msgr2=0x556c59844d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c59844d50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7f7007b9b0 tx=0x7f7f70040000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 -- 172.21.15.66:0/4014367400 shutdown_connections 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c59844d50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823950 0x556c59844810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 --2- 172.21.15.66:0/4014367400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823570 0x556c59850e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 -- 172.21.15.66:0/4014367400 >> 172.21.15.66:0/4014367400 conn(0x556c597f6a50 msgr2=0x556c5983ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 -- 172.21.15.66:0/4014367400 shutdown_connections 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.095+0000 7f7f817f3740 1 -- 172.21.15.66:0/4014367400 wait complete. 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.096+0000 7f7f817f3740 1 -- start start 2024-03-30T13:12:00.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.096+0000 7f7f817f3740 1 Processor -- start 2024-03-30T13:12:00.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.096+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c59844810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.096+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59844d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.096+0000 7f7f817f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c598bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.096+0000 7f7f817f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c598517b0 con 0x556c59825db0 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f781ffc60 con 0x556c59823950 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f817f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x556c598276d0 con 0x556c59823570 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f817f3740 10 client.0 _ll_register_callbacks cb 0x7f7f70009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c598bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c598bd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57658/0 (socket says 172.21.15.66:57658) 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/804402898 learned_addr learned my addr 172.21.15.66:0/804402898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f6ffff640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59844d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c59844810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 msgr2=0x556c59844810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c59844810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 msgr2=0x556c59844d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59844d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 -- 172.21.15.66:0/804402898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c598643d0 con 0x556c59825db0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f7d0d5640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c598bd1c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7f700073c0 tx=0x7f7f700ac0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11905781f081253f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f577fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f577fe640 1 -- 172.21.15.66:0/804402898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f700847e0 con 0x556c59825db0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f577fe640 1 -- 172.21.15.66:0/804402898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f7f7007b310 con 0x556c59825db0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7002cb30 con 0x556c59825db0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f577fe640 1 -- 172.21.15.66:0/804402898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7f7008b050 con 0x556c59825db0 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f817f3740 10 client.5952 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.097+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x556c597fb530 con 0x556c59825db0 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f577fe640 1 -- 172.21.15.66:0/804402898 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7002cb30 con 0x556c59825db0 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f577fe640 1 -- 172.21.15.66:0/804402898 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7f7008e020 con 0x556c59825db0 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f817f3740 10 client.5952 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f817f3740 10 client.5952 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f817f3740 20 client.5952 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f817f3740 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c5981e570 0x556c598bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f817f3740 4 client.5952 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628341}] 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628341}) -- 0x556c59767e20 con 0x556c5981e570 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c5981e570 0x556c598bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f577fe640 10 client.5952 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.098+0000 7f7f7c8d4640 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c5981e570 0x556c598bd700 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde14544a99324e8 server_cookie=be72d4c4d015207d in_seq=0 out_seq=0 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f577fe640 1 -- 172.21.15.66:0/804402898 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4344 (crc 0 0 0) 0x556c59767e20 con 0x556c5981e570 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f577fe640 10 client.5952 handle_command_reply: tid=0 2024-03-30T13:12:00.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 client.5952 shutdown 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 msgr2=0x556c598bd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c598bd1c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7f700073c0 tx=0x7f7f700ac0e0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 shutdown_connections 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556c5981e570 0x556c598bd700 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f577fe640 0 client.5952 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c59825db0 0x556c598bd1c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c59823950 0x556c59844d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 --2- 172.21.15.66:0/804402898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c59823570 0x556c59844810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 >> 172.21.15.66:0/804402898 conn(0x556c597f6a50 msgr2=0x556c59850e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 shutdown_connections 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 1 -- 172.21.15.66:0/804402898 wait complete. 2024-03-30T13:12:00.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.099+0000 7f7f817f3740 20 client.5952 trim_cache size 0 max 16384 2024-03-30T13:12:00.111 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:00.111 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debug.h", 2024-03-30T13:12:00.111 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628341, 2024-03-30T13:12:00.111 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:00.111 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.901519+0000", 2024-03-30T13:12:00.111 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.460202+0000", 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:00.112 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:00.113 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:00.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.113 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:00.113 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:00.113 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:00.113 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:00.114 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:00.114 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:00.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.114 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:00.114 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 2579, 2024-03-30T13:12:00.114 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:00.115 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:00.115 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:00.115 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:00.115 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:00.115 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.460202+0000", 2024-03-30T13:12:00.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 3, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:12:00.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.117 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2579, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.901519+0000" 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2579, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.901519+0000" 2024-03-30T13:12:00.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 86, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 28, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:12:00.119 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:00.120 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:00.121 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.122 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:00.123 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:00.124 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:00.125 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:00.126 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:00.127 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:00.128 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7f6406c530 msgr2=0x7f7f6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7f6406c530 0x7f7f6406e9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f78160270 tx=0x7f7f74009040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 msgr2=0x7f7f781f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 0x7f7f781f0ff0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7f70036040 tx=0x7f7f70004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 shutdown_connections 2024-03-30T13:12:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7f6406c530 0x7f7f6406e9f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f780ba090 0x7f7f781f0ff0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f780b73d0 0x7f7f78158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 --2- 172.21.15.66:0/3089909646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f780b5dc0 0x7f7f780b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 >> 172.21.15.66:0/3089909646 conn(0x7f7f780678f0 msgr2=0x7f7f780b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 shutdown_connections 2024-03-30T13:12:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.100+0000 7f7f7eb38640 1 -- 172.21.15.66:0/3089909646 wait complete. 2024-03-30T13:12:00.130 DEBUG:tasks.cephfs.test_uninlining:getting inode info #16 of 23:1099511628348 2024-03-30T13:12:00.130 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628348 2024-03-30T13:12:00.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628348 2024-03-30T13:12:00.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.237+0000 7fd680a93640 1 Processor -- start 2024-03-30T13:12:00.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.237+0000 7fd680a93640 1 -- start start 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd680a93640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd67c158a20 0x7fd67c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd680a93640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd67c158e00 0x7fd67c06bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd680a93640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c072570 con 0x7fd67c0b0030 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd680a93640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c072850 con 0x7fd67c158e00 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd680a93640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c072b30 con 0x7fd67c158a20 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd67c158e00 0x7fd67c06bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd67c158e00 0x7fd67c06bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57666/0 (socket says 172.21.15.66:57666) 2024-03-30T13:12:00.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 -- 172.21.15.66:0/1637880862 learned_addr learned my addr 172.21.15.66:0/1637880862 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd67ad9d640 1 --1- 172.21.15.66:0/1637880862 >> v1:172.21.15.66:6789/0 conn(0x7fd67c0b0030 0x7fd67c0b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37518/0 (socket says 172.21.15.66:37518) 2024-03-30T13:12:00.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd67a59c640 1 --2- 172.21.15.66:0/1637880862 >> v2:172.21.15.117:3300/0 conn(0x7fd67c158a20 0x7fd67c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 -- 172.21.15.66:0/1637880862 >> v2:172.21.15.117:3300/0 conn(0x7fd67c158a20 msgr2=0x7fd67c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 --2- 172.21.15.66:0/1637880862 >> v2:172.21.15.117:3300/0 conn(0x7fd67c158a20 0x7fd67c06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 -- 172.21.15.66:0/1637880862 >> v1:172.21.15.66:6789/0 conn(0x7fd67c0b0030 legacy=0x7fd67c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.238+0000 7fd679d9b640 1 -- 172.21.15.66:0/1637880862 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c072e10 con 0x7fd67c158e00 2024-03-30T13:12:00.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.239+0000 7fd679d9b640 1 --2- 172.21.15.66:0/1637880862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158e00 0x7fd67c06bac0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0b7380 tx=0x7fd67002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7f12383910fb59 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.239+0000 7fd67959a640 1 -- 172.21.15.66:0/1637880862 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670034070 con 0x7fd67c158e00 2024-03-30T13:12:00.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.239+0000 7fd67959a640 1 -- 172.21.15.66:0/1637880862 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670030860 con 0x7fd67c158e00 2024-03-30T13:12:00.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.239+0000 7fd67959a640 1 -- 172.21.15.66:0/1637880862 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670030c30 con 0x7fd67c158e00 2024-03-30T13:12:00.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 -- 172.21.15.66:0/1637880862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158e00 msgr2=0x7fd67c06bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 --2- 172.21.15.66:0/1637880862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158e00 0x7fd67c06bac0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0b7380 tx=0x7fd67002b9a0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 -- 172.21.15.66:0/1637880862 shutdown_connections 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 --2- 172.21.15.66:0/1637880862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158e00 0x7fd67c06bac0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 --2- 172.21.15.66:0/1637880862 >> v2:172.21.15.117:3300/0 conn(0x7fd67c158a20 0x7fd67c06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 -- 172.21.15.66:0/1637880862 >> 172.21.15.66:0/1637880862 conn(0x7fd67c0d1160 msgr2=0x7fd67c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 -- 172.21.15.66:0/1637880862 shutdown_connections 2024-03-30T13:12:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.240+0000 7fd680a93640 1 -- 172.21.15.66:0/1637880862 wait complete. 2024-03-30T13:12:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.242+0000 7fd680a93640 1 Processor -- start 2024-03-30T13:12:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.242+0000 7fd680a93640 1 -- start start 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd680a93640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd67c0b0030 0x7fd67c149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd680a93640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 0x7fd67c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd680a93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c158e00 0x7fd67c0ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd680a93640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0b4290 con 0x7fd67c158a20 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd680a93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0b4110 con 0x7fd67c158e00 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd680a93640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c0b4410 con 0x7fd67c0b0030 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 0x7fd67c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 0x7fd67c1553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57668/0 (socket says 172.21.15.66:57668) 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 -- 172.21.15.66:0/3587504899 learned_addr learned my addr 172.21.15.66:0/3587504899 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c158e00 0x7fd67c0ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd67c0b0030 0x7fd67c149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 -- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd67c0b0030 msgr2=0x7fd67c149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd67c0b0030 0x7fd67c149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 -- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c158e00 msgr2=0x7fd67c0ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c158e00 0x7fd67c0ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 -- 172.21.15.66:0/3587504899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c156bc0 con 0x7fd67c158a20 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c158e00 0x7fd67c0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd67c0b0030 0x7fd67c149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.244+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 0x7fd67c1553b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd67002b9e0 tx=0x7fd6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.245+0000 7fd66affd640 1 -- 172.21.15.66:0/3587504899 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670034040 con 0x7fd67c158a20 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.245+0000 7fd66affd640 1 -- 172.21.15.66:0/3587504899 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670039d90 con 0x7fd67c158a20 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.245+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c156e60 con 0x7fd67c158a20 2024-03-30T13:12:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.245+0000 7fd66affd640 1 -- 172.21.15.66:0/3587504899 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670043610 con 0x7fd67c158a20 2024-03-30T13:12:00.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.245+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c14a370 con 0x7fd67c158a20 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.246+0000 7fd66affd640 1 -- 172.21.15.66:0/3587504899 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd67004a060 con 0x7fd67c158a20 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.247+0000 7fd66affd640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd65c06c560 0x7fd65c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.247+0000 7fd66affd640 1 -- 172.21.15.66:0/3587504899 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd6700ad8f0 con 0x7fd67c158a20 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.247+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd65c06c560 0x7fd65c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.247+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd65c06c560 0x7fd65c06ea20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd66c006fd0 tx=0x7fd66c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.247+0000 7fd683322740 1 Processor -- start 2024-03-30T13:12:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 -- start start 2024-03-30T13:12:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c452434510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452464300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 0x55c452464840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c45239e590 con 0x55c452434a50 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c45233cd30 con 0x55c452461570 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c45239a540 con 0x55c452461950 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.248+0000 7fd67ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 0x55c452464840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 0x55c452464840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57682/0 (socket says 172.21.15.66:57682) 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 -- 172.21.15.66:0/3249270088 learned_addr learned my addr 172.21.15.66:0/3249270088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c452434510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452464300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 -- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 msgr2=0x55c452464300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452464300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 -- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 msgr2=0x55c452434510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c452434510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 -- 172.21.15.66:0/3249270088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45217e920 con 0x55c452434a50 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 0x55c452464840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd674007500 tx=0x7fd6740079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec4fc4e92107ec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd65bfff640 1 -- 172.21.15.66:0/3249270088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674019070 con 0x55c452434a50 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd65bfff640 1 -- 172.21.15.66:0/3249270088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd674001ca0 con 0x55c452434a50 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.249+0000 7fd65bfff640 1 -- 172.21.15.66:0/3249270088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674002050 con 0x55c452434a50 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 -- 172.21.15.66:0/3249270088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 msgr2=0x55c452464840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 0x55c452464840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd674007500 tx=0x7fd6740079c0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 -- 172.21.15.66:0/3249270088 shutdown_connections 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452434a50 0x55c452464840 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452464300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 --2- 172.21.15.66:0/3249270088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c452434510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 -- 172.21.15.66:0/3249270088 >> 172.21.15.66:0/3249270088 conn(0x55c452436780 msgr2=0x55c45247f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 -- 172.21.15.66:0/3249270088 shutdown_connections 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 -- 172.21.15.66:0/3249270088 wait complete. 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.250+0000 7fd683322740 1 -- start start 2024-03-30T13:12:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.251+0000 7fd683322740 1 Processor -- start 2024-03-30T13:12:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.251+0000 7fd683322740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c4524548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461950 0x55c4524663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c45246b110 0x55c45246b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c452438e20 con 0x55c452461570 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c45233a9f0 con 0x55c452461950 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c45243b660 con 0x55c45246b110 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c4524548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 10 client.0 _ll_register_callbacks cb 0x55c45233ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c4524548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57694/0 (socket says 172.21.15.66:57694) 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 -- 172.21.15.66:0/3071471619 learned_addr learned my addr 172.21.15.66:0/3071471619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461950 0x55c4524663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c45246b110 0x55c45246b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 -- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c45246b110 msgr2=0x55c45246b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c45246b110 0x55c45246b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 -- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461950 msgr2=0x55c4524663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461950 0x55c4524663e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 -- 172.21.15.66:0/3071471619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c452072210 con 0x55c452461570 2024-03-30T13:12:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c45246b110 0x55c45246b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd67a59c640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c4524548e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd66c014740 tx=0x7fd66c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3bed8433de88f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd6597fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd6597fa640 1 -- 172.21.15.66:0/3071471619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c025070 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd6597fa640 1 -- 172.21.15.66:0/3071471619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd66c0159e0 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd674001ca0 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.252+0000 7fd6597fa640 1 -- 172.21.15.66:0/3071471619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd66c030050 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 10 client.5967 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55c4521df0b0 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd6597fa640 1 -- 172.21.15.66:0/3071471619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd674001ca0 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd6597fa640 1 -- 172.21.15.66:0/3071471619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fd66c033020 con 0x55c452461570 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 10 client.5967 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 10 client.5967 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 20 client.5967 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c45248f630 0x55c452469f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 4 client.5967 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4523abad0 con 0x55c45248f630 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c45248f630 0x55c452469f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd6597fa640 10 client.5967 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.253+0000 7fd679d9b640 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c45248f630 0x55c452469f60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c948c7a26dc933 server_cookie=61e3d34f60fe2e7d in_seq=0 out_seq=0 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd6597fa640 1 -- 172.21.15.66:0/3071471619 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c4523abad0 con 0x55c45248f630 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd6597fa640 10 client.5967 handle_command_reply: tid=0 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 client.5967 shutdown 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 msgr2=0x55c4524548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c4524548e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd66c014740 tx=0x7fd66c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 shutdown_connections 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c45248f630 0x55c452469f60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c45246b110 0x55c45246b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461950 0x55c4524663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd6597fa640 0 client.5967 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 --2- 172.21.15.66:0/3071471619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c4524548e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 >> 172.21.15.66:0/3071471619 conn(0x55c452434a50 msgr2=0x55c452438640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 shutdown_connections 2024-03-30T13:12:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 1 -- 172.21.15.66:0/3071471619 wait complete. 2024-03-30T13:12:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.254+0000 7fd683322740 20 client.5967 trim_cache size 0 max 16384 2024-03-30T13:12:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.269+0000 7fd683322740 1 Processor -- start 2024-03-30T13:12:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.269+0000 7fd683322740 1 -- start start 2024-03-30T13:12:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd683322740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c452433590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452433ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45246b110 0x55c452439630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd683322740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c2012d0 con 0x55c452461570 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c200310 con 0x55c45246b110 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c150bb0 con 0x55c452461950 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c452433590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c452433590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57698/0 (socket says 172.21.15.66:57698) 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 -- 172.21.15.66:0/2715070841 learned_addr learned my addr 172.21.15.66:0/2715070841 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 -- 172.21.15.66:0/2715070841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 msgr2=0x55c452433ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452433ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 -- 172.21.15.66:0/2715070841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45246b110 msgr2=0x55c452439630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45246b110 0x55c452439630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 -- 172.21.15.66:0/2715070841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45204b2d0 con 0x55c452461570 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd67a59c640 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c452433590 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00fe00 tx=0x7fd66c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20eaddc0b35e4035 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd65b7fe640 1 -- 172.21.15.66:0/2715070841 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c039070 con 0x55c452461570 2024-03-30T13:12:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd65b7fe640 1 -- 172.21.15.66:0/2715070841 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66c01c040 con 0x55c452461570 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.270+0000 7fd65b7fe640 1 -- 172.21.15.66:0/2715070841 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c025070 con 0x55c452461570 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 -- 172.21.15.66:0/2715070841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 msgr2=0x55c452433590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c452433590 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00fe00 tx=0x7fd66c022000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 -- 172.21.15.66:0/2715070841 shutdown_connections 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c45246b110 0x55c452439630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c452433ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 --2- 172.21.15.66:0/2715070841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c452461570 0x55c452433590 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 -- 172.21.15.66:0/2715070841 >> 172.21.15.66:0/2715070841 conn(0x55c452434a50 msgr2=0x55c4524a3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 -- 172.21.15.66:0/2715070841 shutdown_connections 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 -- 172.21.15.66:0/2715070841 wait complete. 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.271+0000 7fd683322740 1 -- start start 2024-03-30T13:12:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 Processor -- start 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c4524fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c45248e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 0x55c4524807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c452437d90 con 0x55c45246b110 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c1ffcf0 con 0x55c452461570 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.272+0000 7fd683322740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c4524357e0 con 0x55c452461950 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd683322740 10 client.0 _ll_register_callbacks cb 0x7fd66c00fed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 0x55c4524807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 0x55c4524807b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57702/0 (socket says 172.21.15.66:57702) 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 -- 172.21.15.66:0/37946728 learned_addr learned my addr 172.21.15.66:0/37946728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd679d9b640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c45248e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67a59c640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c4524fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 -- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 msgr2=0x55c45248e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c45248e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 -- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 msgr2=0x55c4524fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c4524fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 -- 172.21.15.66:0/37946728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c452460590 con 0x55c45246b110 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd67ad9d640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 0x55c4524807b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd674013000 tx=0x7fd6740014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e461206ab7ad9aae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd6597fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd6597fa640 1 -- 172.21.15.66:0/37946728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674019070 con 0x55c45246b110 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd6597fa640 1 -- 172.21.15.66:0/37946728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd674004d70 con 0x55c45246b110 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.273+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66c01c040 con 0x55c45246b110 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd6597fa640 1 -- 172.21.15.66:0/37946728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fd67402c050 con 0x55c45246b110 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 10 client.5973 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55c452439080 con 0x55c45246b110 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd6597fa640 1 -- 172.21.15.66:0/37946728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66c01c040 con 0x55c45246b110 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd6597fa640 1 -- 172.21.15.66:0/37946728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fd67402f020 con 0x55c45246b110 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 10 client.5973 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 10 client.5973 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 20 client.5973 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c452438620 0x55c452438a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 4 client.5973 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628348}] 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628348}) -- 0x55c4523a5e20 con 0x55c452438620 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd67a59c640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c452438620 0x55c452438a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd67a59c640 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c452438620 0x55c452438a30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1229f368b81955 server_cookie=7552b877be507b47 in_seq=0 out_seq=0 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.274+0000 7fd6597fa640 10 client.5973 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd6597fa640 1 -- 172.21.15.66:0/37946728 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4338 (crc 0 0 0) 0x55c4523a5e20 con 0x55c452438620 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd6597fa640 10 client.5973 handle_command_reply: tid=0 2024-03-30T13:12:00.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 client.5973 shutdown 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 msgr2=0x55c4524807b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 0x55c4524807b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd674013000 tx=0x7fd6740014a0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 shutdown_connections 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c452438620 0x55c452438a30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c45246b110 0x55c4524807b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c452461950 0x55c45248e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd6597fa640 0 client.5973 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 --2- 172.21.15.66:0/37946728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c452461570 0x55c4524fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 >> 172.21.15.66:0/37946728 conn(0x55c452434a50 msgr2=0x55c4524a3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 shutdown_connections 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 1 -- 172.21.15.66:0/37946728 wait complete. 2024-03-30T13:12:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.275+0000 7fd683322740 20 client.5973 trim_cache size 0 max 16384 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-sysfs.h", 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628348, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.906932+0000", 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.466202+0000", 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:00.285 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 77, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:00.286 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.466202+0000", 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.287 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.288 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 77, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.906932+0000" 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 77, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.906932+0000" 2024-03-30T13:12:00.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 92, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 42, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:12:00.290 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:00.291 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:12:00.292 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.293 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.294 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:00.296 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:00.297 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:00.298 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:00.298 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:00.298 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd65c06c560 msgr2=0x7fd65c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd65c06c560 0x7fd65c06ea20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd66c006fd0 tx=0x7fd66c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 msgr2=0x7fd67c1553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 0x7fd67c1553b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd67002b9e0 tx=0x7fd6700046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 shutdown_connections 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd65c06c560 0x7fd65c06ea20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd67c158e00 0x7fd67c0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd67c158a20 0x7fd67c1553b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 --2- 172.21.15.66:0/3587504899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd67c0b0030 0x7fd67c149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 >> 172.21.15.66:0/3587504899 conn(0x7fd67c0d1160 msgr2=0x7fd67c06be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 shutdown_connections 2024-03-30T13:12:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.276+0000 7fd680a93640 1 -- 172.21.15.66:0/3587504899 wait complete. 2024-03-30T13:12:00.300 DEBUG:tasks.cephfs.test_uninlining:getting inode info #17 of 23:1099511628331 2024-03-30T13:12:00.301 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628331 2024-03-30T13:12:00.301 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628331 2024-03-30T13:12:00.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.407+0000 7f6a482d9640 1 Processor -- start 2024-03-30T13:12:00.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.407+0000 7f6a482d9640 1 -- start start 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a482d9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6a400ba090 0x7f6a400d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a482d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a482d9640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a400d2ff0 con 0x7f6a400b5dc0 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a482d9640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6a400d32d0 con 0x7f6a400ba090 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a482d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400d35b0 con 0x7f6a400b73d0 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45874640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6a400ba090 0x7f6a400d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a46075640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6a400b5dc0 0x7f6a400b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37532/0 (socket says 172.21.15.66:37532) 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a46075640 1 -- 172.21.15.66:0/600593747 learned_addr learned my addr 172.21.15.66:0/600593747 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a46876640 1 --2- 172.21.15.66:0/600593747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45073640 1 -- 172.21.15.66:0/600593747 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614818667 0 0) 0x7f6a400d2ff0 con 0x7f6a400b5dc0 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45073640 1 -- 172.21.15.66:0/600593747 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a2c003520 con 0x7f6a400b5dc0 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45874640 1 -- 172.21.15.66:0/600593747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 msgr2=0x7f6a400b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45874640 1 --2- 172.21.15.66:0/600593747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45874640 1 -- 172.21.15.66:0/600593747 >> v1:172.21.15.66:6789/0 conn(0x7f6a400b5dc0 legacy=0x7f6a400b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.408+0000 7f6a45874640 1 -- 172.21.15.66:0/600593747 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a400d3890 con 0x7f6a400ba090 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.409+0000 7f6a45874640 1 --2- 172.21.15.66:0/600593747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400ba090 0x7f6a400d2680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097d0 tx=0x7f6a3002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b801ccd41c587b37 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.409+0000 7f6a45073640 1 -- 172.21.15.66:0/600593747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30035020 con 0x7f6a400ba090 2024-03-30T13:12:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.409+0000 7f6a45073640 1 -- 172.21.15.66:0/600593747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a30031800 con 0x7f6a400ba090 2024-03-30T13:12:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.409+0000 7f6a45073640 1 -- 172.21.15.66:0/600593747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30031bb0 con 0x7f6a400ba090 2024-03-30T13:12:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 -- 172.21.15.66:0/600593747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400ba090 msgr2=0x7f6a400d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 --2- 172.21.15.66:0/600593747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400ba090 0x7f6a400d2680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097d0 tx=0x7f6a3002b8a0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 -- 172.21.15.66:0/600593747 shutdown_connections 2024-03-30T13:12:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 --2- 172.21.15.66:0/600593747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 --2- 172.21.15.66:0/600593747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400ba090 0x7f6a400d2680 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 -- 172.21.15.66:0/600593747 >> 172.21.15.66:0/600593747 conn(0x7f6a400678f0 msgr2=0x7f6a400d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 -- 172.21.15.66:0/600593747 shutdown_connections 2024-03-30T13:12:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.410+0000 7f6a482d9640 1 -- 172.21.15.66:0/600593747 wait complete. 2024-03-30T13:12:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.411+0000 7f6a482d9640 1 Processor -- start 2024-03-30T13:12:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.411+0000 7f6a482d9640 1 -- start start 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a482d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 0x7f6a400cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a482d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a482d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a400ba090 0x7f6a40157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a482d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4015e880 con 0x7f6a400b5dc0 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a482d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400d4090 con 0x7f6a400ba090 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a482d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4015ea00 con 0x7f6a400b73d0 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 0x7f6a400cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 0x7f6a400cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57730/0 (socket says 172.21.15.66:57730) 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 -- 172.21.15.66:0/2526783681 learned_addr learned my addr 172.21.15.66:0/2526783681 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46876640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a400ba090 0x7f6a40157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a45874640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 -- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 msgr2=0x7f6a400cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 -- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a400ba090 msgr2=0x7f6a40157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a400ba090 0x7f6a40157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 -- 172.21.15.66:0/2526783681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a400ce250 con 0x7f6a400b5dc0 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.412+0000 7f6a46075640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 0x7f6a400cbdd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c430 tx=0x7f6a3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.413+0000 7f6a36ffd640 1 -- 172.21.15.66:0/2526783681 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c017070 con 0x7f6a400b5dc0 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.413+0000 7f6a36ffd640 1 -- 172.21.15.66:0/2526783681 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c005ce0 con 0x7f6a400b5dc0 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.413+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a400ccd50 con 0x7f6a400b5dc0 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.413+0000 7f6a36ffd640 1 -- 172.21.15.66:0/2526783681 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0072f0 con 0x7f6a400b5dc0 2024-03-30T13:12:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.413+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a400cc790 con 0x7f6a400b5dc0 2024-03-30T13:12:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.414+0000 7f6a36ffd640 1 -- 172.21.15.66:0/2526783681 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6a3c01a070 con 0x7f6a400b5dc0 2024-03-30T13:12:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.414+0000 7f6a36ffd640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6a2406c530 0x7f6a2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.415+0000 7f6a36ffd640 1 -- 172.21.15.66:0/2526783681 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f6a3c08ad00 con 0x7f6a400b5dc0 2024-03-30T13:12:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.415+0000 7f6a45874640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6a2406c530 0x7f6a2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.415+0000 7f6a45874640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6a2406c530 0x7f6a2406e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6a30036000 tx=0x7f6a30008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 Processor -- start 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 -- start start 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea8023570 0x555ea7ff6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8026300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea7ff6a50 0x555ea8026840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ea7f60590 con 0x555ea8023950 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ea7efed30 con 0x555ea8023570 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.416+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555ea7f5c540 con 0x555ea7ff6a50 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8026300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8026300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57746/0 (socket says 172.21.15.66:57746) 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 -- 172.21.15.66:0/3374083623 learned_addr learned my addr 172.21.15.66:0/3374083623 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a46876640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea7ff6a50 0x555ea8026840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a46075640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea8023570 0x555ea7ff6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 -- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea7ff6a50 msgr2=0x555ea8026840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea7ff6a50 0x555ea8026840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 -- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea8023570 msgr2=0x555ea7ff6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea8023570 0x555ea7ff6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 -- 172.21.15.66:0/3374083623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea7d40920 con 0x555ea8023950 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a46075640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea8023570 0x555ea7ff6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a46876640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea7ff6a50 0x555ea8026840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a45874640 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8026300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6a3002beb0 tx=0x7f6a30030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1f4d82c5b8ecf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a23fff640 1 -- 172.21.15.66:0/3374083623 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3004b070 con 0x555ea8023950 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a23fff640 1 -- 172.21.15.66:0/3374083623 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a30031600 con 0x555ea8023950 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.417+0000 7f6a23fff640 1 -- 172.21.15.66:0/3374083623 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30035040 con 0x555ea8023950 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3374083623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 msgr2=0x555ea8026300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8026300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6a3002beb0 tx=0x7f6a30030000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3374083623 shutdown_connections 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea7ff6a50 0x555ea8026840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8026300 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3374083623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea8023570 0x555ea7ff6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3374083623 >> 172.21.15.66:0/3374083623 conn(0x555ea7ff8780 msgr2=0x555ea8041300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3374083623 shutdown_connections 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.418+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3374083623 wait complete. 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.419+0000 7f6a4afa4740 1 -- start start 2024-03-30T13:12:00.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.419+0000 7f6a4afa4740 1 Processor -- start 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea802a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea80551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ea7ffae20 con 0x555ea8023950 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ea7efc9f0 con 0x555ea801bcd0 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555ea7ffd660 con 0x555ea8023570 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a4afa4740 10 client.0 _ll_register_callbacks cb 0x555ea7f00c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea80551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea80551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57762/0 (socket says 172.21.15.66:57762) 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 -- 172.21.15.66:0/2641718892 learned_addr learned my addr 172.21.15.66:0/2641718892 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a46075640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea802a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a46876640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 -- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 msgr2=0x555ea802a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea802a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 -- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 msgr2=0x555ea80556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 -- 172.21.15.66:0/2641718892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea7c34210 con 0x555ea8023950 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a46075640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea802a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a45874640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea80551a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000c00 tx=0x7f6a3002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df22b7994405fa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.420+0000 7f6a217fa640 1 -- 172.21.15.66:0/2641718892 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30035070 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a217fa640 1 -- 172.21.15.66:0/2641718892 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f6a30050870 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a30031600 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a217fa640 1 -- 172.21.15.66:0/2641718892 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6a3004b040 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 10 client.5988 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x555ea7da10b0 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a217fa640 1 -- 172.21.15.66:0/2641718892 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a30031600 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a217fa640 1 -- 172.21.15.66:0/2641718892 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f6a30063080 con 0x555ea8023950 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 10 client.5988 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 10 client.5988 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 20 client.5988 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea8025610 0x555ea8025a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 4 client.5988 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:00.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea7f6dad0 con 0x555ea8025610 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a46075640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea8025610 0x555ea8025a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a46075640 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea8025610 0x555ea8025a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8c363a8a4aadad server_cookie=af4892df4e259418 in_seq=0 out_seq=0 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.421+0000 7f6a217fa640 10 client.5988 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a217fa640 1 -- 172.21.15.66:0/2641718892 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x555ea7f6dad0 con 0x555ea8025610 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a217fa640 10 client.5988 handle_command_reply: tid=0 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 client.5988 shutdown 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 msgr2=0x555ea80551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea80551a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000c00 tx=0x7f6a3002e000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 shutdown_connections 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea8025610 0x555ea8025a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea80551a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/2641718892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea802a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a217fa640 0 client.5988 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 >> 172.21.15.66:0/2641718892 conn(0x555ea7ff6a50 msgr2=0x555ea7ffa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 shutdown_connections 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 1 -- 172.21.15.66:0/2641718892 wait complete. 2024-03-30T13:12:00.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.422+0000 7f6a4afa4740 20 client.5988 trim_cache size 0 max 16384 2024-03-30T13:12:00.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.437+0000 7f6a4afa4740 1 Processor -- start 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.437+0000 7f6a4afa4740 1 -- start start 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea803fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea80265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8027380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a40200f30 con 0x555ea8023950 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a401fff70 con 0x555ea801bcd0 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f6a401ff2d0 con 0x555ea8023570 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8027380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46075640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea803fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8027380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57768/0 (socket says 172.21.15.66:57768) 2024-03-30T13:12:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a45874640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea80265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 -- 172.21.15.66:0/583149658 learned_addr learned my addr 172.21.15.66:0/583149658 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 -- 172.21.15.66:0/583149658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 msgr2=0x555ea80265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea80265a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 -- 172.21.15.66:0/583149658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 msgr2=0x555ea803fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea803fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 -- 172.21.15.66:0/583149658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea802a3f0 con 0x555ea8023950 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a46876640 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8027380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009d60 tx=0x7f6a38007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe6df19bce8e905 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a237fe640 1 -- 172.21.15.66:0/583149658 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3801b020 con 0x555ea8023950 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a237fe640 1 -- 172.21.15.66:0/583149658 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38007e10 con 0x555ea8023950 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.438+0000 7f6a237fe640 1 -- 172.21.15.66:0/583149658 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38017680 con 0x555ea8023950 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 -- 172.21.15.66:0/583149658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 msgr2=0x555ea8027380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8027380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009d60 tx=0x7f6a38007700 comp rx=0 tx=0).stop 2024-03-30T13:12:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 -- 172.21.15.66:0/583149658 shutdown_connections 2024-03-30T13:12:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8027380 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea80265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/583149658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea803fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 -- 172.21.15.66:0/583149658 >> 172.21.15.66:0/583149658 conn(0x555ea7ff6a50 msgr2=0x555ea7ff1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 -- 172.21.15.66:0/583149658 shutdown_connections 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.439+0000 7f6a4afa4740 1 -- 172.21.15.66:0/583149658 wait complete. 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.440+0000 7f6a4afa4740 1 -- start start 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.440+0000 7f6a4afa4740 1 Processor -- start 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea8022350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8022890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a401ff920 con 0x555ea8023950 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ea7ffd280 con 0x555ea801bcd0 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555ea8024cb0 con 0x555ea8023570 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 10 client.0 _ll_register_callbacks cb 0x7f6a30009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8022890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8022890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57780/0 (socket says 172.21.15.66:57780) 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 -- 172.21.15.66:0/3098790620 learned_addr learned my addr 172.21.15.66:0/3098790620 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46075640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a45874640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea8022350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 -- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 msgr2=0x555ea8022350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea8022350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 -- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 msgr2=0x555ea80c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 -- 172.21.15.66:0/3098790620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea7c0d2d0 con 0x555ea8023950 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a45874640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea8022350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a46876640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8022890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6a380179e0 tx=0x7f6a38004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8d363c6393f81c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a217fa640 1 -- 172.21.15.66:0/3098790620 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38017dc0 con 0x555ea8023950 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a217fa640 1 -- 172.21.15.66:0/3098790620 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f6a3802c480 con 0x555ea8023950 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.441+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a38007e10 con 0x555ea8023950 2024-03-30T13:12:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a217fa640 1 -- 172.21.15.66:0/3098790620 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f6a3802c680 con 0x555ea8023950 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 10 client.5994 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x555ea7ff1be0 con 0x555ea8023950 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a217fa640 1 -- 172.21.15.66:0/3098790620 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38007e10 con 0x555ea8023950 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a217fa640 1 -- 172.21.15.66:0/3098790620 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f6a3802c820 con 0x555ea8023950 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 10 client.5994 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 10 client.5994 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 20 client.5994 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea80296e0 0x555ea8023d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 4 client.5994 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628331}] 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628331}) -- 0x555ea7f67e20 con 0x555ea80296e0 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a46075640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea80296e0 0x555ea8023d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a217fa640 10 client.5994 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.442+0000 7f6a46075640 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea80296e0 0x555ea8023d90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc90c03734371738 server_cookie=6c47d1748cf2e07d in_seq=0 out_seq=0 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a217fa640 1 -- 172.21.15.66:0/3098790620 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4335 (crc 0 0 0) 0x555ea7f67e20 con 0x555ea80296e0 2024-03-30T13:12:00.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a217fa640 10 client.5994 handle_command_reply: tid=0 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 client.5994 shutdown 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 msgr2=0x555ea8022890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8022890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6a380179e0 tx=0x7f6a38004ab0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 shutdown_connections 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x555ea80296e0 0x555ea8023d90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ea8023950 0x555ea8022890 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555ea8023570 0x555ea8022350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a217fa640 0 client.5994 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 --2- 172.21.15.66:0/3098790620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ea801bcd0 0x555ea80c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 >> 172.21.15.66:0/3098790620 conn(0x555ea7ff6a50 msgr2=0x555ea8027280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 shutdown_connections 2024-03-30T13:12:00.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 1 -- 172.21.15.66:0/3098790620 wait complete. 2024-03-30T13:12:00.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.443+0000 7f6a4afa4740 20 client.5994 trim_cache size 0 max 16384 2024-03-30T13:12:00.459 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/dcache.c", 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628331, 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.910789+0000", 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.449202+0000", 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:00.460 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:00.461 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 3670, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:00.462 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.449202+0000", 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 3, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [ 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "client": 5120, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "byte range": { 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "first": 0, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "last": 4194304 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.463 INFO:teuthology.orchestra.run.smithi066.stdout: "follows": 1 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3670, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.464 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.910789+0000" 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3670, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.910789+0000" 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 96, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 8, 2024-03-30T13:12:00.465 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 0, 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:00.466 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:00.467 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 1, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 1, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 1, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:00.468 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 1, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 6, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.469 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": { 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "excl", 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "ifile", 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 1, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:00.470 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.471 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:00.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:00.473 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:00.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6a2406c530 msgr2=0x7f6a2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6a2406c530 0x7f6a2406e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6a30036000 tx=0x7f6a30008030 comp rx=0 tx=0).stop 2024-03-30T13:12:00.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 msgr2=0x7f6a400cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 0x7f6a400cbdd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c430 tx=0x7f6a3c00c8f0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 shutdown_connections 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f6a2406c530 0x7f6a2406e9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a400ba090 0x7f6a40157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a400b73d0 0x7f6a400cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 --2- 172.21.15.66:0/2526783681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a400b5dc0 0x7f6a400cbdd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 >> 172.21.15.66:0/2526783681 conn(0x7f6a400678f0 msgr2=0x7f6a400b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 shutdown_connections 2024-03-30T13:12:00.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.444+0000 7f6a482d9640 1 -- 172.21.15.66:0/2526783681 wait complete. 2024-03-30T13:12:00.478 DEBUG:tasks.cephfs.test_uninlining:getting inode info #18 of 23:1099511628339 2024-03-30T13:12:00.478 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628339 2024-03-30T13:12:00.479 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628339 2024-03-30T13:12:00.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.603+0000 7fda3c631640 1 Processor -- start 2024-03-30T13:12:00.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.603+0000 7fda3c631640 1 -- start start 2024-03-30T13:12:00.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.604+0000 7fda3c631640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fda340b5dc0 0x7fda340b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.604+0000 7fda3c631640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda340b66e0 con 0x7fda340b73d0 2024-03-30T13:12:00.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.604+0000 7fda3c631640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda340d2ff0 con 0x7fda340ba090 2024-03-30T13:12:00.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.604+0000 7fda3c631640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fda340d2d90 con 0x7fda340b5dc0 2024-03-30T13:12:00.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.604+0000 7fda3abce640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fda340b73d0 0x7fda340b7800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37546/0 (socket says 172.21.15.66:37546) 2024-03-30T13:12:00.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.604+0000 7fda3abce640 1 -- 172.21.15.66:0/401240023 learned_addr learned my addr 172.21.15.66:0/401240023 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:00.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/401240023 >> v2:172.21.15.117:3300/0 conn(0x7fda340b5dc0 0x7fda340b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988144735 0 0) 0x7fda340b66e0 con 0x7fda340b73d0 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda20003520 con 0x7fda340b73d0 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766526962 0 0) 0x7fda340d2ff0 con 0x7fda340ba090 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda340b66e0 con 0x7fda340ba090 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2759885519 0 0) 0x7fda20003520 con 0x7fda340b73d0 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 >> v2:172.21.15.117:3300/0 conn(0x7fda340b5dc0 msgr2=0x7fda340b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 --2- 172.21.15.66:0/401240023 >> v2:172.21.15.117:3300/0 conn(0x7fda340b5dc0 0x7fda340b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 >> v1:172.21.15.117:6789/0 conn(0x7fda340ba090 legacy=0x7fda340d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda340d32d0 con 0x7fda340b73d0 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7fda2c0019f0 con 0x7fda340b73d0 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/401240023 >> v2:172.21.15.117:3300/0 conn(0x7fda340b5dc0 0x7fda340b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 >> v1:172.21.15.66:6789/0 conn(0x7fda340b73d0 legacy=0x7fda340b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda20003b40 0x7fda20003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20004800 0x7fda20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda200051c0 0x7fda2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda340d2d90 con 0x7fda20003b40 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda2000c040 con 0x7fda200051c0 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fda2000c240 con 0x7fda20004800 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda20003b40 0x7fda20003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda3abce640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda200051c0 0x7fda2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.605+0000 7fda39bcc640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20004800 0x7fda20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3a3cd640 1 -- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20004800 msgr2=0x7fda20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20004800 0x7fda20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3a3cd640 1 -- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda200051c0 msgr2=0x7fda2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda200051c0 0x7fda2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3abce640 1 -- 172.21.15.66:0/401240023 reap_dead start 2024-03-30T13:12:00.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3a3cd640 1 -- 172.21.15.66:0/401240023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda20003520 con 0x7fda20003b40 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda39bcc640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20004800 0x7fda20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda20003b40 0x7fda20003f40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fda30009c90 tx=0x7fda20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a8c8d0d58f76ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.606+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30039070 con 0x7fda20003b40 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3002c8f0 con 0x7fda20003b40 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda393cb640 1 -- 172.21.15.66:0/401240023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30009860 con 0x7fda20003b40 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 -- 172.21.15.66:0/401240023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda20003b40 msgr2=0x7fda20003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda20003b40 0x7fda20003f40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fda30009c90 tx=0x7fda20005830 comp rx=0 tx=0).stop 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 -- 172.21.15.66:0/401240023 shutdown_connections 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 --2- 172.21.15.66:0/401240023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda20003b40 0x7fda20003f40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 -- 172.21.15.66:0/401240023 >> 172.21.15.66:0/401240023 conn(0x7fda340678f0 msgr2=0x7fda340d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 -- 172.21.15.66:0/401240023 shutdown_connections 2024-03-30T13:12:00.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.607+0000 7fda3c631640 1 -- 172.21.15.66:0/401240023 wait complete. 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.608+0000 7fda3c631640 1 Processor -- start 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.608+0000 7fda3c631640 1 -- start start 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3c631640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20003b40 0x7fda34152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3c631640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda34152bb0 0x7fda34160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3c631640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 0x7fda341610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3c631640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda340d3b70 con 0x7fda34160c50 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3c631640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda340d3890 con 0x7fda34152bb0 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3c631640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fda340d4090 con 0x7fda20003b40 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3abce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 0x7fda341610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3abce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 0x7fda341610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57786/0 (socket says 172.21.15.66:57786) 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3abce640 1 -- 172.21.15.66:0/2364062151 learned_addr learned my addr 172.21.15.66:0/2364062151 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20003b40 0x7fda34152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.609+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda34152bb0 0x7fda34160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3abce640 1 -- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20003b40 msgr2=0x7fda34152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3abce640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20003b40 0x7fda34152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3abce640 1 -- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda34152bb0 msgr2=0x7fda34160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3abce640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda34152bb0 0x7fda34160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3abce640 1 -- 172.21.15.66:0/2364062151 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341528b0 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3abce640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 0x7fda341610d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fda3408d7e0 tx=0x7fda2c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda26ffd640 1 -- 172.21.15.66:0/2364062151 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00ab30 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda26ffd640 1 -- 172.21.15.66:0/2364062151 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c019070 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3415f790 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda26ffd640 1 -- 172.21.15.66:0/2364062151 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0133f0 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.610+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3415fd50 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.612+0000 7fda26ffd640 1 -- 172.21.15.66:0/2364062151 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fda2c013590 con 0x7fda34160c50 2024-03-30T13:12:00.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.612+0000 7fda26ffd640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fda1806c560 0x7fda1806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.612+0000 7fda26ffd640 1 -- 172.21.15.66:0/2364062151 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fda2c08bcf0 con 0x7fda34160c50 2024-03-30T13:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.612+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fda1806c560 0x7fda1806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.613+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fda1806c560 0x7fda1806ea20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda30036000 tx=0x7fda30030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.613+0000 7fda3f218740 1 Processor -- start 2024-03-30T13:12:00.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.613+0000 7fda3f218740 1 -- start start 2024-03-30T13:12:00.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3f218740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x558572198510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558572198a50 0x5585721c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3f218740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558572102590 con 0x5585721c5570 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5585720a0d30 con 0x558572198a50 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5585720fe540 con 0x5585721c5950 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x558572198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x558572198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57798/0 (socket says 172.21.15.66:57798) 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 -- 172.21.15.66:0/2207032762 learned_addr learned my addr 172.21.15.66:0/2207032762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3abce640 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558572198a50 0x5585721c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 -- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 msgr2=0x5585721c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 -- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558572198a50 msgr2=0x5585721c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558572198a50 0x5585721c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda3a3cd640 1 -- 172.21.15.66:0/2207032762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558571ee2920 con 0x5585721c5570 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.614+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x558572198510 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fda3002c4a0 tx=0x7fda3002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92c16ccf67011c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda17fff640 1 -- 172.21.15.66:0/2207032762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3004f070 con 0x5585721c5570 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda17fff640 1 -- 172.21.15.66:0/2207032762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3003f7c0 con 0x5585721c5570 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda17fff640 1 -- 172.21.15.66:0/2207032762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30039040 con 0x5585721c5570 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 -- 172.21.15.66:0/2207032762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 msgr2=0x558572198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x558572198510 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fda3002c4a0 tx=0x7fda3002e000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 -- 172.21.15.66:0/2207032762 shutdown_connections 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558572198a50 0x5585721c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 --2- 172.21.15.66:0/2207032762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x558572198510 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.615+0000 7fda3f218740 1 -- 172.21.15.66:0/2207032762 >> 172.21.15.66:0/2207032762 conn(0x55857219a780 msgr2=0x5585721e3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.616+0000 7fda3f218740 1 -- 172.21.15.66:0/2207032762 shutdown_connections 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.616+0000 7fda3f218740 1 -- 172.21.15.66:0/2207032762 wait complete. 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.616+0000 7fda3f218740 1 -- start start 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.616+0000 7fda3f218740 1 Processor -- start 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5570 0x5585721f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 0x5585721cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558572102590 con 0x5585721c3420 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55857209e9f0 con 0x5585721c5570 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55857219ce20 con 0x5585721c5950 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3f218740 10 client.0 _ll_register_callbacks cb 0x5585720a2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 0x5585721cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 0x5585721cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57810/0 (socket says 172.21.15.66:57810) 2024-03-30T13:12:00.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 -- 172.21.15.66:0/2011691452 learned_addr learned my addr 172.21.15.66:0/2011691452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5570 0x5585721f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 -- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 msgr2=0x5585721ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 -- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5570 msgr2=0x5585721f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5570 0x5585721f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda3abce640 1 -- 172.21.15.66:0/2011691452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558571dd6210 con 0x5585721c3420 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.617+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3abce640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 0x5585721cebc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001b60 tx=0x7fda2c05a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6c640da28b38a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda157fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda157fa640 1 -- 172.21.15.66:0/2011691452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c088060 con 0x5585721c3420 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda157fa640 1 -- 172.21.15.66:0/2011691452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fda2c0623b0 con 0x5585721c3420 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3003f7c0 con 0x5585721c3420 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda157fa640 1 -- 172.21.15.66:0/2011691452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fda2c062640 con 0x5585721c3420 2024-03-30T13:12:00.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 10 client.6006 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558571f430b0 con 0x5585721c3420 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda157fa640 1 -- 172.21.15.66:0/2011691452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3003f7c0 con 0x5585721c3420 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda157fa640 1 -- 172.21.15.66:0/2011691452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fda2c060070 con 0x5585721c3420 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 10 client.6006 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 10 client.6006 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 20 client.6006 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721f33f0 0x5585721f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 4 client.6006 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55857210fad0 con 0x5585721f33f0 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.618+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721f33f0 0x5585721f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda157fa640 10 client.6006 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721f33f0 0x5585721f37d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1203e9e9360ee60 server_cookie=377ebfdce6d2491b in_seq=0 out_seq=0 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda157fa640 1 -- 172.21.15.66:0/2011691452 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55857210fad0 con 0x5585721f33f0 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda157fa640 10 client.6006 handle_command_reply: tid=0 2024-03-30T13:12:00.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda3f218740 1 client.6006 shutdown 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 msgr2=0x5585721cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.619+0000 7fda3f218740 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 0x5585721cebc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001b60 tx=0x7fda2c05a220 comp rx=0 tx=0).stop 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 shutdown_connections 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721f33f0 0x5585721f37d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c3420 0x5585721cebc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda157fa640 0 client.6006 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 --2- 172.21.15.66:0/2011691452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5570 0x5585721f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 >> 172.21.15.66:0/2011691452 conn(0x558572198a50 msgr2=0x55857219c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 shutdown_connections 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 1 -- 172.21.15.66:0/2011691452 wait complete. 2024-03-30T13:12:00.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.620+0000 7fda3f218740 20 client.6006 trim_cache size 0 max 16384 2024-03-30T13:12:00.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 Processor -- start 2024-03-30T13:12:00.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 -- start start 2024-03-30T13:12:00.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c3420 0x5585721c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5950 0x5585721e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda342019d0 con 0x5585721c5570 2024-03-30T13:12:00.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda34200a30 con 0x5585721c5950 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fda341ffd90 con 0x5585721c3420 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda39bcc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda39bcc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57818/0 (socket says 172.21.15.66:57818) 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.635+0000 7fda39bcc640 1 -- 172.21.15.66:0/503034261 learned_addr learned my addr 172.21.15.66:0/503034261 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda3abce640 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5950 0x5585721e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda39bcc640 1 -- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c3420 msgr2=0x5585721c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda39bcc640 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c3420 0x5585721c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda39bcc640 1 -- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5950 msgr2=0x5585721e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda39bcc640 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5950 0x5585721e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda39bcc640 1 -- 172.21.15.66:0/503034261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585721cdeb0 con 0x5585721c5570 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda3abce640 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5950 0x5585721e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda39bcc640 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721c7d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fda28001480 tx=0x7fda28001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56e7070d578c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda177fe640 1 -- 172.21.15.66:0/503034261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28005110 con 0x5585721c5570 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda177fe640 1 -- 172.21.15.66:0/503034261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2801b070 con 0x5585721c5570 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.636+0000 7fda177fe640 1 -- 172.21.15.66:0/503034261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28007030 con 0x5585721c5570 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 -- 172.21.15.66:0/503034261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 msgr2=0x5585721c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721c7d40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fda28001480 tx=0x7fda28001940 comp rx=0 tx=0).stop 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 -- 172.21.15.66:0/503034261 shutdown_connections 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c5950 0x5585721e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721c7d40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 --2- 172.21.15.66:0/503034261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c3420 0x5585721c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 -- 172.21.15.66:0/503034261 >> 172.21.15.66:0/503034261 conn(0x558572198a50 msgr2=0x55857219c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 -- 172.21.15.66:0/503034261 shutdown_connections 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.637+0000 7fda3f218740 1 -- 172.21.15.66:0/503034261 wait complete. 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.638+0000 7fda3f218740 1 -- start start 2024-03-30T13:12:00.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.638+0000 7fda3f218740 1 Processor -- start 2024-03-30T13:12:00.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c3420 0x5585721f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5585721cffd0 con 0x5585721c5570 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda342003e0 con 0x5585721c3420 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5585721997e0 con 0x5585721c5950 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3f218740 10 client.0 _ll_register_callbacks cb 0x7fda2c0598d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda39bcc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda39bcc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57824/0 (socket says 172.21.15.66:57824) 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda39bcc640 1 -- 172.21.15.66:0/2223024077 learned_addr learned my addr 172.21.15.66:0/2223024077 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.639+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c3420 0x5585721f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3abce640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda39bcc640 1 -- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 msgr2=0x5585721973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda39bcc640 1 -- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c3420 msgr2=0x5585721f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c3420 0x5585721f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda39bcc640 1 -- 172.21.15.66:0/2223024077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558571daf2d0 con 0x5585721c5570 2024-03-30T13:12:00.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda39bcc640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721f1e10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fda34050b60 tx=0x7fda280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff148cdd4068cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda157fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda157fa640 1 -- 172.21.15.66:0/2223024077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda280069d0 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda157fa640 1 -- 172.21.15.66:0/2223024077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fda28006b70 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2801b070 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda157fa640 1 -- 172.21.15.66:0/2223024077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fda28006dd0 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3f218740 10 client.6012 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5585721c3b10 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda157fa640 1 -- 172.21.15.66:0/2223024077 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2801b070 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda157fa640 1 -- 172.21.15.66:0/2223024077 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fda280035f0 con 0x5585721c5570 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3f218740 10 client.6012 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3f218740 10 client.6012 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.640+0000 7fda3f218740 20 client.6012 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3f218740 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721c4ac0 0x5585721978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3f218740 4 client.6012 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628339}] 2024-03-30T13:12:00.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628339}) -- 0x558572109e20 con 0x5585721c4ac0 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721c4ac0 0x5585721978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda157fa640 10 client.6012 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3a3cd640 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721c4ac0 0x5585721978e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ba6a0b142a1a79 server_cookie=4fa7d3b1e7daad0 in_seq=0 out_seq=0 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda157fa640 1 -- 172.21.15.66:0/2223024077 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3977 (crc 0 0 0) 0x558572109e20 con 0x5585721c4ac0 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda157fa640 10 client.6012 handle_command_reply: tid=0 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3f218740 1 client.6012 shutdown 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 msgr2=0x5585721f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.641+0000 7fda3f218740 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721f1e10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fda34050b60 tx=0x7fda280012d0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 shutdown_connections 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5585721c4ac0 0x5585721978e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5585721c5950 0x5585721973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585721c5570 0x5585721f1e10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda157fa640 0 client.6012 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 --2- 172.21.15.66:0/2223024077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585721c3420 0x5585721f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 >> 172.21.15.66:0/2223024077 conn(0x558572198a50 msgr2=0x5585721e2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 shutdown_connections 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 1 -- 172.21.15.66:0/2223024077 wait complete. 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3f218740 20 client.6012 trim_cache size 0 max 16384 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-bufmap.h", 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628339, 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.911447+0000", 2024-03-30T13:12:00.651 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.458202+0000", 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:00.652 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 925, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.458202+0000", 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:00.653 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 925, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.654 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.911447+0000" 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 925, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.911447+0000" 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 176, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:00.655 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 24, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:11:25.949130+0000", 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 66, 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.656 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:00.657 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:00.658 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.659 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.660 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.662 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.663 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:00.663 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:00.663 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:00.663 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:00.663 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fda1806c560 msgr2=0x7fda1806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fda1806c560 0x7fda1806ea20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fda30036000 tx=0x7fda30030000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 msgr2=0x7fda341610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 0x7fda341610d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fda3408d7e0 tx=0x7fda2c007b00 comp rx=0 tx=0).stop 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 shutdown_connections 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fda1806c560 0x7fda1806ea20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda34160c50 0x7fda341610d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda34152bb0 0x7fda34160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 --2- 172.21.15.66:0/2364062151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda20003b40 0x7fda34152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 >> 172.21.15.66:0/2364062151 conn(0x7fda340678f0 msgr2=0x7fda340b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 shutdown_connections 2024-03-30T13:12:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.642+0000 7fda3c631640 1 -- 172.21.15.66:0/2364062151 wait complete. 2024-03-30T13:12:00.665 DEBUG:tasks.cephfs.test_uninlining:getting inode info #19 of 23:1099511628343 2024-03-30T13:12:00.666 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628343 2024-03-30T13:12:00.666 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628343 2024-03-30T13:12:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.779+0000 7feb3044e640 1 Processor -- start 2024-03-30T13:12:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.779+0000 7feb3044e640 1 -- start start 2024-03-30T13:12:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb3044e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7feb280ba0b0 0x7feb280cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb3044e640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb280cfad0 con 0x7feb280cf620 2024-03-30T13:12:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb3044e640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2815e950 con 0x7feb2815e570 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb3044e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7feb280b94a0 con 0x7feb280ba0b0 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb2d9e9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7feb280ba0b0 0x7feb280cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb2e9eb640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7feb280cf620 0x7feb280d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37550/0 (socket says 172.21.15.66:37550) 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb2e9eb640 1 -- 172.21.15.66:0/912188677 learned_addr learned my addr 172.21.15.66:0/912188677 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.780+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/912188677 >> v2:172.21.15.66:3300/0 conn(0x7feb280ba0b0 0x7feb280cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57828/0 (socket says 172.21.15.66:57828) 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.781+0000 7feb2d9e9640 1 -- 172.21.15.66:0/912188677 >> v1:172.21.15.117:6789/0 conn(0x7feb2815e570 legacy=0x7feb280b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.781+0000 7feb2d9e9640 1 -- 172.21.15.66:0/912188677 >> v1:172.21.15.66:6789/0 conn(0x7feb280cf620 legacy=0x7feb280d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.781+0000 7feb2d9e9640 1 -- 172.21.15.66:0/912188677 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb280b8890 con 0x7feb280ba0b0 2024-03-30T13:12:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.781+0000 7feb2d1e8640 1 -- 172.21.15.66:0/912188677 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2354312633 0 0) 0x7feb280cfad0 con 0x7feb280cf620 2024-03-30T13:12:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.781+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/912188677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280ba0b0 0x7feb280cf0e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7feb240097d0 tx=0x7feb2402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a635b131854c1dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb2d1e8640 1 -- 172.21.15.66:0/912188677 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24034070 con 0x7feb280ba0b0 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb2d1e8640 1 -- 172.21.15.66:0/912188677 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb24030610 con 0x7feb280ba0b0 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb2d1e8640 1 -- 172.21.15.66:0/912188677 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240309e0 con 0x7feb280ba0b0 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 -- 172.21.15.66:0/912188677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280ba0b0 msgr2=0x7feb280cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 --2- 172.21.15.66:0/912188677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280ba0b0 0x7feb280cf0e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7feb240097d0 tx=0x7feb2402b960 comp rx=0 tx=0).stop 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 -- 172.21.15.66:0/912188677 shutdown_connections 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 --2- 172.21.15.66:0/912188677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280ba0b0 0x7feb280cf0e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 -- 172.21.15.66:0/912188677 >> 172.21.15.66:0/912188677 conn(0x7feb280676e0 msgr2=0x7feb28067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 -- 172.21.15.66:0/912188677 shutdown_connections 2024-03-30T13:12:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.782+0000 7feb3044e640 1 -- 172.21.15.66:0/912188677 wait complete. 2024-03-30T13:12:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.783+0000 7feb3044e640 1 Processor -- start 2024-03-30T13:12:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.783+0000 7feb3044e640 1 -- start start 2024-03-30T13:12:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb3044e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb280ba0b0 0x7feb2815a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb3044e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 0x7feb281e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb3044e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2815e570 0x7feb281e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb3044e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb280b9130 con 0x7feb280cf620 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb3044e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb280b8e50 con 0x7feb280ba0b0 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb3044e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7feb280d2e40 con 0x7feb2815e570 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb2d9e9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 0x7feb281e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb2d9e9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 0x7feb281e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57836/0 (socket says 172.21.15.66:57836) 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb2d9e9640 1 -- 172.21.15.66:0/3682034423 learned_addr learned my addr 172.21.15.66:0/3682034423 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb280ba0b0 0x7feb2815a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb2e9eb640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2815e570 0x7feb281e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.784+0000 7feb2d9e9640 1 -- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2815e570 msgr2=0x7feb281e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2815e570 0x7feb281e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb2d9e9640 1 -- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb280ba0b0 msgr2=0x7feb2815a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb280ba0b0 0x7feb2815a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb2d9e9640 1 -- 172.21.15.66:0/3682034423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb280cb880 con 0x7feb280cf620 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb280ba0b0 0x7feb2815a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 0x7feb281e8070 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb240098f0 tx=0x7feb240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb1effd640 1 -- 172.21.15.66:0/3682034423 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24034040 con 0x7feb280cf620 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb1effd640 1 -- 172.21.15.66:0/3682034423 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb24039a10 con 0x7feb280cf620 2024-03-30T13:12:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb1effd640 1 -- 172.21.15.66:0/3682034423 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24039de0 con 0x7feb280cf620 2024-03-30T13:12:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb280cbb60 con 0x7feb280cf620 2024-03-30T13:12:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.785+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb280cc120 con 0x7feb280cf620 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.787+0000 7feb1effd640 1 -- 172.21.15.66:0/3682034423 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feb24037070 con 0x7feb280cf620 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.787+0000 7feb1effd640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7feb0c06c530 0x7feb0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.787+0000 7feb1effd640 1 -- 172.21.15.66:0/3682034423 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7feb240ada40 con 0x7feb280cf620 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.787+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7feb0c06c530 0x7feb0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.788+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7feb0c06c530 0x7feb0c06e9f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb18007fd0 tx=0x7feb18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.788+0000 7feb3303e740 1 Processor -- start 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.788+0000 7feb3303e740 1 -- start start 2024-03-30T13:12:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb3303e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e8e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e913300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e8e3a50 0x55c61e913840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb3303e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c61e84d590 con 0x55c61e910570 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c61e7ebd30 con 0x55c61e8e3a50 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c61e849540 con 0x55c61e910950 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb2e1ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e8e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb2e1ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e8e3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57846/0 (socket says 172.21.15.66:57846) 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb2e1ea640 1 -- 172.21.15.66:0/3231147345 learned_addr learned my addr 172.21.15.66:0/3231147345 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e913300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.789+0000 7feb2e9eb640 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e8e3a50 0x55c61e913840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb2e1ea640 1 -- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 msgr2=0x55c61e913300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e913300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb2e1ea640 1 -- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e8e3a50 msgr2=0x55c61e913840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e8e3a50 0x55c61e913840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb2e1ea640 1 -- 172.21.15.66:0/3231147345 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61e62d920 con 0x55c61e910570 2024-03-30T13:12:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e8e3510 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feb18005100 tx=0x7feb18011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4548083fe30f840 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb0bfff640 1 -- 172.21.15.66:0/3231147345 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180035f0 con 0x55c61e910570 2024-03-30T13:12:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb0bfff640 1 -- 172.21.15.66:0/3231147345 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb18003790 con 0x55c61e910570 2024-03-30T13:12:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.790+0000 7feb0bfff640 1 -- 172.21.15.66:0/3231147345 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18003b40 con 0x55c61e910570 2024-03-30T13:12:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 -- 172.21.15.66:0/3231147345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 msgr2=0x55c61e8e3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e8e3510 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feb18005100 tx=0x7feb18011000 comp rx=0 tx=0).stop 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 -- 172.21.15.66:0/3231147345 shutdown_connections 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e8e3a50 0x55c61e913840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e913300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 --2- 172.21.15.66:0/3231147345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e8e3510 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 -- 172.21.15.66:0/3231147345 >> 172.21.15.66:0/3231147345 conn(0x55c61e8e5780 msgr2=0x55c61e92e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 -- 172.21.15.66:0/3231147345 shutdown_connections 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 -- 172.21.15.66:0/3231147345 wait complete. 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.791+0000 7feb3303e740 1 -- start start 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.792+0000 7feb3303e740 1 Processor -- start 2024-03-30T13:12:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.792+0000 7feb3303e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e9131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.792+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910950 0x55c61e93e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.792+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e9151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.792+0000 7feb3303e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c61e8e7e20 con 0x55c61e910570 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.792+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c61e7e99f0 con 0x55c61e910950 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c61e8ea660 con 0x55c61e92ed70 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb3303e740 10 client.0 _ll_register_callbacks cb 0x55c61e7edc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e9131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e9131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57862/0 (socket says 172.21.15.66:57862) 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2458265879 learned_addr learned my addr 172.21.15.66:0/2458265879 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e9eb640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e9151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910950 0x55c61e93e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 msgr2=0x55c61e9151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e9151e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910950 msgr2=0x55c61e93e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910950 0x55c61e93e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2458265879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61e521210 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e9131a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feb1800eb80 tx=0x7feb18000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d982f3f6e2b55a00 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb097fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb097fa640 1 -- 172.21.15.66:0/2458265879 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1802f060 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb097fa640 1 -- 172.21.15.66:0/2458265879 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7feb18021450 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.793+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb18003790 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb097fa640 1 -- 172.21.15.66:0/2458265879 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7feb18035050 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 10 client.6027 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55c61e68e0b0 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb097fa640 1 -- 172.21.15.66:0/2458265879 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb18003790 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb097fa640 1 -- 172.21.15.66:0/2458265879 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7feb18038020 con 0x55c61e910570 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 10 client.6027 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 10 client.6027 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 20 client.6027 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e90b9d0 0x55c61e90be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 4 client.6027 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61e85aad0 con 0x55c61e90b9d0 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e90b9d0 0x55c61e90be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e90b9d0 0x55c61e90be10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316162e91bc4884d server_cookie=5dc3a43137943196 in_seq=0 out_seq=0 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.794+0000 7feb097fa640 10 client.6027 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb097fa640 1 -- 172.21.15.66:0/2458265879 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c61e85aad0 con 0x55c61e90b9d0 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb097fa640 10 client.6027 handle_command_reply: tid=0 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 client.6027 shutdown 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 msgr2=0x55c61e9131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e9131a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feb1800eb80 tx=0x7feb18000ed0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 shutdown_connections 2024-03-30T13:12:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e90b9d0 0x55c61e90be10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e9151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910950 0x55c61e93e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 --2- 172.21.15.66:0/2458265879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e9131a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 >> 172.21.15.66:0/2458265879 conn(0x55c61e8e3a50 msgr2=0x55c61e8e7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb097fa640 0 client.6027 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 shutdown_connections 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.795+0000 7feb3303e740 1 -- 172.21.15.66:0/2458265879 wait complete. 2024-03-30T13:12:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.796+0000 7feb3303e740 20 client.6027 trim_cache size 0 max 16384 2024-03-30T13:12:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.810+0000 7feb3303e740 1 Processor -- start 2024-03-30T13:12:00.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.810+0000 7feb3303e740 1 -- start start 2024-03-30T13:12:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb3303e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e93e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e93ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e92ed70 0x55c61e8e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb3303e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c61e90af60 con 0x55c61e910570 2024-03-30T13:12:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb28201170 con 0x55c61e92ed70 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7feb282001b0 con 0x55c61e910950 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e93e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e93e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57870/0 (socket says 172.21.15.66:57870) 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2692116237 learned_addr learned my addr 172.21.15.66:0/2692116237 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e93ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e9eb640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e92ed70 0x55c61e8e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 msgr2=0x55c61e93ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e93ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e92ed70 msgr2=0x55c61e8e7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e92ed70 0x55c61e8e7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 -- 172.21.15.66:0/2692116237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61e4fa2d0 con 0x55c61e910570 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e93ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e9eb640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e92ed70 0x55c61e8e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e93e0e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feb18001050 tx=0x7feb18023050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99b80229988733b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb0b7fe640 1 -- 172.21.15.66:0/2692116237 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18021450 con 0x55c61e910570 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb0b7fe640 1 -- 172.21.15.66:0/2692116237 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb18003640 con 0x55c61e910570 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.811+0000 7feb0b7fe640 1 -- 172.21.15.66:0/2692116237 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180218c0 con 0x55c61e910570 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 -- 172.21.15.66:0/2692116237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 msgr2=0x55c61e93e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e93e0e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feb18001050 tx=0x7feb18023050 comp rx=0 tx=0).stop 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 -- 172.21.15.66:0/2692116237 shutdown_connections 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e92ed70 0x55c61e8e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e910950 0x55c61e93ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 --2- 172.21.15.66:0/2692116237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910570 0x55c61e93e0e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 -- 172.21.15.66:0/2692116237 >> 172.21.15.66:0/2692116237 conn(0x55c61e8e3a50 msgr2=0x55c61e915f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 -- 172.21.15.66:0/2692116237 shutdown_connections 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 -- 172.21.15.66:0/2692116237 wait complete. 2024-03-30T13:12:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.812+0000 7feb3303e740 1 -- start start 2024-03-30T13:12:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 Processor -- start 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910570 0x55c61e93ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 0x55c61e93cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e931080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb281ffcc0 con 0x55c61e910950 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb281f5710 con 0x55c61e910570 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.813+0000 7feb3303e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c61e932070 con 0x55c61e92ed70 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb3303e740 10 client.0 _ll_register_callbacks cb 0x7feb20005bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 0x55c61e93cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 0x55c61e93cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57882/0 (socket says 172.21.15.66:57882) 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 -- 172.21.15.66:0/391553266 learned_addr learned my addr 172.21.15.66:0/391553266 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2e9eb640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e931080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910570 0x55c61e93ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 -- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 msgr2=0x55c61e931080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e931080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 -- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910570 msgr2=0x55c61e93ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910570 0x55c61e93ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 -- 172.21.15.66:0/391553266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61e8e8fd0 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb2d9e9640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 0x55c61e93cf60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feb2408ffd0 tx=0x7feb2404d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c7d655d69f858 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb097fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb097fa640 1 -- 172.21.15.66:0/391553266 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240aa060 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb097fa640 1 -- 172.21.15.66:0/391553266 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7feb240aba60 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb18003640 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb097fa640 1 -- 172.21.15.66:0/391553266 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7feb2407b7b0 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb3303e740 10 client.6033 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.814+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55c61e8eaa30 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb097fa640 1 -- 172.21.15.66:0/391553266 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb18003640 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb097fa640 1 -- 172.21.15.66:0/391553266 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7feb24041070 con 0x55c61e910950 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb3303e740 10 client.6033 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb3303e740 10 client.6033 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb3303e740 20 client.6033 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb3303e740 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e914750 0x55c61e9315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb3303e740 4 client.6033 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628343}] 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628343}) -- 0x55c61e854e20 con 0x55c61e914750 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e914750 0x55c61e9315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb097fa640 10 client.6033 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.815+0000 7feb2e1ea640 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e914750 0x55c61e9315c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0659c739ca2a4a server_cookie=a4d2911c6521fc59 in_seq=0 out_seq=0 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb097fa640 1 -- 172.21.15.66:0/391553266 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55c61e854e20 con 0x55c61e914750 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb097fa640 10 client.6033 handle_command_reply: tid=0 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 client.6033 shutdown 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 msgr2=0x55c61e93cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 0x55c61e93cf60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feb2408ffd0 tx=0x7feb2404d040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 shutdown_connections 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c61e914750 0x55c61e9315c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c61e92ed70 0x55c61e931080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c61e910950 0x55c61e93cf60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 --2- 172.21.15.66:0/391553266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c61e910570 0x55c61e93ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb097fa640 0 client.6033 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 >> 172.21.15.66:0/391553266 conn(0x55c61e8e3a50 msgr2=0x55c61e915ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 shutdown_connections 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 1 -- 172.21.15.66:0/391553266 wait complete. 2024-03-30T13:12:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3303e740 20 client.6033 trim_cache size 0 max 16384 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debugfs.h", 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628343, 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.940269+0000", 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.462202+0000", 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:00.828 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:00.829 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 311, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:00.830 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.462202+0000", 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.831 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 311, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.940269+0000" 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:00.832 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 311, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.940269+0000" 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 194, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 32, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:00.833 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:11:25.964045+0000", 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 88, 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.834 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.835 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:00.836 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:00.837 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.838 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:00.839 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:00.840 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:00.841 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:00.842 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:00.842 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:00.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7feb0c06c530 msgr2=0x7feb0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7feb0c06c530 0x7feb0c06e9f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb18007fd0 tx=0x7feb18009040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 msgr2=0x7feb281e8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 0x7feb281e8070 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb240098f0 tx=0x7feb240046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 shutdown_connections 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7feb0c06c530 0x7feb0c06e9f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb2815e570 0x7feb281e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb280cf620 0x7feb281e8070 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.816+0000 7feb3044e640 1 --2- 172.21.15.66:0/3682034423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb280ba0b0 0x7feb2815a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.817+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 >> 172.21.15.66:0/3682034423 conn(0x7feb280676e0 msgr2=0x7feb280c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.817+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 shutdown_connections 2024-03-30T13:12:00.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.817+0000 7feb3044e640 1 -- 172.21.15.66:0/3682034423 wait complete. 2024-03-30T13:12:00.844 DEBUG:tasks.cephfs.test_uninlining:getting inode info #20 of 23:1099511628344 2024-03-30T13:12:00.844 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628344 2024-03-30T13:12:00.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628344 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.950+0000 7ff7fab52640 1 Processor -- start 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.950+0000 7ff7fab52640 1 -- start start 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7fab52640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7f40b5dc0 0x7ff7f40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7fab52640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40b73d0 0x7ff7f40b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7fab52640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40b66e0 con 0x7ff7f40ba090 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7fab52640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f40d2d90 con 0x7ff7f40b5dc0 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7fab52640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40d2ff0 con 0x7ff7f40b73d0 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7f90ef640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40b73d0 0x7ff7f40b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.951+0000 7ff7f88ee640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7f40b5dc0 0x7ff7f40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.952+0000 7ff7f3fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff7f40ba090 0x7ff7f40d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:56730/0 (socket says 172.21.15.66:56730) 2024-03-30T13:12:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.952+0000 7ff7f3fff640 1 -- 172.21.15.66:0/2149551776 learned_addr learned my addr 172.21.15.66:0/2149551776 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.952+0000 7ff7f90ef640 1 -- 172.21.15.66:0/2149551776 >> v2:172.21.15.117:3300/0 conn(0x7ff7f40b5dc0 msgr2=0x7ff7f40b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.952+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2149551776 >> v2:172.21.15.117:3300/0 conn(0x7ff7f40b5dc0 0x7ff7f40b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.952+0000 7ff7f90ef640 1 -- 172.21.15.66:0/2149551776 >> v1:172.21.15.117:6789/0 conn(0x7ff7f40ba090 legacy=0x7ff7f40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.952+0000 7ff7f90ef640 1 -- 172.21.15.66:0/2149551776 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f40d32d0 con 0x7ff7f40b73d0 2024-03-30T13:12:00.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.953+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2149551776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40b73d0 0x7ff7f40b7840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009d90 tx=0x7ff7ec02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c730fde62619734b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.953+0000 7ff7f37fe640 1 -- 172.21.15.66:0/2149551776 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec039070 con 0x7ff7f40b73d0 2024-03-30T13:12:00.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.953+0000 7ff7f37fe640 1 -- 172.21.15.66:0/2149551776 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec02cc50 con 0x7ff7f40b73d0 2024-03-30T13:12:00.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.953+0000 7ff7f37fe640 1 -- 172.21.15.66:0/2149551776 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec034440 con 0x7ff7f40b73d0 2024-03-30T13:12:00.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 -- 172.21.15.66:0/2149551776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40b73d0 msgr2=0x7ff7f40b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2149551776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40b73d0 0x7ff7f40b7840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009d90 tx=0x7ff7ec02bea0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 -- 172.21.15.66:0/2149551776 shutdown_connections 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2149551776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40b73d0 0x7ff7f40b7840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2149551776 >> v2:172.21.15.117:3300/0 conn(0x7ff7f40b5dc0 0x7ff7f40b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 -- 172.21.15.66:0/2149551776 >> 172.21.15.66:0/2149551776 conn(0x7ff7f40678f0 msgr2=0x7ff7f40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 -- 172.21.15.66:0/2149551776 shutdown_connections 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.954+0000 7ff7fab52640 1 -- 172.21.15.66:0/2149551776 wait complete. 2024-03-30T13:12:00.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.956+0000 7ff7fab52640 1 Processor -- start 2024-03-30T13:12:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.956+0000 7ff7fab52640 1 -- start start 2024-03-30T13:12:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7fab52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f40b5dc0 0x7ff7f41e9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7fab52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 0x7ff7f4160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7fab52640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40ba090 0x7ff7f4160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7fab52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40d3b70 con 0x7ff7f40b73d0 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7fab52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40d3890 con 0x7ff7f40b5dc0 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7fab52640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40d4090 con 0x7ff7f40ba090 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7f3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 0x7ff7f4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7f3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 0x7ff7f4160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57888/0 (socket says 172.21.15.66:57888) 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.957+0000 7ff7f3fff640 1 -- 172.21.15.66:0/2126328007 learned_addr learned my addr 172.21.15.66:0/2126328007 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f40b5dc0 0x7ff7f41e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40ba090 0x7ff7f4160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f3fff640 1 -- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40ba090 msgr2=0x7ff7f4160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40ba090 0x7ff7f4160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f3fff640 1 -- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f40b5dc0 msgr2=0x7ff7f41e9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f40b5dc0 0x7ff7f41e9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f3fff640 1 -- 172.21.15.66:0/2126328007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4160da0 con 0x7ff7f40b73d0 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40ba090 0x7ff7f4160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f40b5dc0 0x7ff7f41e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 0x7ff7f4160200 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000c3c0 tx=0x7ff7e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f17fa640 1 -- 172.21.15.66:0/2126328007 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0017070 con 0x7ff7f40b73d0 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f17fa640 1 -- 172.21.15.66:0/2126328007 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e0005ce0 con 0x7ff7f40b73d0 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7f17fa640 1 -- 172.21.15.66:0/2126328007 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00072f0 con 0x7ff7f40b73d0 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f4161080 con 0x7ff7f40b73d0 2024-03-30T13:12:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.958+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f414a1d0 con 0x7ff7f40b73d0 2024-03-30T13:12:00.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.961+0000 7ff7f17fa640 1 -- 172.21.15.66:0/2126328007 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff7e001a070 con 0x7ff7f40b73d0 2024-03-30T13:12:00.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.961+0000 7ff7f17fa640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7d806c530 0x7ff7d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.961+0000 7ff7fd81d740 1 Processor -- start 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.961+0000 7ff7f17fa640 1 -- 172.21.15.66:0/2126328007 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff7e008ac00 con 0x7ff7f40b73d0 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.961+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7d806c530 0x7ff7d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.961+0000 7ff7fd81d740 1 -- start start 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7d806c530 0x7ff7d806e9f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009b90 tx=0x7ff7e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0570 0x5639f8ea3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8ed3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ea3a50 0x5639f8ed3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5639f8e0d590 con 0x5639f8ed0950 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5639f8dabd30 con 0x5639f8ea3a50 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5639f8e09540 con 0x5639f8ed0570 2024-03-30T13:12:00.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8ed3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8ed3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57896/0 (socket says 172.21.15.66:57896) 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 -- 172.21.15.66:0/4120401772 learned_addr learned my addr 172.21.15.66:0/4120401772 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ea3a50 0x5639f8ed3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0570 0x5639f8ea3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 -- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0570 msgr2=0x5639f8ea3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0570 0x5639f8ea3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 -- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ea3a50 msgr2=0x5639f8ed3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ea3a50 0x5639f8ed3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 -- 172.21.15.66:0/4120401772 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f8bed920 con 0x5639f8ed0950 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ea3a50 0x5639f8ed3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.962+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8ed3300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0088f80 tx=0x7ff7e0058a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878e02465a38d0d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.963+0000 7ff7de7fc640 1 -- 172.21.15.66:0/4120401772 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0056c70 con 0x5639f8ed0950 2024-03-30T13:12:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.963+0000 7ff7de7fc640 1 -- 172.21.15.66:0/4120401772 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e005f0f0 con 0x5639f8ed0950 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.963+0000 7ff7de7fc640 1 -- 172.21.15.66:0/4120401772 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00610c0 con 0x5639f8ed0950 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.963+0000 7ff7fd81d740 1 -- 172.21.15.66:0/4120401772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 msgr2=0x5639f8ed3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.963+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8ed3300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0088f80 tx=0x7ff7e0058a30 comp rx=0 tx=0).stop 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 -- 172.21.15.66:0/4120401772 shutdown_connections 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ea3a50 0x5639f8ed3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8ed3300 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/4120401772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0570 0x5639f8ea3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 -- 172.21.15.66:0/4120401772 >> 172.21.15.66:0/4120401772 conn(0x5639f8ea5780 msgr2=0x5639f8eee300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 -- 172.21.15.66:0/4120401772 shutdown_connections 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.964+0000 7ff7fd81d740 1 -- 172.21.15.66:0/4120401772 wait complete. 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.965+0000 7ff7fd81d740 1 -- start start 2024-03-30T13:12:00.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.965+0000 7ff7fd81d740 1 Processor -- start 2024-03-30T13:12:00.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 0x5639f8ed4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8efe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ec3580 0x5639f8eda110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5639f8ea7e20 con 0x5639f8ed0570 2024-03-30T13:12:00.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5639f8da99f0 con 0x5639f8ec3580 2024-03-30T13:12:00.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5639f8eaa660 con 0x5639f8ed0950 2024-03-30T13:12:00.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7fd81d740 10 client.0 _ll_register_callbacks cb 0x5639f8dadc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:00.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7f88ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 0x5639f8ed4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7f88ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 0x5639f8ed4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57904/0 (socket says 172.21.15.66:57904) 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2659078417 learned_addr learned my addr 172.21.15.66:0/2659078417 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.966+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8efe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ec3580 0x5639f8eda110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 msgr2=0x5639f8efe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8efe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ec3580 msgr2=0x5639f8eda110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ec3580 0x5639f8eda110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2659078417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f8ae1210 con 0x5639f8ed0570 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ec3580 0x5639f8eda110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 0x5639f8ed4eb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8015890 tx=0x7ff7e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00f4f32b9f742e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:00.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2659078417 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8025070 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2659078417 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff7e8014960 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e005f0f0 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2659078417 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff7e8030050 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7fd81d740 10 client.6042 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5639f8c4e0b0 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.967+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2659078417 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e005f0f0 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2659078417 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7ff7e8033020 con 0x5639f8ed0570 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7fd81d740 10 client.6042 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7fd81d740 10 client.6042 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7fd81d740 20 client.6042 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8f022d0 0x5639f8f026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7fd81d740 4 client.6042 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f8e1aad0 con 0x5639f8f022d0 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8f022d0 0x5639f8f026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7d37fe640 10 client.6042 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.968+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8f022d0 0x5639f8f026e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e0d81429936d39 server_cookie=2a68f495777fe284 in_seq=0 out_seq=0 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2659078417 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5639f8e1aad0 con 0x5639f8f022d0 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7d37fe640 10 client.6042 handle_command_reply: tid=0 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 client.6042 shutdown 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 msgr2=0x5639f8ed4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 0x5639f8ed4eb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8015890 tx=0x7ff7e80046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 shutdown_connections 2024-03-30T13:12:00.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8f022d0 0x5639f8f026e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ec3580 0x5639f8eda110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8efe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2659078417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0570 0x5639f8ed4eb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 >> 172.21.15.66:0/2659078417 conn(0x5639f8ea3a50 msgr2=0x5639f8ea7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7d37fe640 0 client.6042 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:00.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 shutdown_connections 2024-03-30T13:12:00.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2659078417 wait complete. 2024-03-30T13:12:00.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:00.969+0000 7ff7fd81d740 20 client.6042 trim_cache size 0 max 16384 2024-03-30T13:12:01.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.000+0000 7ff7fd81d740 1 Processor -- start 2024-03-30T13:12:01.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.000+0000 7ff7fd81d740 1 -- start start 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ec3580 0x5639f8eda060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8eaa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8f60870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4201330 con 0x5639f8ed0950 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4200350 con 0x5639f8ed0570 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f414c810 con 0x5639f8ec3580 2024-03-30T13:12:01.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8f60870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8f60870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57920/0 (socket says 172.21.15.66:57920) 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 -- 172.21.15.66:0/1471163932 learned_addr learned my addr 172.21.15.66:0/1471163932 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ec3580 0x5639f8eda060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8eaa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 -- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ec3580 msgr2=0x5639f8eda060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ec3580 0x5639f8eda060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 -- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 msgr2=0x5639f8eaa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8eaa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f90ef640 1 -- 172.21.15.66:0/1471163932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f8aba2d0 con 0x5639f8ed0950 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.001+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8eaa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.002+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8f60870 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009e80 tx=0x7ff7ec02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99fdfe157c99538 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.002+0000 7ff7ddffb640 1 -- 172.21.15.66:0/1471163932 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec039040 con 0x5639f8ed0950 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.002+0000 7ff7ddffb640 1 -- 172.21.15.66:0/1471163932 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec02cc00 con 0x5639f8ed0950 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.002+0000 7ff7ddffb640 1 -- 172.21.15.66:0/1471163932 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec034660 con 0x5639f8ed0950 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 -- 172.21.15.66:0/1471163932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 msgr2=0x5639f8f60870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8f60870 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009e80 tx=0x7ff7ec02e000 comp rx=0 tx=0).stop 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 -- 172.21.15.66:0/1471163932 shutdown_connections 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ed0950 0x5639f8f60870 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8eaa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/1471163932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ec3580 0x5639f8eda060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 -- 172.21.15.66:0/1471163932 >> 172.21.15.66:0/1471163932 conn(0x5639f8ea3a50 msgr2=0x5639f8e9e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 -- 172.21.15.66:0/1471163932 shutdown_connections 2024-03-30T13:12:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 -- 172.21.15.66:0/1471163932 wait complete. 2024-03-30T13:12:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 -- start start 2024-03-30T13:12:01.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.003+0000 7ff7fd81d740 1 Processor -- start 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 0x5639f8f41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8f00890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8f00dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5639f8eee650 con 0x5639f8ec3580 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ffd30 con 0x5639f8ed0570 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5639f8eeed70 con 0x5639f8ed0950 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7fd81d740 10 client.0 _ll_register_callbacks cb 0x7ff7e8011a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7f88ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 0x5639f8f41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7f88ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 0x5639f8f41210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57932/0 (socket says 172.21.15.66:57932) 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.004+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2758482985 learned_addr learned my addr 172.21.15.66:0/2758482985 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8f00890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f90ef640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8f00dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 msgr2=0x5639f8f00dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8f00dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 msgr2=0x5639f8f00890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8f00890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f88ee640 1 -- 172.21.15.66:0/2758482985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f8ed44a0 con 0x5639f8ec3580 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7f88ee640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 0x5639f8f41210 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8008010 tx=0x7ff7e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7485a6f01571dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2758482985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8025040 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2758482985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff7e802a440 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ec02cc00 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2758482985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ff7e8046050 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7fd81d740 10 client.6048 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5639f8ed4d40 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.005+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2758482985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec02cc00 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2758482985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7ff7e8033040 con 0x5639f8ec3580 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7fd81d740 10 client.6048 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7fd81d740 10 client.6048 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7fd81d740 20 client.6048 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8ef1820 0x5639f8f55800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7fd81d740 4 client.6048 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628344}] 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628344}) -- 0x5639f8e14e20 con 0x5639f8ef1820 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8ef1820 0x5639f8f55800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7d37fe640 10 client.6048 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.006+0000 7ff7f3fff640 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8ef1820 0x5639f8f55800 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4bb343b69eaed1 server_cookie=5eed84537b60629b in_seq=0 out_seq=0 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7d37fe640 1 -- 172.21.15.66:0/2758482985 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3984 (crc 0 0 0) 0x5639f8e14e20 con 0x5639f8ef1820 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7d37fe640 10 client.6048 handle_command_reply: tid=0 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 client.6048 shutdown 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 msgr2=0x5639f8f41210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 0x5639f8f41210 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8008010 tx=0x7ff7e8017000 comp rx=0 tx=0).stop 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 shutdown_connections 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5639f8ef1820 0x5639f8f55800 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5639f8ed0950 0x5639f8f00dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639f8ed0570 0x5639f8f00890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7d37fe640 0 client.6048 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 --2- 172.21.15.66:0/2758482985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639f8ec3580 0x5639f8f41210 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 >> 172.21.15.66:0/2758482985 conn(0x5639f8ea3a50 msgr2=0x5639f8eaa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 shutdown_connections 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 1 -- 172.21.15.66:0/2758482985 wait complete. 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fd81d740 20 client.6048 trim_cache size 0 max 16384 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-dev-proto.h", 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628344, 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.951674+0000", 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.463202+0000", 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:01.020 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 2111, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.463202+0000", 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 3, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.022 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2111, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.951674+0000" 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2111, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.951674+0000" 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 182, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 34, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:11:30.599588+0000", 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 102, 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:01.027 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.028 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.029 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:01.031 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:01.032 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:01.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7d806c530 msgr2=0x7ff7d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7d806c530 0x7ff7d806e9f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009b90 tx=0x7ff7e8008040 comp rx=0 tx=0).stop 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 msgr2=0x7ff7f4160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.007+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 0x7ff7f4160200 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000c3c0 tx=0x7ff7e000c880 comp rx=0 tx=0).stop 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 shutdown_connections 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff7d806c530 0x7ff7d806e9f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7f40ba090 0x7ff7f4160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7f40b73d0 0x7ff7f4160200 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 --2- 172.21.15.66:0/2126328007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f40b5dc0 0x7ff7f41e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 >> 172.21.15.66:0/2126328007 conn(0x7ff7f40678f0 msgr2=0x7ff7f40ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 shutdown_connections 2024-03-30T13:12:01.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.008+0000 7ff7fab52640 1 -- 172.21.15.66:0/2126328007 wait complete. 2024-03-30T13:12:01.035 DEBUG:tasks.cephfs.test_uninlining:getting inode info #21 of 23:1099511628329 2024-03-30T13:12:01.035 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628329 2024-03-30T13:12:01.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628329 2024-03-30T13:12:01.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.141+0000 7fe0aab38640 1 Processor -- start 2024-03-30T13:12:01.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.141+0000 7fe0aab38640 1 -- start start 2024-03-30T13:12:01.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0aab38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406db00 0x7fe0a40cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0aab38640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe0a406c690 0x7fe0a4162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0aab38640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0a41476b0 con 0x7fe0a41472d0 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0aab38640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a4148140 con 0x7fe0a406c690 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0aab38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a4071040 con 0x7fe0a406db00 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0a90d5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe0a406c690 0x7fe0a4162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0a88d4640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe0a41472d0 0x7fe0a406d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37556/0 (socket says 172.21.15.66:37556) 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.142+0000 7fe0a88d4640 1 -- 172.21.15.66:0/458618298 learned_addr learned my addr 172.21.15.66:0/458618298 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.143+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/458618298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406db00 0x7fe0a40cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.143+0000 7fe0a90d5640 1 -- 172.21.15.66:0/458618298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406db00 msgr2=0x7fe0a40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.143+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/458618298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406db00 0x7fe0a40cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.143+0000 7fe0a90d5640 1 -- 172.21.15.66:0/458618298 >> v1:172.21.15.66:6789/0 conn(0x7fe0a41472d0 legacy=0x7fe0a406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.143+0000 7fe0a90d5640 1 -- 172.21.15.66:0/458618298 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a406cb00 con 0x7fe0a406c690 2024-03-30T13:12:01.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.143+0000 7fe0a37fe640 1 -- 172.21.15.66:0/458618298 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711293930 0 0) 0x7fe0a41476b0 con 0x7fe0a41472d0 2024-03-30T13:12:01.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/458618298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a406c690 0x7fe0a4162110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009d90 tx=0x7fe09c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10000f3727cddcf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0a37fe640 1 -- 172.21.15.66:0/458618298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c039070 con 0x7fe0a406c690 2024-03-30T13:12:01.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0a37fe640 1 -- 172.21.15.66:0/458618298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09c02cbc0 con 0x7fe0a406c690 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0a37fe640 1 -- 172.21.15.66:0/458618298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c034440 con 0x7fe0a406c690 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 -- 172.21.15.66:0/458618298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a406c690 msgr2=0x7fe0a4162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 --2- 172.21.15.66:0/458618298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a406c690 0x7fe0a4162110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009d90 tx=0x7fe09c02be10 comp rx=0 tx=0).stop 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 -- 172.21.15.66:0/458618298 shutdown_connections 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 --2- 172.21.15.66:0/458618298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a406c690 0x7fe0a4162110 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 --2- 172.21.15.66:0/458618298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406db00 0x7fe0a40cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 -- 172.21.15.66:0/458618298 >> 172.21.15.66:0/458618298 conn(0x7fe0a40b3e70 msgr2=0x7fe0a40b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 -- 172.21.15.66:0/458618298 shutdown_connections 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.144+0000 7fe0aab38640 1 -- 172.21.15.66:0/458618298 wait complete. 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.145+0000 7fe0aab38640 1 Processor -- start 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 -- start start 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406c690 0x7fe0a4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a406db00 0x7fe0a415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 0x7fe0a4142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4071850 con 0x7fe0a41472d0 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4071570 con 0x7fe0a406db00 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.146+0000 7fe0aab38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a4071b30 con 0x7fe0a406c690 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 0x7fe0a4142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 0x7fe0a4142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57958/0 (socket says 172.21.15.66:57958) 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 -- 172.21.15.66:0/577188945 learned_addr learned my addr 172.21.15.66:0/577188945 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a406db00 0x7fe0a415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406c690 0x7fe0a4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 -- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406c690 msgr2=0x7fe0a4154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406c690 0x7fe0a4154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 -- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a406db00 msgr2=0x7fe0a415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a406db00 0x7fe0a415eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 -- 172.21.15.66:0/577188945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a415f080 con 0x7fe0a41472d0 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406c690 0x7fe0a4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 0x7fe0a4142390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009eb0 tx=0x7fe09c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a17fa640 1 -- 172.21.15.66:0/577188945 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c039050 con 0x7fe0a41472d0 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a17fa640 1 -- 172.21.15.66:0/577188945 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09c045070 con 0x7fe0a41472d0 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a40b5100 con 0x7fe0a41472d0 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0a17fa640 1 -- 172.21.15.66:0/577188945 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c009450 con 0x7fe0a41472d0 2024-03-30T13:12:01.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.147+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a40b56c0 con 0x7fe0a41472d0 2024-03-30T13:12:01.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.149+0000 7fe0a17fa640 1 -- 172.21.15.66:0/577188945 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe09c03e850 con 0x7fe0a41472d0 2024-03-30T13:12:01.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.150+0000 7fe0a17fa640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe08806c530 0x7fe08806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.150+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe08806c530 0x7fe08806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.150+0000 7fe0a17fa640 1 -- 172.21.15.66:0/577188945 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe09c0ae670 con 0x7fe0a41472d0 2024-03-30T13:12:01.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.150+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe08806c530 0x7fe08806e9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe098000f60 tx=0x7fe098007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.150+0000 7fe0ad7f3740 1 Processor -- start 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.150+0000 7fe0ad7f3740 1 -- start start 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065b8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065b8a50 0x55de065e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de06522590 con 0x55de065e5950 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de064c0d30 con 0x55de065b8a50 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55de0651e540 con 0x55de065e5570 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065e8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57960/0 (socket says 172.21.15.66:57960) 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0a3fff640 1 -- 172.21.15.66:0/1589654381 learned_addr learned my addr 172.21.15.66:0/1589654381 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065b8a50 0x55de065e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.151+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0a3fff640 1 -- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 msgr2=0x55de065b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0a3fff640 1 -- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065b8a50 msgr2=0x55de065e8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065b8a50 0x55de065e8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0a3fff640 1 -- 172.21.15.66:0/1589654381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de06302920 con 0x55de065e5950 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065e8300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe090009bc0 tx=0x7fe090011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ce62547dfd17c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe08e7fc640 1 -- 172.21.15.66:0/1589654381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090019070 con 0x55de065e5950 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe08e7fc640 1 -- 172.21.15.66:0/1589654381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe090004030 con 0x55de065e5950 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe08e7fc640 1 -- 172.21.15.66:0/1589654381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0900043e0 con 0x55de065e5950 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1589654381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 msgr2=0x55de065e8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065e8300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe090009bc0 tx=0x7fe090011690 comp rx=0 tx=0).stop 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1589654381 shutdown_connections 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065b8a50 0x55de065e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065e8300 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1589654381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.152+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1589654381 >> 172.21.15.66:0/1589654381 conn(0x55de065ba780 msgr2=0x55de06603300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.153+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1589654381 shutdown_connections 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.153+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1589654381 wait complete. 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.153+0000 7fe0ad7f3740 1 -- start start 2024-03-30T13:12:01.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.153+0000 7fe0ad7f3740 1 Processor -- start 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065ef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de06613250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de065bce20 con 0x55de065e5950 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de064be9f0 con 0x55de06612dd0 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55de065bf660 con 0x55de065e5570 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0ad7f3740 10 client.0 _ll_register_callbacks cb 0x55de064c2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065ea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57964/0 (socket says 172.21.15.66:57964) 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3006844919 learned_addr learned my addr 172.21.15.66:0/3006844919 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de06613250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.154+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 msgr2=0x55de065ef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065ef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 msgr2=0x55de06613250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de06613250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3006844919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de061f6210 con 0x55de065e5950 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065ea1e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe090011b90 tx=0x7fe090004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e056c0f23dd35434 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0837fe640 1 -- 172.21.15.66:0/3006844919 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090019040 con 0x55de065e5950 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0837fe640 1 -- 172.21.15.66:0/3006844919 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe090003b80 con 0x55de065e5950 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe090004030 con 0x55de065e5950 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0837fe640 1 -- 172.21.15.66:0/3006844919 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fe090003d80 con 0x55de065e5950 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 10 client.6063 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55de063630b0 con 0x55de065e5950 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0837fe640 1 -- 172.21.15.66:0/3006844919 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe090004030 con 0x55de065e5950 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0837fe640 1 -- 172.21.15.66:0/3006844919 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fe09001e450 con 0x55de065e5950 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 10 client.6063 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 10 client.6063 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 20 client.6063 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.155+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06617430 0x55de06617880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.156+0000 7fe0ad7f3740 4 client.6063 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.156+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de0652fad0 con 0x55de06617430 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.156+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06617430 0x55de06617880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.156+0000 7fe0837fe640 10 client.6063 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.156+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06617430 0x55de06617880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f5b660015d41a8 server_cookie=fd177975abf453db in_seq=0 out_seq=0 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0837fe640 1 -- 172.21.15.66:0/3006844919 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55de0652fad0 con 0x55de06617430 2024-03-30T13:12:01.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0837fe640 10 client.6063 handle_command_reply: tid=0 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 client.6063 shutdown 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 msgr2=0x55de065ea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065ea1e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe090011b90 tx=0x7fe090004730 comp rx=0 tx=0).stop 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 shutdown_connections 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06617430 0x55de06617880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de06613250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065ea1e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0837fe640 0 client.6063 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3006844919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5570 0x55de065ef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 >> 172.21.15.66:0/3006844919 conn(0x55de065b8a50 msgr2=0x55de065bc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 shutdown_connections 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3006844919 wait complete. 2024-03-30T13:12:01.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.157+0000 7fe0ad7f3740 20 client.6063 trim_cache size 0 max 16384 2024-03-30T13:12:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 Processor -- start 2024-03-30T13:12:01.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 -- start start 2024-03-30T13:12:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 0x55de06606a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5950 0x55de065e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de065b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4201060 con 0x55de065e5570 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a42000e0 con 0x55de06612dd0 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a40b7990 con 0x55de065e5950 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0a88d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 0x55de06606a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0a88d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 0x55de06606a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57966/0 (socket says 172.21.15.66:57966) 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.172+0000 7fe0a88d4640 1 -- 172.21.15.66:0/1121536931 learned_addr learned my addr 172.21.15.66:0/1121536931 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0a88d4640 1 -- 172.21.15.66:0/1121536931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5950 msgr2=0x55de065e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5950 0x55de065e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0a88d4640 1 -- 172.21.15.66:0/1121536931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 msgr2=0x55de065b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de065b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0a88d4640 1 -- 172.21.15.66:0/1121536931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de06603980 con 0x55de065e5570 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 0x55de06606a20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe098001720 tx=0x7fe098012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9771e1e28d8499dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe08dffb640 1 -- 172.21.15.66:0/1121536931 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980281d0 con 0x55de065e5570 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe08dffb640 1 -- 172.21.15.66:0/1121536931 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe098028370 con 0x55de065e5570 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe08dffb640 1 -- 172.21.15.66:0/1121536931 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098028520 con 0x55de065e5570 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1121536931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 msgr2=0x55de06606a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.173+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 0x55de06606a20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe098001720 tx=0x7fe098012000 comp rx=0 tx=0).stop 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1121536931 shutdown_connections 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de06612dd0 0x55de065b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de065e5950 0x55de065e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/1121536931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5570 0x55de06606a20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1121536931 >> 172.21.15.66:0/1121536931 conn(0x55de065b8a50 msgr2=0x55de065f0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1121536931 shutdown_connections 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/1121536931 wait complete. 2024-03-30T13:12:01.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 -- start start 2024-03-30T13:12:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.174+0000 7fe0ad7f3740 1 Processor -- start 2024-03-30T13:12:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065e5570 0x55de065d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065d8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de06612dd0 0x55de065d69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de065bec10 con 0x55de065e5950 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41ffa90 con 0x55de065e5570 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55de065f00f0 con 0x55de06612dd0 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0ad7f3740 10 client.0 _ll_register_callbacks cb 0x7fe09c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065d8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57980/0 (socket says 172.21.15.66:57980) 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3976764794 learned_addr learned my addr 172.21.15.66:0/3976764794 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065e5570 0x55de065d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.175+0000 7fe0a90d5640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de06612dd0 0x55de065d69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de06612dd0 msgr2=0x55de065d69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de06612dd0 0x55de065d69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065e5570 msgr2=0x55de065d8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065e5570 0x55de065d8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0a3fff640 1 -- 172.21.15.66:0/3976764794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de061cf2d0 con 0x55de065e5950 2024-03-30T13:12:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0a3fff640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065d8540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe09001e760 tx=0x7fe0900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8d5f6f22b4672b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0837fe640 1 -- 172.21.15.66:0/3976764794 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09003b060 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0837fe640 1 -- 172.21.15.66:0/3976764794 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fe090003b80 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe098028370 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0837fe640 1 -- 172.21.15.66:0/3976764794 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fe090041050 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 10 client.6069 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55de0667f900 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0837fe640 1 -- 172.21.15.66:0/3976764794 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe098028370 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0837fe640 1 -- 172.21.15.66:0/3976764794 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fe090044020 con 0x55de065e5950 2024-03-30T13:12:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 10 client.6069 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 10 client.6069 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 20 client.6069 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06627ac0 0x55de065d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.176+0000 7fe0ad7f3740 4 client.6069 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628329}] 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628329}) -- 0x55de06529e20 con 0x55de06627ac0 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06627ac0 0x55de065d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0a88d4640 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06627ac0 0x55de065d6f20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17cb98bc1004800 server_cookie=e33d3f8f873e8935 in_seq=0 out_seq=0 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0837fe640 10 client.6069 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0837fe640 1 -- 172.21.15.66:0/3976764794 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3968 (crc 0 0 0) 0x55de06529e20 con 0x55de06627ac0 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0837fe640 10 client.6069 handle_command_reply: tid=0 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.177+0000 7fe0ad7f3740 1 client.6069 shutdown 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 msgr2=0x55de065d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065d8540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe09001e760 tx=0x7fe0900046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 shutdown_connections 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55de06627ac0 0x55de065d6f20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55de06612dd0 0x55de065d69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de065e5950 0x55de065d8540 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0837fe640 0 client.6069 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 --2- 172.21.15.66:0/3976764794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de065e5570 0x55de065d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 >> 172.21.15.66:0/3976764794 conn(0x55de065b8a50 msgr2=0x55de065e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 shutdown_connections 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 1 -- 172.21.15.66:0/3976764794 wait complete. 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0ad7f3740 20 client.6069 trim_cache size 0 max 16384 2024-03-30T13:12:01.187 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/Makefile", 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628329, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.953333+0000", 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.447202+0000", 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:01.188 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 356, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.447202+0000", 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.953333+0000" 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.953333+0000" 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 170, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 4, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:11:30.599003+0000", 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 108, 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:01.199 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:01.200 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe08806c530 msgr2=0x7fe08806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe08806c530 0x7fe08806e9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe098000f60 tx=0x7fe098007ca0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 msgr2=0x7fe0a4142390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 0x7fe0a4142390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009eb0 tx=0x7fe09c004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 shutdown_connections 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe08806c530 0x7fe08806e9f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0a41472d0 0x7fe0a4142390 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0a406db00 0x7fe0a415eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 --2- 172.21.15.66:0/577188945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0a406c690 0x7fe0a4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.178+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 >> 172.21.15.66:0/577188945 conn(0x7fe0a40b3e70 msgr2=0x7fe0a40b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.179+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 shutdown_connections 2024-03-30T13:12:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.179+0000 7fe0aab38640 1 -- 172.21.15.66:0/577188945 wait complete. 2024-03-30T13:12:01.203 DEBUG:tasks.cephfs.test_uninlining:getting inode info #22 of 23:1099511628352 2024-03-30T13:12:01.203 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628352 2024-03-30T13:12:01.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628352 2024-03-30T13:12:01.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.310+0000 7fa6b297c640 1 Processor -- start 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.310+0000 7fa6b297c640 1 -- start start 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b297c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6ac072d50 0x7fa6ac0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b297c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6ac15f800 0x7fa6ac0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b297c640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac15fc70 con 0x7fa6ac144cb0 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b297c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ac0b3e80 con 0x7fa6ac15f800 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b297c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ac0b35e0 con 0x7fa6ac072d50 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b0f19640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6ac15f800 0x7fa6ac0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b0f19640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa6ac15f800 0x7fa6ac0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57988/0 (socket says 172.21.15.66:57988) 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6b0f19640 1 -- 172.21.15.66:0/64748366 learned_addr learned my addr 172.21.15.66:0/64748366 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.311+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/64748366 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac072d50 0x7fa6ac0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.312+0000 7fa6b0f19640 1 -- 172.21.15.66:0/64748366 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac072d50 msgr2=0x7fa6ac0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.312+0000 7fa6b0f19640 1 --2- 172.21.15.66:0/64748366 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac072d50 0x7fa6ac0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.312+0000 7fa6b0f19640 1 -- 172.21.15.66:0/64748366 >> v1:172.21.15.66:6789/0 conn(0x7fa6ac144cb0 legacy=0x7fa6ac072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.312+0000 7fa6aaffd640 1 -- 172.21.15.66:0/64748366 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220467657 0 0) 0x7fa6ac15fc70 con 0x7fa6ac144cb0 2024-03-30T13:12:01.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.312+0000 7fa6b0f19640 1 -- 172.21.15.66:0/64748366 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac0b38c0 con 0x7fa6ac15f800 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.312+0000 7fa6b0f19640 1 --2- 172.21.15.66:0/64748366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac15f800 0x7fa6ac0ce050 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac15ed50 tx=0x7fa6a402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaed0fcd379f34c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6aaffd640 1 -- 172.21.15.66:0/64748366 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a402cb20 con 0x7fa6ac15f800 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6aaffd640 1 -- 172.21.15.66:0/64748366 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a402ccc0 con 0x7fa6ac15f800 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6aaffd640 1 -- 172.21.15.66:0/64748366 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4034610 con 0x7fa6ac15f800 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 -- 172.21.15.66:0/64748366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac15f800 msgr2=0x7fa6ac0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 --2- 172.21.15.66:0/64748366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac15f800 0x7fa6ac0ce050 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac15ed50 tx=0x7fa6a402bf80 comp rx=0 tx=0).stop 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 -- 172.21.15.66:0/64748366 shutdown_connections 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 --2- 172.21.15.66:0/64748366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac15f800 0x7fa6ac0ce050 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 --2- 172.21.15.66:0/64748366 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac072d50 0x7fa6ac0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 -- 172.21.15.66:0/64748366 >> 172.21.15.66:0/64748366 conn(0x7fa6ac0b5b00 msgr2=0x7fa6ac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 -- 172.21.15.66:0/64748366 shutdown_connections 2024-03-30T13:12:01.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.313+0000 7fa6b297c640 1 -- 172.21.15.66:0/64748366 wait complete. 2024-03-30T13:12:01.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.315+0000 7fa6b297c640 1 Processor -- start 2024-03-30T13:12:01.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.315+0000 7fa6b297c640 1 -- start start 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b297c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 0x7fa6ac1364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b297c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac144cb0 0x7fa6ac1432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b297c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac15f800 0x7fa6ac143830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b297c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac0ce590 con 0x7fa6ac072d50 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b297c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac0cdb30 con 0x7fa6ac144cb0 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b297c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac0ce710 con 0x7fa6ac15f800 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 0x7fa6ac1364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 0x7fa6ac1364d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57996/0 (socket says 172.21.15.66:57996) 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6abfff640 1 -- 172.21.15.66:0/852505304 learned_addr learned my addr 172.21.15.66:0/852505304 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6b0f19640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac15f800 0x7fa6ac143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.316+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac144cb0 0x7fa6ac1432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6abfff640 1 -- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac15f800 msgr2=0x7fa6ac143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6abfff640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac15f800 0x7fa6ac143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6abfff640 1 -- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac144cb0 msgr2=0x7fa6ac1432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6abfff640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac144cb0 0x7fa6ac1432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6abfff640 1 -- 172.21.15.66:0/852505304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac159310 con 0x7fa6ac072d50 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6abfff640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 0x7fa6ac1364d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c4b0 tx=0x7fa69400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6a8ff9640 1 -- 172.21.15.66:0/852505304 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694017070 con 0x7fa6ac072d50 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6a8ff9640 1 -- 172.21.15.66:0/852505304 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694005480 con 0x7fa6ac072d50 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac155bf0 con 0x7fa6ac072d50 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6a8ff9640 1 -- 172.21.15.66:0/852505304 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694003830 con 0x7fa6ac072d50 2024-03-30T13:12:01.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.317+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1575c0 con 0x7fa6ac072d50 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.318+0000 7fa6a8ff9640 1 -- 172.21.15.66:0/852505304 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa69401a070 con 0x7fa6ac072d50 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.319+0000 7fa6a8ff9640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa68c06c530 0x7fa68c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.319+0000 7fa6a8ff9640 1 -- 172.21.15.66:0/852505304 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa69408ab80 con 0x7fa6ac072d50 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.319+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa68c06c530 0x7fa68c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.319+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa68c06c530 0x7fa68c06e9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac13b1a0 tx=0x7fa69c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.320+0000 7fa6b5565740 1 Processor -- start 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.320+0000 7fa6b5565740 1 -- start start 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.320+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cd94510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.320+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.320+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cd94a50 0x55d17cdc4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6b5565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d17ccfe590 con 0x55d17cdc1570 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d17cc9cd30 con 0x55d17cd94a50 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d17ccfa540 con 0x55d17cdc1950 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cd94510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cd94510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58000/0 (socket says 172.21.15.66:58000) 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 -- 172.21.15.66:0/3992033232 learned_addr learned my addr 172.21.15.66:0/3992033232 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 -- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 msgr2=0x55d17cdc4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6b0f19640 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cd94a50 0x55d17cdc4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 -- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cd94a50 msgr2=0x55d17cdc4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cd94a50 0x55d17cdc4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 -- 172.21.15.66:0/3992033232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17cade920 con 0x55d17cdc1570 2024-03-30T13:12:01.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6abfff640 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cd94510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa694088f30 tx=0x7fa694001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad02e87e7fdba87 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6a1ffb640 1 -- 172.21.15.66:0/3992033232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940550a0 con 0x55d17cdc1570 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6a1ffb640 1 -- 172.21.15.66:0/3992033232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69405f0b0 con 0x55d17cdc1570 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.321+0000 7fa6a1ffb640 1 -- 172.21.15.66:0/3992033232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694061310 con 0x55d17cdc1570 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 -- 172.21.15.66:0/3992033232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 msgr2=0x55d17cd94510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cd94510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa694088f30 tx=0x7fa694001080 comp rx=0 tx=0).stop 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 -- 172.21.15.66:0/3992033232 shutdown_connections 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cd94a50 0x55d17cdc4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3992033232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cd94510 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 -- 172.21.15.66:0/3992033232 >> 172.21.15.66:0/3992033232 conn(0x55d17cd96780 msgr2=0x55d17cddf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 -- 172.21.15.66:0/3992033232 shutdown_connections 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 -- 172.21.15.66:0/3992033232 wait complete. 2024-03-30T13:12:01.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.322+0000 7fa6b5565740 1 -- start start 2024-03-30T13:12:01.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 Processor -- start 2024-03-30T13:12:01.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17cdef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 0x55d17cdb9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdbc6b0 0x55d17cdba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d17cd98e20 con 0x55d17cdc1950 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d17cc9a9f0 con 0x55d17cdc1570 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.323+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d17cd9b660 con 0x55d17cdbc6b0 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6b5565740 10 client.0 _ll_register_callbacks cb 0x55d17cc9ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 0x55d17cdb9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 0x55d17cdb9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58002/0 (socket says 172.21.15.66:58002) 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2020443993 learned_addr learned my addr 172.21.15.66:0/2020443993 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6abfff640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17cdef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6b0f19640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdbc6b0 0x55d17cdba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdbc6b0 msgr2=0x55d17cdba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdbc6b0 0x55d17cdba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 msgr2=0x55d17cdef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17cdef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2020443993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d17c9d2210 con 0x55d17cdc1950 2024-03-30T13:12:01.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 0x55d17cdb9c30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa69c004620 tx=0x7fa69c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2556ce013682b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa68affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa68affd640 1 -- 172.21.15.66:0/2020443993 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c01e050 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa68affd640 1 -- 172.21.15.66:0/2020443993 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa69c003db0 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69405f0b0 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa68affd640 1 -- 172.21.15.66:0/2020443993 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa69c023440 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6b5565740 10 client.6084 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.324+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55d17cb3f0b0 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa68affd640 1 -- 172.21.15.66:0/2020443993 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69405f0b0 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa68affd640 1 -- 172.21.15.66:0/2020443993 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fa69c033080 con 0x55d17cdc1950 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6b5565740 10 client.6084 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6b5565740 10 client.6084 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6b5565740 20 client.6084 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdf30b0 0x55d17cdf3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6b5565740 4 client.6084 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d17cd0bad0 con 0x55d17cdf30b0 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6abfff640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdf30b0 0x55d17cdf3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa68affd640 10 client.6084 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.325+0000 7fa6abfff640 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdf30b0 0x55d17cdf3510 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a07553526d6483 server_cookie=84da0565f7cb154d in_seq=0 out_seq=0 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa68affd640 1 -- 172.21.15.66:0/2020443993 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d17cd0bad0 con 0x55d17cdf30b0 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa68affd640 10 client.6084 handle_command_reply: tid=0 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 client.6084 shutdown 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 msgr2=0x55d17cdb9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 0x55d17cdb9c30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa69c004620 tx=0x7fa69c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 shutdown_connections 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdf30b0 0x55d17cdf3510 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdbc6b0 0x55d17cdba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1950 0x55d17cdb9c30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa68affd640 0 client.6084 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2020443993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17cdef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 >> 172.21.15.66:0/2020443993 conn(0x55d17cd94a50 msgr2=0x55d17cde2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 shutdown_connections 2024-03-30T13:12:01.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 1 -- 172.21.15.66:0/2020443993 wait complete. 2024-03-30T13:12:01.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.326+0000 7fa6b5565740 20 client.6084 trim_cache size 0 max 16384 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 Processor -- start 2024-03-30T13:12:01.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 -- start start 2024-03-30T13:12:01.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdbc6b0 0x55d17cdc7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cde06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cde0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1fed30 con 0x55d17cdc1570 2024-03-30T13:12:01.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1fddd0 con 0x55d17cdbc6b0 2024-03-30T13:12:01.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.341+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac13aac0 con 0x55d17cdc1950 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cde06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cde06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58008/0 (socket says 172.21.15.66:58008) 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2187313476 learned_addr learned my addr 172.21.15.66:0/2187313476 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2187313476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 msgr2=0x55d17cde0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cde0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2187313476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdbc6b0 msgr2=0x55d17cdc7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdbc6b0 0x55d17cdc7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 -- 172.21.15.66:0/2187313476 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17cdba950 con 0x55d17cdc1570 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cde06d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0096a0 tx=0x7fa69c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b77a28148beb2c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6a17fa640 1 -- 172.21.15.66:0/2187313476 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c01e040 con 0x55d17cdc1570 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6a17fa640 1 -- 172.21.15.66:0/2187313476 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c028070 con 0x55d17cdc1570 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.342+0000 7fa6a17fa640 1 -- 172.21.15.66:0/2187313476 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c023d50 con 0x55d17cdc1570 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 -- 172.21.15.66:0/2187313476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 msgr2=0x55d17cde06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cde06d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0096a0 tx=0x7fa69c022000 comp rx=0 tx=0).stop 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 -- 172.21.15.66:0/2187313476 shutdown_connections 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cde0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdc1570 0x55d17cde06d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 --2- 172.21.15.66:0/2187313476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdbc6b0 0x55d17cdc7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 -- 172.21.15.66:0/2187313476 >> 172.21.15.66:0/2187313476 conn(0x55d17cd94a50 msgr2=0x55d17cde1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 -- 172.21.15.66:0/2187313476 shutdown_connections 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.343+0000 7fa6b5565740 1 -- 172.21.15.66:0/2187313476 wait complete. 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.344+0000 7fa6b5565740 1 -- start start 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.344+0000 7fa6b5565740 1 Processor -- start 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 0x55d17cdb8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17ce52410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d17cdbc050 con 0x55d17cdbc6b0 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac154690 con 0x55d17cdc1570 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d17cdc8510 con 0x55d17cdc1950 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 10 client.0 _ll_register_callbacks cb 0x7fa69c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 0x55d17cdb8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 0x55d17cdb8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58024/0 (socket says 172.21.15.66:58024) 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 -- 172.21.15.66:0/3230218669 learned_addr learned my addr 172.21.15.66:0/3230218669 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17ce52410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b0f19640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 -- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 msgr2=0x55d17cdc6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 -- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 msgr2=0x55d17ce52410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17ce52410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 -- 172.21.15.66:0/3230218669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d17c9ab2d0 con 0x55d17cdbc6b0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17ce52410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6abfff640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 0x55d17cdb8840 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa6940586e0 tx=0x7fa694000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683341735950775e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa68affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa68affd640 1 -- 172.21.15.66:0/3230218669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940610d0 con 0x55d17cdbc6b0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa68affd640 1 -- 172.21.15.66:0/3230218669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa694061270 con 0x55d17cdbc6b0 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.345+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69c028070 con 0x55d17cdbc6b0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa68affd640 1 -- 172.21.15.66:0/3230218669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa694069050 con 0x55d17cdbc6b0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 10 client.6090 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55d17ce03330 con 0x55d17cdbc6b0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa68affd640 1 -- 172.21.15.66:0/3230218669 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c028070 con 0x55d17cdbc6b0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa68affd640 1 -- 172.21.15.66:0/3230218669 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fa69406c020 con 0x55d17cdbc6b0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 10 client.6090 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 10 client.6090 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 20 client.6090 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdb8ed0 0x55d17cdc6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 4 client.6090 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628352}] 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628352}) -- 0x55d17cd05e20 con 0x55d17cdb8ed0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdb8ed0 0x55d17cdc6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.346+0000 7fa68affd640 10 client.6090 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6ab7fe640 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdb8ed0 0x55d17cdc6790 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6cd8b7da5cee730 server_cookie=ab3e1f924a1f930b in_seq=0 out_seq=0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa68affd640 1 -- 172.21.15.66:0/3230218669 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3970 (crc 0 0 0) 0x55d17cd05e20 con 0x55d17cdb8ed0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa68affd640 10 client.6090 handle_command_reply: tid=0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 client.6090 shutdown 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 msgr2=0x55d17cdb8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 0x55d17cdb8840 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa6940586e0 tx=0x7fa694000ed0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 shutdown_connections 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d17cdb8ed0 0x55d17cdc6790 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d17cdc1950 0x55d17cdc6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17cdc1570 0x55d17ce52410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 --2- 172.21.15.66:0/3230218669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17cdbc6b0 0x55d17cdb8840 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa68affd640 0 client.6090 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 >> 172.21.15.66:0/3230218669 conn(0x55d17cd94a50 msgr2=0x55d17ce02090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 shutdown_connections 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 1 -- 172.21.15.66:0/3230218669 wait complete. 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.347+0000 7fa6b5565740 20 client.6090 trim_cache size 0 max 16384 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/symlink.c", 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628352, 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.955252+0000", 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.469202+0000", 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:01.358 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 497, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.469202+0000", 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:01.359 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 497, 2024-03-30T13:12:01.360 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.955252+0000" 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 497, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.955252+0000" 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 220, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:01.361 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 50, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:11:30.600162+0000", 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 110, 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:01.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.363 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:01.364 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.365 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.366 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.368 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.369 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:01.369 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:01.369 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:01.369 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:01.369 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa68c06c530 msgr2=0x7fa68c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa68c06c530 0x7fa68c06e9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac13b1a0 tx=0x7fa69c009190 comp rx=0 tx=0).stop 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 msgr2=0x7fa6ac1364d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 0x7fa6ac1364d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c4b0 tx=0x7fa69400c970 comp rx=0 tx=0).stop 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 shutdown_connections 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa68c06c530 0x7fa68c06e9f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac15f800 0x7fa6ac143830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac144cb0 0x7fa6ac1432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 --2- 172.21.15.66:0/852505304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6ac072d50 0x7fa6ac1364d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 >> 172.21.15.66:0/852505304 conn(0x7fa6ac0b5b00 msgr2=0x7fa6ac072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 shutdown_connections 2024-03-30T13:12:01.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.348+0000 7fa6b297c640 1 -- 172.21.15.66:0/852505304 wait complete. 2024-03-30T13:12:01.372 DEBUG:tasks.cephfs.test_uninlining:getting inode info #23 of 23:1099511628334 2024-03-30T13:12:01.372 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628334 2024-03-30T13:12:01.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628334 2024-03-30T13:12:01.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.479+0000 7f01e2dd4640 1 Processor -- start 2024-03-30T13:12:01.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.479+0000 7f01e2dd4640 1 -- start start 2024-03-30T13:12:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e2dd4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01dc0b5dc0 0x7f01dc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e2dd4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0ba090 0x7f01dc0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e2dd4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01dc0b66e0 con 0x7f01dc06c620 2024-03-30T13:12:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e2dd4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f01dc0d2ff0 con 0x7f01dc0b5dc0 2024-03-30T13:12:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e2dd4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc0d32d0 con 0x7f01dc0ba090 2024-03-30T13:12:01.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e1371640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f01dc06c620 0x7f01dc06cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37582/0 (socket says 172.21.15.66:37582) 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e1371640 1 -- 172.21.15.66:0/3362402403 learned_addr learned my addr 172.21.15.66:0/3362402403 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3362402403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0ba090 0x7f01dc0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3362402403 >> v2:172.21.15.117:3300/0 conn(0x7f01dc0b5dc0 0x7f01dc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01db7fe640 1 -- 172.21.15.66:0/3362402403 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664358353 0 0) 0x7f01dc0b66e0 con 0x7f01dc06c620 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01db7fe640 1 -- 172.21.15.66:0/3362402403 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01cc003520 con 0x7f01dc06c620 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01dbfff640 1 -- 172.21.15.66:0/3362402403 >> v2:172.21.15.117:3300/0 conn(0x7f01dc0b5dc0 msgr2=0x7f01dc0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3362402403 >> v2:172.21.15.117:3300/0 conn(0x7f01dc0b5dc0 0x7f01dc0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01dbfff640 1 -- 172.21.15.66:0/3362402403 >> v1:172.21.15.66:6789/0 conn(0x7f01dc06c620 legacy=0x7f01dc06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01dbfff640 1 -- 172.21.15.66:0/3362402403 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc0d35b0 con 0x7f01dc0ba090 2024-03-30T13:12:01.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.480+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3362402403 >> v2:172.21.15.117:3300/0 conn(0x7f01dc0b5dc0 0x7f01dc0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.481+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3362402403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0ba090 0x7f01dc0d2680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097a0 tx=0x7f01c802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b7b5b9751c4040 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.481+0000 7f01db7fe640 1 -- 172.21.15.66:0/3362402403 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035020 con 0x7f01dc0ba090 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.481+0000 7f01db7fe640 1 -- 172.21.15.66:0/3362402403 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8031830 con 0x7f01dc0ba090 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.481+0000 7f01db7fe640 1 -- 172.21.15.66:0/3362402403 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8031be0 con 0x7f01dc0ba090 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 -- 172.21.15.66:0/3362402403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0ba090 msgr2=0x7f01dc0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/3362402403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0ba090 0x7f01dc0d2680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097a0 tx=0x7f01c802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 -- 172.21.15.66:0/3362402403 shutdown_connections 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/3362402403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0ba090 0x7f01dc0d2680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/3362402403 >> v2:172.21.15.117:3300/0 conn(0x7f01dc0b5dc0 0x7f01dc0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 -- 172.21.15.66:0/3362402403 >> 172.21.15.66:0/3362402403 conn(0x7f01dc0678f0 msgr2=0x7f01dc0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 -- 172.21.15.66:0/3362402403 shutdown_connections 2024-03-30T13:12:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.482+0000 7f01e2dd4640 1 -- 172.21.15.66:0/3362402403 wait complete. 2024-03-30T13:12:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.483+0000 7f01e2dd4640 1 Processor -- start 2024-03-30T13:12:01.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.483+0000 7f01e2dd4640 1 -- start start 2024-03-30T13:12:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e2dd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 0x7f01dc0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e2dd4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0b5dc0 0x7f01dc0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e2dd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc0ba090 0x7f01dc0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e2dd4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0d4090 con 0x7f01dc06c620 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e2dd4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0d3b70 con 0x7f01dc0ba090 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e2dd4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc0b83e0 con 0x7f01dc0b5dc0 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 0x7f01dc0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 0x7f01dc0ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58036/0 (socket says 172.21.15.66:58036) 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 -- 172.21.15.66:0/1672743099 learned_addr learned my addr 172.21.15.66:0/1672743099 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01dbfff640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0b5dc0 0x7f01dc0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e1371640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc0ba090 0x7f01dc0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 -- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0b5dc0 msgr2=0x7f01dc0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0b5dc0 0x7f01dc0cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 -- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc0ba090 msgr2=0x7f01dc0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.484+0000 7f01e0b70640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc0ba090 0x7f01dc0cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01e0b70640 1 -- 172.21.15.66:0/1672743099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc1f86a0 con 0x7f01dc06c620 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01dbfff640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0b5dc0 0x7f01dc0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01e0b70640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 0x7f01dc0ca7e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f01d000ebe0 tx=0x7f01d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01d97fa640 1 -- 172.21.15.66:0/1672743099 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0017070 con 0x7f01dc06c620 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01d97fa640 1 -- 172.21.15.66:0/1672743099 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d000cc30 con 0x7f01dc06c620 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc1583f0 con 0x7f01dc06c620 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01d97fa640 1 -- 172.21.15.66:0/1672743099 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0013470 con 0x7f01dc06c620 2024-03-30T13:12:01.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.485+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1589b0 con 0x7f01dc06c620 2024-03-30T13:12:01.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.486+0000 7f01d97fa640 1 -- 172.21.15.66:0/1672743099 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f01d001a070 con 0x7f01dc06c620 2024-03-30T13:12:01.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.487+0000 7f01d97fa640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f01c006c530 0x7f01c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.487+0000 7f01d97fa640 1 -- 172.21.15.66:0/1672743099 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f01d008b7b0 con 0x7f01dc06c620 2024-03-30T13:12:01.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.487+0000 7f01dbfff640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f01c006c530 0x7f01c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.487+0000 7f01dbfff640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f01c006c530 0x7f01c006e9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01c8036000 tx=0x7f01c803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.489+0000 7f01e59c2740 1 Processor -- start 2024-03-30T13:12:01.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.489+0000 7f01e59c2740 1 -- start start 2024-03-30T13:12:01.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 0x561ce60eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce611b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce60eba50 0x561ce611b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e59c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561ce6055590 con 0x561ce6118570 2024-03-30T13:12:01.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ce5ff3d30 con 0x561ce6118950 2024-03-30T13:12:01.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561ce6051540 con 0x561ce60eba50 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 0x561ce60eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 0x561ce60eb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58040/0 (socket says 172.21.15.66:58040) 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 -- 172.21.15.66:0/3666434811 learned_addr learned my addr 172.21.15.66:0/3666434811 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e1371640 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce60eba50 0x561ce611b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce611b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 -- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce60eba50 msgr2=0x561ce611b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce60eba50 0x561ce611b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 -- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 msgr2=0x561ce611b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce611b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 -- 172.21.15.66:0/3666434811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce5e35920 con 0x561ce6118570 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce611b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 0x561ce60eb510 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f01d0089aa0 tx=0x7f01d00595d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db248a728b6c0ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01c67fc640 1 -- 172.21.15.66:0/3666434811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0021040 con 0x561ce6118570 2024-03-30T13:12:01.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01c67fc640 1 -- 172.21.15.66:0/3666434811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0062100 con 0x561ce6118570 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.491+0000 7f01c67fc640 1 -- 172.21.15.66:0/3666434811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0055c40 con 0x561ce6118570 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 -- 172.21.15.66:0/3666434811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 msgr2=0x561ce60eb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 0x561ce60eb510 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f01d0089aa0 tx=0x7f01d00595d0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 -- 172.21.15.66:0/3666434811 shutdown_connections 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce60eba50 0x561ce611b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce611b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3666434811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118570 0x561ce60eb510 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 -- 172.21.15.66:0/3666434811 >> 172.21.15.66:0/3666434811 conn(0x561ce60ed780 msgr2=0x561ce6136300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 -- 172.21.15.66:0/3666434811 shutdown_connections 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.492+0000 7f01e59c2740 1 -- 172.21.15.66:0/3666434811 wait complete. 2024-03-30T13:12:01.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.493+0000 7f01e59c2740 1 -- start start 2024-03-30T13:12:01.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.493+0000 7f01e59c2740 1 Processor -- start 2024-03-30T13:12:01.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 0x561ce61055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce611fc30 0x561ce614a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561ce60efe20 con 0x561ce6118950 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ce5ff19f0 con 0x561ce611fc30 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561ce60f2660 con 0x561ce6118570 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 10 client.0 _ll_register_callbacks cb 0x561ce5ff5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 0x561ce61055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 0x561ce61055e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58046/0 (socket says 172.21.15.66:58046) 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 -- 172.21.15.66:0/3544034069 learned_addr learned my addr 172.21.15.66:0/3544034069 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e1371640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce611fc30 0x561ce614a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 -- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 msgr2=0x561ce61050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 -- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce611fc30 msgr2=0x561ce614a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce611fc30 0x561ce614a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 -- 172.21.15.66:0/3544034069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce5d29210 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 0x561ce61055e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009890 tx=0x7f01c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74e18d3b34b169f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3544034069 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8054060 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3544034069 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f01c803a420 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.494+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d0062100 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3544034069 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f01c805a050 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 10 client.6102 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x561ce5e960b0 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3544034069 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0062100 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3544034069 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f01c805d020 con 0x561ce6118950 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 10 client.6102 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 10 client.6102 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 20 client.6102 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce6106230 0x561ce611e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 4 client.6102 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ce6062ad0 con 0x561ce6106230 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce6106230 0x561ce611e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce6106230 0x561ce611e7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933eaae9219504ee server_cookie=fa7fdfded889a68c in_seq=0 out_seq=0 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.495+0000 7f01bb7fe640 10 client.6102 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3544034069 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x561ce6062ad0 con 0x561ce6106230 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01bb7fe640 10 client.6102 handle_command_reply: tid=0 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 client.6102 shutdown 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 msgr2=0x561ce61055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 0x561ce61055e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009890 tx=0x7f01c8004730 comp rx=0 tx=0).stop 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 shutdown_connections 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce6106230 0x561ce611e7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce611fc30 0x561ce614a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce6118950 0x561ce61055e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01bb7fe640 0 client.6102 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3544034069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 >> 172.21.15.66:0/3544034069 conn(0x561ce60eba50 msgr2=0x561ce60ef540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 shutdown_connections 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 1 -- 172.21.15.66:0/3544034069 wait complete. 2024-03-30T13:12:01.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.496+0000 7f01e59c2740 20 client.6102 trim_cache size 0 max 16384 2024-03-30T13:12:01.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.527+0000 7f01e59c2740 1 Processor -- start 2024-03-30T13:12:01.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.527+0000 7f01e59c2740 1 -- start start 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce615ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce6139200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce6148a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e59c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc200fe0 con 0x561ce611fc30 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc200040 con 0x561ce6118950 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc15f190 con 0x561ce6118570 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e1371640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce6148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e1371640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce6148a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58050/0 (socket says 172.21.15.66:58050) 2024-03-30T13:12:01.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01e1371640 1 -- 172.21.15.66:0/141108964 learned_addr learned my addr 172.21.15.66:0/141108964 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.528+0000 7f01dbfff640 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce6139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e0b70640 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce615ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e1371640 1 -- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 msgr2=0x561ce615ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e1371640 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce615ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e1371640 1 -- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 msgr2=0x561ce6139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e1371640 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce6139200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e1371640 1 -- 172.21.15.66:0/141108964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce610bf50 con 0x561ce611fc30 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01e1371640 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce6148a40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f01d4002010 tx=0x7f01d4001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792e68ed471da1b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01c5ffb640 1 -- 172.21.15.66:0/141108964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d401a070 con 0x561ce611fc30 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01c5ffb640 1 -- 172.21.15.66:0/141108964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4016420 con 0x561ce611fc30 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.529+0000 7f01c5ffb640 1 -- 172.21.15.66:0/141108964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d40167d0 con 0x561ce611fc30 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 -- 172.21.15.66:0/141108964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 msgr2=0x561ce6148a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce6148a40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f01d4002010 tx=0x7f01d4001a10 comp rx=0 tx=0).stop 2024-03-30T13:12:01.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 -- 172.21.15.66:0/141108964 shutdown_connections 2024-03-30T13:12:01.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce6148a40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce6139200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 --2- 172.21.15.66:0/141108964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce615ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 -- 172.21.15.66:0/141108964 >> 172.21.15.66:0/141108964 conn(0x561ce60eba50 msgr2=0x561ce614a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 -- 172.21.15.66:0/141108964 shutdown_connections 2024-03-30T13:12:01.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.530+0000 7f01e59c2740 1 -- 172.21.15.66:0/141108964 wait complete. 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.531+0000 7f01e59c2740 1 -- start start 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.531+0000 7f01e59c2740 1 Processor -- start 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce61ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce61554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc1ffbe0 con 0x561ce611fc30 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ce60ecee0 con 0x561ce6118950 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561ce61b36f0 con 0x561ce6118570 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e59c2740 10 client.0 _ll_register_callbacks cb 0x7f01d0009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.532+0000 7f01e1371640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce61554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce61554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58054/0 (socket says 172.21.15.66:58054) 2024-03-30T13:12:01.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 -- 172.21.15.66:0/3147420586 learned_addr learned my addr 172.21.15.66:0/3147420586 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce61ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 -- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 msgr2=0x561ce61b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 -- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 msgr2=0x561ce61ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce61ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 -- 172.21.15.66:0/3147420586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce5d022d0 con 0x561ce611fc30 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01dbfff640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce61ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e1371640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce61554b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f01d4019000 tx=0x7f01d4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c6ffbfd86635ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3147420586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d401a040 con 0x561ce611fc30 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3147420586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f01d401f450 con 0x561ce611fc30 2024-03-30T13:12:01.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d4016420 con 0x561ce611fc30 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3147420586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f01d4032050 con 0x561ce611fc30 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e59c2740 10 client.6108 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.533+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x561ce61b3430 con 0x561ce611fc30 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3147420586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4016420 con 0x561ce611fc30 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3147420586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f01d4035020 con 0x561ce611fc30 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 10 client.6108 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 10 client.6108 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 20 client.6108 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce61559f0 0x561ce6116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 4 client.6108 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628334}] 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628334}) -- 0x561ce6120ab0 con 0x561ce61559f0 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce61559f0 0x561ce6116ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01bb7fe640 10 client.6108 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e0b70640 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce61559f0 0x561ce6116ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794793152ef6495d server_cookie=ba712bae4e13b860 in_seq=0 out_seq=0 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01bb7fe640 1 -- 172.21.15.66:0/3147420586 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3974 (crc 0 0 0) 0x561ce6120ab0 con 0x561ce61559f0 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01bb7fe640 10 client.6108 handle_command_reply: tid=0 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.534+0000 7f01e59c2740 1 client.6108 shutdown 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 msgr2=0x561ce61554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce61554b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f01d4019000 tx=0x7f01d4001250 comp rx=0 tx=0).stop 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 shutdown_connections 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561ce61559f0 0x561ce6116ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561ce611fc30 0x561ce61554b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ce6118950 0x561ce61ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01bb7fe640 0 client.6108 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 --2- 172.21.15.66:0/3147420586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561ce6118570 0x561ce61b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 >> 172.21.15.66:0/3147420586 conn(0x561ce60eba50 msgr2=0x561ce60e6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 shutdown_connections 2024-03-30T13:12:01.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 1 -- 172.21.15.66:0/3147420586 wait complete. 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e59c2740 20 client.6108 trim_cache size 0 max 16384 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/downcall.h", 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628334, 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:11:25.961410+0000", 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:11:24.452202+0000", 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:01.546 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 2953, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:11:24.452202+0000", 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:01.547 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2953, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.548 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.961410+0000" 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2953, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:11:25.961410+0000" 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 172, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:01.549 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 14, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:11:30.603021+0000", 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 122, 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:01.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:01.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:01.552 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:01.553 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:01.554 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:01.555 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:01.557 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f01c006c530 msgr2=0x7f01c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f01c006c530 0x7f01c006e9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01c8036000 tx=0x7f01c803c040 comp rx=0 tx=0).stop 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 msgr2=0x7f01dc0ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.535+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 0x7f01dc0ca7e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f01d000ebe0 tx=0x7f01d000c520 comp rx=0 tx=0).stop 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 shutdown_connections 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f01c006c530 0x7f01c006e9f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc0ba090 0x7f01dc0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01dc0b5dc0 0x7f01dc0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 --2- 172.21.15.66:0/1672743099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01dc06c620 0x7f01dc0ca7e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 >> 172.21.15.66:0/1672743099 conn(0x7f01dc0678f0 msgr2=0x7f01dc0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 shutdown_connections 2024-03-30T13:12:01.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.536+0000 7f01e2dd4640 1 -- 172.21.15.66:0/1672743099 wait complete. 2024-03-30T13:12:01.560 DEBUG:tasks.cephfs.test_uninlining:removing dir linux-5.4 2024-03-30T13:12:01.560 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && rm -rf linux-5.4/ 2024-03-30T13:12:01.774 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_length == 0 2024-03-30T13:12:01.774 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_version == CEPH_INLINE_NONE 2024-03-30T13:12:01.774 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, with snapshots 2024-03-30T13:12:01.774 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-30T13:12:01.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-30T13:12:01.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.880+0000 7f9dcb515640 1 Processor -- start 2024-03-30T13:12:01.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dcb515640 1 -- start start 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dcb515640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9dc40b5dc0 0x7f9dc40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dcb515640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc40b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dcb515640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc40b66e0 con 0x7f9dc40ba090 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dcb515640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc40d2d90 con 0x7f9dc40b5dc0 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dcb515640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc40d2ff0 con 0x7f9dc40b73d0 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.881+0000 7f9dc92b1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9dc40b5dc0 0x7f9dc40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.882+0000 7f9dc92b1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9dc40b5dc0 0x7f9dc40b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33090/0 (socket says 172.21.15.66:33090) 2024-03-30T13:12:01.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.882+0000 7f9dc92b1640 1 -- 172.21.15.66:0/1479418529 learned_addr learned my addr 172.21.15.66:0/1479418529 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.882+0000 7f9dc92b1640 1 -- 172.21.15.66:0/1479418529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 msgr2=0x7f9dc40b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.882+0000 7f9dc92b1640 1 --2- 172.21.15.66:0/1479418529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc40b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.882+0000 7f9dc92b1640 1 -- 172.21.15.66:0/1479418529 >> v1:172.21.15.117:6789/0 conn(0x7f9dc40ba090 legacy=0x7f9dc40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.882+0000 7f9dc92b1640 1 -- 172.21.15.66:0/1479418529 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc40d32d0 con 0x7f9dc40b5dc0 2024-03-30T13:12:01.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dc92b1640 1 --2- 172.21.15.66:0/1479418529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc40b61a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009e50 tx=0x7f9db402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21953fc214c559d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dc3fff640 1 -- 172.21.15.66:0/1479418529 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db402ca70 con 0x7f9dc40b5dc0 2024-03-30T13:12:01.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dc3fff640 1 -- 172.21.15.66:0/1479418529 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db402cc10 con 0x7f9dc40b5dc0 2024-03-30T13:12:01.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dc3fff640 1 -- 172.21.15.66:0/1479418529 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40343f0 con 0x7f9dc40b5dc0 2024-03-30T13:12:01.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 -- 172.21.15.66:0/1479418529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40b5dc0 msgr2=0x7f9dc40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 --2- 172.21.15.66:0/1479418529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc40b61a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009e50 tx=0x7f9db402bed0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 -- 172.21.15.66:0/1479418529 shutdown_connections 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 --2- 172.21.15.66:0/1479418529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc40b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 --2- 172.21.15.66:0/1479418529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc40b61a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 -- 172.21.15.66:0/1479418529 >> 172.21.15.66:0/1479418529 conn(0x7f9dc40678f0 msgr2=0x7f9dc40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.883+0000 7f9dcb515640 1 -- 172.21.15.66:0/1479418529 shutdown_connections 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.884+0000 7f9dcb515640 1 -- 172.21.15.66:0/1479418529 wait complete. 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 Processor -- start 2024-03-30T13:12:01.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 -- start start 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc4152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc41f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 0x7f9dc41605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc40d3b70 con 0x7f9dc40ba090 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc40d3890 con 0x7f9dc40b5dc0 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dcb515640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc40d4090 con 0x7f9dc40b73d0 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.885+0000 7f9dc9ab2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 0x7f9dc41605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 0x7f9dc41605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33094/0 (socket says 172.21.15.66:33094) 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 -- 172.21.15.66:0/3720977506 learned_addr learned my addr 172.21.15.66:0/3720977506 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc8ab0640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc41f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 -- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 msgr2=0x7f9dc41f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc92b1640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc4152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc41f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 -- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc40b5dc0 msgr2=0x7f9dc4152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc4152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 -- 172.21.15.66:0/3720977506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc4157fb0 con 0x7f9dc40ba090 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc8ab0640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc41f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc92b1640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc9ab2640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 0x7f9dc41605c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00b450 tx=0x7f9dbc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc015020 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc011890 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc41f8520 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc011c60 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.886+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc415edb0 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.887+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc4075890 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.888+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9dbc018020 con 0x7f9dc40ba090 2024-03-30T13:12:01.894 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.888+0000 7f9dc1ffb640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9db006c530 0x7f9db006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.888+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9dbc08b990 con 0x7f9dc40ba090 2024-03-30T13:12:01.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.888+0000 7f9dc92b1640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9db006c530 0x7f9db006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.889+0000 7f9dc92b1640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9db006c530 0x7f9db006e9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9db4035000 tx=0x7f9db4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:01.893+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9dbc01f020 con 0x7f9dc40ba090 2024-03-30T13:12:02.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.120+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f9dc40b9c10 con 0x7f9dc40ba090 2024-03-30T13:12:02.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.977+0000 7f9dc1ffb640 1 -- 172.21.15.66:0/3720977506 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f9dc41f8520 con 0x7f9dc40ba090 2024-03-30T13:12:02.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9db006c530 msgr2=0x7f9db006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9db006c530 0x7f9db006e9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9db4035000 tx=0x7f9db4030000 comp rx=0 tx=0).stop 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 msgr2=0x7f9dc41605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 0x7f9dc41605c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00b450 tx=0x7f9dbc00b910 comp rx=0 tx=0).stop 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 shutdown_connections 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f9db006c530 0x7f9db006e9f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ba090 0x7f9dc41605c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc40b73d0 0x7f9dc41f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 --2- 172.21.15.66:0/3720977506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc40b5dc0 0x7f9dc4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 >> 172.21.15.66:0/3720977506 conn(0x7f9dc40678f0 msgr2=0x7f9dc40b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 shutdown_connections 2024-03-30T13:12:02.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:02.980+0000 7f9dcb515640 1 -- 172.21.15.66:0/3720977506 wait complete. 2024-03-30T13:12:02.995 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-30T13:12:02.995 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-30T13:12:02.995 DEBUG:tasks.cephfs.test_uninlining:starting snapshotter thread 2024-03-30T13:12:02.995 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4 2024-03-30T13:12:03.020 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_0 2024-03-30T13:12:03.020 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-30T13:12:03.021 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-30T13:12:04.083 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_1 2024-03-30T13:12:05.105 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_2 2024-03-30T13:12:06.127 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_3 2024-03-30T13:12:07.160 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_4 2024-03-30T13:12:07.450 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-30T13:12:07.450 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-30T13:12:07.467 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph 2024-03-30T13:12:07.467 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2 2024-03-30T13:12:07.467 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs 2024-03-30T13:12:07.467 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-30T13:12:07.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:07.467 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-30T13:12:07.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:07.522 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-30T13:12:07.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:07.579 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-30T13:12:07.634 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-30T13:12:07.634 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-30T13:12:07.701 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/Makefile 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/io.h 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/Makefile 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/acl.h 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:12:07.702 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:12:07.703 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:12:07.704 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-30T13:12:07.704 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-30T13:12:07.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:07.704 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-30T13:12:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.849+0000 7fc6fc20d640 1 Processor -- start 2024-03-30T13:12:07.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.849+0000 7fc6fc20d640 1 -- start start 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6fc20d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc6f415e440 0x7fc6f40b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6fc20d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc6f406d4a0 0x7fc6f40d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6fc20d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f40d2910 con 0x7fc6f40d3ef0 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6fc20d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f40d3520 con 0x7fc6f406d4a0 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6fc20d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f40d2bf0 con 0x7fc6f415e440 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc6f406d4a0 0x7fc6f40d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc6f406d4a0 0x7fc6f40d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33104/0 (socket says 172.21.15.66:33104) 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3552962500 learned_addr learned my addr 172.21.15.66:0/3552962500 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6fa7aa640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc6f40d3ef0 0x7fc6f40cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52534/0 (socket says 172.21.15.66:52534) 2024-03-30T13:12:07.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/3552962500 >> v2:172.21.15.117:3300/0 conn(0x7fc6f415e440 0x7fc6f40b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3552962500 >> v2:172.21.15.117:3300/0 conn(0x7fc6f415e440 msgr2=0x7fc6f40b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/3552962500 >> v2:172.21.15.117:3300/0 conn(0x7fc6f415e440 0x7fc6f40b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3552962500 >> v1:172.21.15.66:6789/0 conn(0x7fc6f40d3ef0 legacy=0x7fc6f40cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f8fa7640 1 -- 172.21.15.66:0/3552962500 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4279041181 0 0) 0x7fc6f40d2910 con 0x7fc6f40d3ef0 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3552962500 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f40d2ed0 con 0x7fc6f406d4a0 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.850+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/3552962500 >> v2:172.21.15.117:3300/0 conn(0x7fc6f415e440 0x7fc6f40b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.851+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/3552962500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40d39b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406d430 tx=0x7fc6e402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85bc21abedcb71a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.851+0000 7fc6f8fa7640 1 -- 172.21.15.66:0/3552962500 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e402f850 con 0x7fc6f406d4a0 2024-03-30T13:12:07.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.851+0000 7fc6f8fa7640 1 -- 172.21.15.66:0/3552962500 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e402f9f0 con 0x7fc6f406d4a0 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.851+0000 7fc6f8fa7640 1 -- 172.21.15.66:0/3552962500 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e402fdc0 con 0x7fc6f406d4a0 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 -- 172.21.15.66:0/3552962500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f406d4a0 msgr2=0x7fc6f40d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/3552962500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40d39b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc6f406d430 tx=0x7fc6e402ba30 comp rx=0 tx=0).stop 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 -- 172.21.15.66:0/3552962500 shutdown_connections 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/3552962500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40d39b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/3552962500 >> v2:172.21.15.117:3300/0 conn(0x7fc6f415e440 0x7fc6f40b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 -- 172.21.15.66:0/3552962500 >> 172.21.15.66:0/3552962500 conn(0x7fc6f40676b0 msgr2=0x7fc6f4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 -- 172.21.15.66:0/3552962500 shutdown_connections 2024-03-30T13:12:07.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.852+0000 7fc6fc20d640 1 -- 172.21.15.66:0/3552962500 wait complete. 2024-03-30T13:12:07.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.853+0000 7fc6fc20d640 1 Processor -- start 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.853+0000 7fc6fc20d640 1 -- start start 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fc20d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fc20d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f415e440 0x7fc6f4161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fc20d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 0x7fc6f4142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fc20d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f415e820 con 0x7fc6f41432d0 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fc20d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f40ba220 con 0x7fc6f406d4a0 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fc20d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f415e9a0 con 0x7fc6f415e440 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 0x7fc6f4142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 0x7fc6f4142600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33114/0 (socket says 172.21.15.66:33114) 2024-03-30T13:12:07.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/4066463602 learned_addr learned my addr 172.21.15.66:0/4066463602 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f415e440 0x7fc6f4161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f415e440 msgr2=0x7fc6f4161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f415e440 0x7fc6f4161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f406d4a0 msgr2=0x7fc6f40cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/4066463602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f415bf90 con 0x7fc6f41432d0 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f415e440 0x7fc6f4161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:07.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 0x7fc6f4142600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c800 tx=0x7fc6ec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6ea7fc640 1 -- 172.21.15.66:0/4066463602 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec018070 con 0x7fc6f41432d0 2024-03-30T13:12:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.854+0000 7fc6ea7fc640 1 -- 172.21.15.66:0/4066463602 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ec00a5e0 con 0x7fc6f41432d0 2024-03-30T13:12:07.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.855+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f415c270 con 0x7fc6f41432d0 2024-03-30T13:12:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.855+0000 7fc6ea7fc640 1 -- 172.21.15.66:0/4066463602 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00a9d0 con 0x7fc6f41432d0 2024-03-30T13:12:07.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.855+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f40cb980 con 0x7fc6f41432d0 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.856+0000 7fc6ea7fc640 1 -- 172.21.15.66:0/4066463602 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc6ec01b070 con 0x7fc6f41432d0 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.857+0000 7fc6ea7fc640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc6d806c560 0x7fc6d806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.857+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc6d806c560 0x7fc6d806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.857+0000 7fc6ea7fc640 1 -- 172.21.15.66:0/4066463602 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc6ec08aee0 con 0x7fc6f41432d0 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.857+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc6d806c560 0x7fc6d806ea20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 Processor -- start 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 -- start start 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea42e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 0x55cea43108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea42e1000 0x55cea4310df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cea424a410 con 0x55cea430df00 2024-03-30T13:12:07.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cea41e8a80 con 0x55cea430db20 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.858+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55cea42463c0 con 0x55cea42e1000 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 0x55cea43108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 0x55cea43108b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33120/0 (socket says 172.21.15.66:33120) 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 -- 172.21.15.66:0/728935013 learned_addr learned my addr 172.21.15.66:0/728935013 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea42e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea42e1000 0x55cea4310df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 -- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea42e1000 msgr2=0x55cea4310df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea42e1000 0x55cea4310df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 -- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 msgr2=0x55cea42e0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea42e0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 -- 172.21.15.66:0/728935013 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea402a7b0 con 0x55cea430df00 2024-03-30T13:12:07.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea42e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 0x55cea43108b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009960 tx=0x7fc6e4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c03abbbf9fe4ad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6d77fe640 1 -- 172.21.15.66:0/728935013 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4047070 con 0x55cea430df00 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6d77fe640 1 -- 172.21.15.66:0/728935013 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e402fd20 con 0x55cea430df00 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.859+0000 7fc6d77fe640 1 -- 172.21.15.66:0/728935013 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4042690 con 0x55cea430df00 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 -- 172.21.15.66:0/728935013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 msgr2=0x55cea43108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 0x55cea43108b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009960 tx=0x7fc6e4008030 comp rx=0 tx=0).stop 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 -- 172.21.15.66:0/728935013 shutdown_connections 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea42e1000 0x55cea4310df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430df00 0x55cea43108b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/728935013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea42e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 -- 172.21.15.66:0/728935013 >> 172.21.15.66:0/728935013 conn(0x55cea42e2d30 msgr2=0x55cea432b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 -- 172.21.15.66:0/728935013 shutdown_connections 2024-03-30T13:12:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.860+0000 7fc6fedfa740 1 -- 172.21.15.66:0/728935013 wait complete. 2024-03-30T13:12:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.861+0000 7fc6fedfa740 1 -- start start 2024-03-30T13:12:07.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.861+0000 7fc6fedfa740 1 Processor -- start 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea433bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea4312d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea433fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cea42e53d0 con 0x55cea43064a0 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cea41e6740 con 0x55cea430db20 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55cea42e7c10 con 0x55cea430df00 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fedfa740 10 client.0 _ll_register_callbacks cb 0x55cea41f3b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea433fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea433fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33126/0 (socket says 172.21.15.66:33126) 2024-03-30T13:12:07.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/858226812 learned_addr learned my addr 172.21.15.66:0/858226812 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea4312d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea433bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 msgr2=0x55cea4312d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea4312d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 msgr2=0x55cea433bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea433bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 -- 172.21.15.66:0/858226812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea408af40 con 0x55cea43064a0 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea433fef0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00a430 tx=0x7fc6ec058ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d39aac45547278 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.862+0000 7fc6d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:07.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/858226812 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec082da0 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/858226812 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc6ec00ab70 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e402fd20 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/858226812 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc6ec00ad70 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 10 client.6129 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55cea3f1e0a0 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/858226812 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e402fd20 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/858226812 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fc6ec057ff0 con 0x55cea43064a0 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 10 client.6129 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 10 client.6129 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 20 client.6129 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea43132b0 0x55cea430fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 4 client.6129 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cea4257950 con 0x55cea43132b0 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.863+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea43132b0 0x55cea430fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6d4ff9640 10 client.6129 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea43132b0 0x55cea430fde0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca947d180af2d139 server_cookie=ccc3e29f6d9a8322 in_seq=0 out_seq=0 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/858226812 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55cea4257950 con 0x55cea43132b0 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6d4ff9640 10 client.6129 handle_command_reply: tid=0 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 client.6129 shutdown 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 msgr2=0x55cea433fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea433fef0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00a430 tx=0x7fc6ec058ba0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 shutdown_connections 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea43132b0 0x55cea430fde0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea433fef0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea4312d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/858226812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea433bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.864+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 >> 172.21.15.66:0/858226812 conn(0x55cea42e1000 msgr2=0x55cea432dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.865+0000 7fc6d4ff9640 0 client.6129 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.865+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 shutdown_connections 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.865+0000 7fc6fedfa740 1 -- 172.21.15.66:0/858226812 wait complete. 2024-03-30T13:12:07.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.865+0000 7fc6fedfa740 20 client.6129 trim_cache size 0 max 16384 2024-03-30T13:12:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.895+0000 7fc6fedfa740 1 Processor -- start 2024-03-30T13:12:07.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.895+0000 7fc6fedfa740 1 -- start start 2024-03-30T13:12:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea43064a0 0x55cea432ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 0x55cea430e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430df00 0x55cea430ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4201170 con 0x55cea430db20 2024-03-30T13:12:07.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f42001f0 con 0x55cea430df00 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f41ff580 con 0x55cea43064a0 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 0x55cea430e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 0x55cea430e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33128/0 (socket says 172.21.15.66:33128) 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3143049289 learned_addr learned my addr 172.21.15.66:0/3143049289 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea43064a0 msgr2=0x55cea432ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea43064a0 0x55cea432ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.903 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6fa7aa640 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430df00 0x55cea430ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430df00 msgr2=0x55cea430ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430df00 0x55cea430ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.896+0000 7fc6f97a8640 1 -- 172.21.15.66:0/3143049289 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea4305c80 con 0x55cea430db20 2024-03-30T13:12:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.897+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 0x55cea430e9e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4038000 tx=0x7fc6e4031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a219c23c0d1764 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.897+0000 7fc6d6ffd640 1 -- 172.21.15.66:0/3143049289 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4052070 con 0x55cea430db20 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.897+0000 7fc6d6ffd640 1 -- 172.21.15.66:0/3143049289 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e402fd20 con 0x55cea430db20 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.897+0000 7fc6d6ffd640 1 -- 172.21.15.66:0/3143049289 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4047040 con 0x55cea430db20 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.897+0000 7fc6fedfa740 1 -- 172.21.15.66:0/3143049289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 msgr2=0x55cea430e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.897+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 0x55cea430e9e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4038000 tx=0x7fc6e4031000 comp rx=0 tx=0).stop 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 -- 172.21.15.66:0/3143049289 shutdown_connections 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430df00 0x55cea430ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea430db20 0x55cea430e9e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/3143049289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea43064a0 0x55cea432ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 -- 172.21.15.66:0/3143049289 >> 172.21.15.66:0/3143049289 conn(0x55cea42e1000 msgr2=0x55cea433bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 -- 172.21.15.66:0/3143049289 shutdown_connections 2024-03-30T13:12:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 -- 172.21.15.66:0/3143049289 wait complete. 2024-03-30T13:12:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 -- start start 2024-03-30T13:12:07.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.898+0000 7fc6fedfa740 1 Processor -- start 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea43a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea4329ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea432a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cea433c3b0 con 0x55cea43064a0 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ffbd0 con 0x55cea430db20 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55cea4339970 con 0x55cea430df00 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6fedfa740 10 client.0 _ll_register_callbacks cb 0x7fc6ec0018d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6f9fa9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea43a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6f9fa9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea43a6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33130/0 (socket says 172.21.15.66:33130) 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6f9fa9640 1 -- 172.21.15.66:0/335186067 learned_addr learned my addr 172.21.15.66:0/335186067 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea4329ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6f9fa9640 1 -- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 msgr2=0x55cea432a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.899+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea432a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6f9fa9640 1 -- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 msgr2=0x55cea4329ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea4329ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6f9fa9640 1 -- 172.21.15.66:0/335186067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea432ab90 con 0x55cea43064a0 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea4329ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6f9fa9640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea43a6f30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000de30 tx=0x7fc6f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d95c53a7139bb79 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/335186067 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f002c060 con 0x55cea43064a0 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/335186067 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc6f0012ad0 con 0x55cea43064a0 2024-03-30T13:12:07.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e402fd20 con 0x55cea43064a0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/335186067 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc6f0032050 con 0x55cea43064a0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 10 client.6135 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55cea43387b0 con 0x55cea43064a0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/335186067 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e402fd20 con 0x55cea43064a0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/335186067 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fc6f0035020 con 0x55cea43064a0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 10 client.6135 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 10 client.6135 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 20 client.6135 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea432c1f0 0x55cea434ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 4 client.6135 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55cea4251ca0 con 0x55cea432c1f0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.900+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea432c1f0 0x55cea434ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.901+0000 7fc6d4ff9640 10 client.6135 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.901+0000 7fc6f97a8640 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea432c1f0 0x55cea434ef20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95577084ea08d41 server_cookie=c310c7f77930cc9f in_seq=0 out_seq=0 2024-03-30T13:12:07.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6d4ff9640 1 -- 172.21.15.66:0/335186067 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cea4251ca0 con 0x55cea432c1f0 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6d4ff9640 10 client.6135 handle_command_reply: tid=0 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 client.6135 shutdown 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 msgr2=0x55cea43a6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea43a6f30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000de30 tx=0x7fc6f00046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 shutdown_connections 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cea432c1f0 0x55cea434ef20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6d4ff9640 0 client.6135 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cea430df00 0x55cea432a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cea430db20 0x55cea4329ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 --2- 172.21.15.66:0/335186067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cea43064a0 0x55cea43a6f30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 >> 172.21.15.66:0/335186067 conn(0x55cea42e1000 msgr2=0x55cea43391e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 shutdown_connections 2024-03-30T13:12:07.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 1 -- 172.21.15.66:0/335186067 wait complete. 2024-03-30T13:12:07.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.903+0000 7fc6fedfa740 20 client.6135 trim_cache size 0 max 16384 2024-03-30T13:12:07.915 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:07.915 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2024-03-30T13:12:07.915 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "93fb2aec-7718-42c4-9900-db6335963981", 2024-03-30T13:12:07.915 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2024-03-30T13:12:07.915 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc6d806c560 msgr2=0x7fc6d806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc6d806c560 0x7fc6d806ea20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).stop 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 msgr2=0x7fc6f4142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 0x7fc6f4142600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c800 tx=0x7fc6ec00ccc0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 shutdown_connections 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc6d806c560 0x7fc6d806ea20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6f41432d0 0x7fc6f4142600 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f415e440 0x7fc6f4161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 --2- 172.21.15.66:0/4066463602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f406d4a0 0x7fc6f40cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 >> 172.21.15.66:0/4066463602 conn(0x7fc6f40676b0 msgr2=0x7fc6f40d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 shutdown_connections 2024-03-30T13:12:07.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:07.904+0000 7fc6fc20d640 1 -- 172.21.15.66:0/4066463602 wait complete. 2024-03-30T13:12:07.921 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': '93fb2aec-7718-42c4-9900-db6335963981', 'mode': 'asynchronous'} 2024-03-30T13:12:07.921 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-30T13:12:07.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:07.922 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:12:08.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.076+0000 7ff7ae2d9640 1 Processor -- start 2024-03-30T13:12:08.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.076+0000 7ff7ae2d9640 1 -- start start 2024-03-30T13:12:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7ae2d9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff7a80d1980 0x7ff7a815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7ae2d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a8071aa0 0x7ff7a815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7ae2d9640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a815c990 con 0x7ff7a80b7950 2024-03-30T13:12:08.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7ae2d9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a8071f30 con 0x7ff7a80d1980 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7ae2d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a80ccb70 con 0x7ff7a8071aa0 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7a7fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff7a80b7950 0x7ff7a80d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:52548/0 (socket says 172.21.15.66:52548) 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1324176064 learned_addr learned my addr 172.21.15.66:0/1324176064 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7ac876640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a8071aa0 0x7ff7a815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/1324176064 >> v2:172.21.15.117:3300/0 conn(0x7ff7a80d1980 0x7ff7a815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/1324176064 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211996367 0 0) 0x7ff7a815c990 con 0x7ff7a80b7950 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.077+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/1324176064 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff790003520 con 0x7ff7a80b7950 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.078+0000 7ff7a77fe640 1 -- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a8071aa0 msgr2=0x7ff7a815fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.078+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a8071aa0 0x7ff7a815fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.078+0000 7ff7a77fe640 1 -- 172.21.15.66:0/1324176064 >> v1:172.21.15.66:6789/0 conn(0x7ff7a80b7950 legacy=0x7ff7a80d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.078+0000 7ff7a77fe640 1 -- 172.21.15.66:0/1324176064 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a815cc00 con 0x7ff7a80d1980 2024-03-30T13:12:08.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.078+0000 7ff7ac876640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a8071aa0 0x7ff7a815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:08.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.079+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a80d1980 0x7ff7a815d400 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0097a0 tx=0x7ff79c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc71c156aed7cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.079+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/1324176064 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c035020 con 0x7ff7a80d1980 2024-03-30T13:12:08.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.079+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/1324176064 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c031830 con 0x7ff7a80d1980 2024-03-30T13:12:08.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.079+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/1324176064 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c031c00 con 0x7ff7a80d1980 2024-03-30T13:12:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a80d1980 msgr2=0x7ff7a815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a80d1980 0x7ff7a815d400 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0097a0 tx=0x7ff79c02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/1324176064 shutdown_connections 2024-03-30T13:12:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a8071aa0 0x7ff7a815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/1324176064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a80d1980 0x7ff7a815d400 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/1324176064 >> 172.21.15.66:0/1324176064 conn(0x7ff7a80b3cb0 msgr2=0x7ff7a80b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/1324176064 shutdown_connections 2024-03-30T13:12:08.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.080+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/1324176064 wait complete. 2024-03-30T13:12:08.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.082+0000 7ff7ae2d9640 1 Processor -- start 2024-03-30T13:12:08.086 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.082+0000 7ff7ae2d9640 1 -- start start 2024-03-30T13:12:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7ae2d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a8071aa0 0x7ff7a81f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7ae2d9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 0x7ff7a8156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7ae2d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7a80d1980 0x7ff7a80b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7ae2d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8070790 con 0x7ff7a80d1980 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7ae2d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a80b7d30 con 0x7ff7a8071aa0 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7ae2d9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a80cc7b0 con 0x7ff7a80b7950 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 0x7ff7a8156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 0x7ff7a8156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:42928/0 (socket says 172.21.15.66:42928) 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.083+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2512582487 learned_addr learned my addr 172.21.15.66:0/2512582487 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a8071aa0 0x7ff7a81f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7ac876640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7a80d1980 0x7ff7a80b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a8071aa0 msgr2=0x7ff7a81f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a8071aa0 0x7ff7a81f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7a80d1980 msgr2=0x7ff7a80b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7a80d1980 0x7ff7a80b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2512582487 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a81f3f60 con 0x7ff7a80b7950 2024-03-30T13:12:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7ac876640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7a80d1980 0x7ff7a80b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 0x7ff7a8156640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff79c02bee0 tx=0x7ff79c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a4ff9640 1 -- 172.21.15.66:0/2512582487 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c035040 con 0x7ff7a80b7950 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a4ff9640 1 -- 172.21.15.66:0/2512582487 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c03b920 con 0x7ff7a80b7950 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7a4ff9640 1 -- 172.21.15.66:0/2512582487 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c03bd10 con 0x7ff7a80b7950 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.084+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a81f4240 con 0x7ff7a80b7950 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.085+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81f4800 con 0x7ff7a80b7950 2024-03-30T13:12:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.087+0000 7ff7a4ff9640 1 -- 172.21.15.66:0/2512582487 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff79c038020 con 0x7ff7a80b7950 2024-03-30T13:12:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.087+0000 7ff7a4ff9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff79406c530 0x7ff79406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.087+0000 7ff7a4ff9640 1 -- 172.21.15.66:0/2512582487 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff79c0ad910 con 0x7ff7a80b7950 2024-03-30T13:12:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.087+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff79406c530 0x7ff79406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.088+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff79406c530 0x7ff79406e9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e4a0 tx=0x7ff798007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.088+0000 7ff7b0fa6740 1 Processor -- start 2024-03-30T13:12:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.088+0000 7ff7b0fa6740 1 -- start start 2024-03-30T13:12:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e1c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e4bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e1c6e0 0x55b543e4c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7b0fa6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b543d86880 con 0x55b543e49200 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7b0fa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b543d2e8f0 con 0x55b543e495e0 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7b0fa6740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b543d82830 con 0x55b543e1c6e0 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e1c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e1c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33158/0 (socket says 172.21.15.66:33158) 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a7fff640 1 -- 172.21.15.66:0/2802262094 learned_addr learned my addr 172.21.15.66:0/2802262094 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e4bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7ac876640 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e1c6e0 0x55b543e4c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e1c6e0 msgr2=0x55b543e4c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e1c6e0 0x55b543e4c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2802262094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 msgr2=0x55b543e1c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e1c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 -- 172.21.15.66:0/2802262094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b543d39140 con 0x55b543e495e0 2024-03-30T13:12:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.090+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e4bf90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0abc60 tx=0x7ff79c07b0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b71074965686e33 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.091+0000 7ff78dffb640 1 -- 172.21.15.66:0/2802262094 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c07fa30 con 0x55b543e495e0 2024-03-30T13:12:08.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.091+0000 7ff78dffb640 1 -- 172.21.15.66:0/2802262094 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c084260 con 0x55b543e495e0 2024-03-30T13:12:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.091+0000 7ff78dffb640 1 -- 172.21.15.66:0/2802262094 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c079980 con 0x55b543e495e0 2024-03-30T13:12:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 msgr2=0x55b543e4bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e4bf90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0abc60 tx=0x7ff79c07b0a0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2802262094 shutdown_connections 2024-03-30T13:12:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e1c6e0 0x55b543e4c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e4bf90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2802262094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2802262094 >> 172.21.15.66:0/2802262094 conn(0x55b543e1e410 msgr2=0x55b543e66f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2802262094 shutdown_connections 2024-03-30T13:12:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.092+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2802262094 wait complete. 2024-03-30T13:12:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.093+0000 7ff7b0fa6740 1 -- start start 2024-03-30T13:12:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.094+0000 7ff7b0fa6740 1 Processor -- start 2024-03-30T13:12:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e3b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e4be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33170/0 (socket says 172.21.15.66:33170) 2024-03-30T13:12:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e4d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:42938/0 (socket says 172.21.15.66:42938) 2024-03-30T13:12:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7a7fff640 1 -- 172.21.15.66:0/2839216764 learned_addr learned my addr 172.21.15.66:0/2839216764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7ac876640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e3b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.096+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b543d86880 con 0x55b543e49200 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.097+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b543d41cf0 con 0x55b543e4d9f0 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.097+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b543d2e8f0 con 0x55b543e495e0 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.097+0000 7ff7b0fa6740 10 client.0 _ll_register_callbacks cb 0x55b543d33f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7a7fff640 1 -- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 msgr2=0x55b543e4d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e4d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7a7fff640 1 -- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 msgr2=0x55b543e3b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e3b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7a7fff640 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b543b647d0 con 0x55b543e49200 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e4be90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff79801a720 tx=0x7ff798000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1931a4702f7f7584 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff7ac876640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e3b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff786ffd640 1 -- 172.21.15.66:0/2839216764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980016a0 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.098+0000 7ff786ffd640 1 -- 172.21.15.66:0/2839216764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff798001840 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c084260 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff786ffd640 1 -- 172.21.15.66:0/2839216764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff798001a40 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 10 client.6147 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55b543a9a3f0 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff786ffd640 1 -- 172.21.15.66:0/2839216764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79802b070 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff786ffd640 1 -- 172.21.15.66:0/2839216764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7ff798030450 con 0x55b543e49200 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 10 client.6147 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 10 client.6147 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:08.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 20 client.6147 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:08.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.099+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e76e60 0x55b543e64b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.100+0000 7ff7b0fa6740 4 client.6147 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:08.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.100+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff79c084260 con 0x55b543e76e60 2024-03-30T13:12:08.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.100+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e76e60 0x55b543e64b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.101+0000 7ff786ffd640 10 client.6147 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:08.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.101+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e76e60 0x55b543e64b40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc39318dcce71d5f server_cookie=c1328dbd3728064b in_seq=0 out_seq=0 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.103+0000 7ff786ffd640 1 -- 172.21.15.66:0/2839216764 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x7ff79c084260 con 0x55b543e76e60 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.103+0000 7ff786ffd640 10 client.6147 handle_command_reply: tid=0 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.103+0000 7ff7b0fa6740 1 client.6147 shutdown 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 msgr2=0x55b543e4be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e4be90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff79801a720 tx=0x7ff798000ed0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 shutdown_connections 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e76e60 0x55b543e64b40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e3b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff786ffd640 0 client.6147 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e4d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/2839216764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543e4be90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 >> 172.21.15.66:0/2839216764 conn(0x55b543e1c6e0 msgr2=0x55b543e222f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 shutdown_connections 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/2839216764 wait complete. 2024-03-30T13:12:08.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.104+0000 7ff7b0fa6740 20 client.6147 trim_cache size 0 max 16384 2024-03-30T13:12:08.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.133+0000 7ff7b0fa6740 1 Processor -- start 2024-03-30T13:12:08.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.133+0000 7ff7b0fa6740 1 -- start start 2024-03-30T13:12:08.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543eda030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543eda030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543eda030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33184/0 (socket says 172.21.15.66:33184) 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1024228694 learned_addr learned my addr 172.21.15.66:0/1024228694 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e45ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e6a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81fcd30 con 0x55b543e49200 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81fbe00 con 0x55b543e4d9f0 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.134+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a806e610 con 0x55b543e495e0 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e45ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7ac876640 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e6a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 msgr2=0x55b543e45ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e45ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 msgr2=0x55b543e6a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e6a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1024228694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b543a5a0f0 con 0x55b543e49200 2024-03-30T13:12:08.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7ac876640 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e6a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:08.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543eda030 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff798001380 tx=0x7ff79801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fae7a10d1013ac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.135+0000 7ff78d7fa640 1 -- 172.21.15.66:0/1024228694 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980016a0 con 0x55b543e49200 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.136+0000 7ff78d7fa640 1 -- 172.21.15.66:0/1024228694 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79802b040 con 0x55b543e49200 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.136+0000 7ff78d7fa640 1 -- 172.21.15.66:0/1024228694 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79801a070 con 0x55b543e49200 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.136+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 msgr2=0x55b543eda030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.136+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543eda030 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff798001380 tx=0x7ff79801a000 comp rx=0 tx=0).stop 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 shutdown_connections 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e4d9f0 0x55b543e6a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e495e0 0x55b543e45ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1024228694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543eda030 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 >> 172.21.15.66:0/1024228694 conn(0x55b543e1c6e0 msgr2=0x55b543e4fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 shutdown_connections 2024-03-30T13:12:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.137+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1024228694 wait complete. 2024-03-30T13:12:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.138+0000 7ff7b0fa6740 1 -- start start 2024-03-30T13:12:08.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.138+0000 7ff7b0fa6740 1 Processor -- start 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.139+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543ed96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e67010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7b0fa6740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e4d9f0 0x55b543e67550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7b0fa6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b543e697d0 con 0x55b543e49200 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e67010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543ed96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e67010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:33322/0 (socket says 172.21.15.66:33322) 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a77fe640 1 -- 172.21.15.66:0/1712910617 learned_addr learned my addr 172.21.15.66:0/1712910617 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:08.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7ac876640 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e4d9f0 0x55b543e67550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81fb7b0 con 0x55b543e495e0 2024-03-30T13:12:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b543e46410 con 0x55b543e4d9f0 2024-03-30T13:12:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7b0fa6740 10 client.0 _ll_register_callbacks cb 0x7ff79800de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e4d9f0 msgr2=0x55b543e67550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e4d9f0 0x55b543e67550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 msgr2=0x55b543e67010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e67010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.140+0000 7ff7a7fff640 1 -- 172.21.15.66:0/1712910617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b543e7ce30 con 0x55b543e49200 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543ed96f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff798027000 tx=0x7ff798018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0ad10b101b77bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff786ffd640 1 -- 172.21.15.66:0/1712910617 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79800d230 con 0x55b543e49200 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff786ffd640 1 -- 172.21.15.66:0/1712910617 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff798006860 con 0x55b543e49200 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79802b040 con 0x55b543e49200 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff786ffd640 1 -- 172.21.15.66:0/1712910617 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff79804b050 con 0x55b543e49200 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.141+0000 7ff7b0fa6740 10 client.6153 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55b543e7da10 con 0x55b543e49200 2024-03-30T13:12:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff786ffd640 1 -- 172.21.15.66:0/1712910617 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff798006a60 con 0x55b543e49200 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff786ffd640 1 -- 172.21.15.66:0/1712910617 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7ff79804e020 con 0x55b543e49200 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff7b0fa6740 10 client.6153 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff7b0fa6740 10 client.6153 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff7b0fa6740 20 client.6153 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.142+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e3afb0 0x55b543e39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.143+0000 7ff7b0fa6740 4 client.6153 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.143+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e3afb0 0x55b543e39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.143+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff79802b040 con 0x55b543e3afb0 2024-03-30T13:12:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.144+0000 7ff7a77fe640 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e3afb0 0x55b543e39a90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0bee0480428e07 server_cookie=62c1a6eba466701e in_seq=0 out_seq=0 2024-03-30T13:12:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.144+0000 7ff786ffd640 10 client.6153 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.146+0000 7ff786ffd640 1 -- 172.21.15.66:0/1712910617 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+850 (crc 0 0 0) 0x7ff79802b040 con 0x55b543e3afb0 2024-03-30T13:12:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.146+0000 7ff786ffd640 10 client.6153 handle_command_reply: tid=0 2024-03-30T13:12:08.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.146+0000 7ff7b0fa6740 1 client.6153 shutdown 2024-03-30T13:12:08.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 msgr2=0x55b543ed96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543ed96f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff798027000 tx=0x7ff798018000 comp rx=0 tx=0).stop 2024-03-30T13:12:08.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 shutdown_connections 2024-03-30T13:12:08.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b543e3afb0 0x55b543e39a90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b543e4d9f0 0x55b543e67550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff786ffd640 0 client.6153 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b543e495e0 0x55b543e67010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 --2- 172.21.15.66:0/1712910617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b543e49200 0x55b543ed96f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 >> 172.21.15.66:0/1712910617 conn(0x55b543e1c6e0 msgr2=0x55b543e4f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 shutdown_connections 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.147+0000 7ff7b0fa6740 1 -- 172.21.15.66:0/1712910617 wait complete. 2024-03-30T13:12:08.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7b0fa6740 20 client.6153 trim_cache size 0 max 16384 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "93fb2aec-7718-42c4-9900-db6335963981": { 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "93fb2aec-7718-42c4-9900-db6335963981", 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive" 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_stats": { 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "93fb2aec-7718-42c4-9900-db6335963981": { 2024-03-30T13:12:08.153 INFO:teuthology.orchestra.run.smithi066.stdout: "start_time": "2024-03-30T13:12:07.901806+0000", 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 0, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_passed": 0, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_failed": 0, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_skipped": 0 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "c7ed1273-d477-4170-a25d-5b1410281d2c": { 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "start_time": "2024-03-30T13:11:25.735190+0000", 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 75, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_passed": 75, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_failed": 0, 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_skipped": 0 2024-03-30T13:12:08.154 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:08.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:08.155 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff79406c530 msgr2=0x7ff79406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff79406c530 0x7ff79406e9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e4a0 tx=0x7ff798007d30 comp rx=0 tx=0).stop 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 msgr2=0x7ff7a8156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 0x7ff7a8156640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff79c02bee0 tx=0x7ff79c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 shutdown_connections 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff79406c530 0x7ff79406e9f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7a80d1980 0x7ff7a80b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7a80b7950 0x7ff7a8156640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 --2- 172.21.15.66:0/2512582487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7a8071aa0 0x7ff7a81f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 >> 172.21.15.66:0/2512582487 conn(0x7ff7a80b3cb0 msgr2=0x7ff7a815b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 shutdown_connections 2024-03-30T13:12:08.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:08.148+0000 7ff7ae2d9640 1 -- 172.21.15.66:0/2512582487 wait complete. 2024-03-30T13:12:08.163 INFO:tasks.cephfs.filesystem:scrub status for tag:93fb2aec-7718-42c4-9900-db6335963981 - {'path': '/', 'tag': '93fb2aec-7718-42c4-9900-db6335963981', 'options': 'recursive'} 2024-03-30T13:12:08.211 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_5 2024-03-30T13:12:09.237 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_6 2024-03-30T13:12:10.261 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_7 2024-03-30T13:12:11.284 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_8 2024-03-30T13:12:12.310 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_9 2024-03-30T13:12:12.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:12.427 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:38.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:38.165 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:12:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.289+0000 7fa80dd00640 1 Processor -- start 2024-03-30T13:12:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.289+0000 7fa80dd00640 1 -- start start 2024-03-30T13:12:38.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa80dd00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa8080afd60 0x7fa80815f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa80dd00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa808162070 0x7fa8080b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa80dd00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa80806e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa80dd00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa80806fac0 con 0x7fa808162070 2024-03-30T13:12:38.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa80dd00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa808162610 con 0x7fa8080afd60 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa80dd00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa8081628f0 con 0x7fa8080b3c90 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa808162070 0x7fa8080b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa808162070 0x7fa8080b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52946/0 (socket says 172.21.15.66:52946) 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 -- 172.21.15.66:0/1841331871 learned_addr learned my addr 172.21.15.66:0/1841331871 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa8077fe640 1 --2- 172.21.15.66:0/1841331871 >> v2:172.21.15.117:3300/0 conn(0x7fa8080afd60 0x7fa80815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa807fff640 1 --2- 172.21.15.66:0/1841331871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa80806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 -- 172.21.15.66:0/1841331871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 msgr2=0x7fa80806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 --2- 172.21.15.66:0/1841331871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa80806e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 -- 172.21.15.66:0/1841331871 >> v2:172.21.15.117:3300/0 conn(0x7fa8080afd60 msgr2=0x7fa80815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 --2- 172.21.15.66:0/1841331871 >> v2:172.21.15.117:3300/0 conn(0x7fa8080afd60 0x7fa80815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.290+0000 7fa806ffd640 1 -- 172.21.15.66:0/1841331871 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa808162bd0 con 0x7fa808162070 2024-03-30T13:12:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.291+0000 7fa806ffd640 1 --2- 172.21.15.66:0/1841331871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa808162070 0x7fa8080b3750 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa808067e70 tx=0x7fa7fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07e7c1d9e16b170 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.291+0000 7fa8067fc640 1 -- 172.21.15.66:0/1841331871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc035020 con 0x7fa808162070 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.291+0000 7fa8067fc640 1 -- 172.21.15.66:0/1841331871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc031870 con 0x7fa808162070 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.291+0000 7fa8067fc640 1 -- 172.21.15.66:0/1841331871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc031c40 con 0x7fa808162070 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 -- 172.21.15.66:0/1841331871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa808162070 msgr2=0x7fa8080b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 --2- 172.21.15.66:0/1841331871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa808162070 0x7fa8080b3750 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa808067e70 tx=0x7fa7fc02b910 comp rx=0 tx=0).stop 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 -- 172.21.15.66:0/1841331871 shutdown_connections 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 --2- 172.21.15.66:0/1841331871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa80806e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 --2- 172.21.15.66:0/1841331871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa808162070 0x7fa8080b3750 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 --2- 172.21.15.66:0/1841331871 >> v2:172.21.15.117:3300/0 conn(0x7fa8080afd60 0x7fa80815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 -- 172.21.15.66:0/1841331871 >> 172.21.15.66:0/1841331871 conn(0x7fa8080d0840 msgr2=0x7fa8080d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 -- 172.21.15.66:0/1841331871 shutdown_connections 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.292+0000 7fa80dd00640 1 -- 172.21.15.66:0/1841331871 wait complete. 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.293+0000 7fa80dd00640 1 Processor -- start 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.293+0000 7fa80dd00640 1 -- start start 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa80dd00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 0x7fa808143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa80dd00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa8081e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa80dd00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808162070 0x7fa808153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa80dd00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa80806e5d0 con 0x7fa8080afd60 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa80dd00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa808162eb0 con 0x7fa808162070 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa80dd00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa80806e750 con 0x7fa8080b3c90 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 0x7fa808143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 0x7fa808143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52956/0 (socket says 172.21.15.66:52956) 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 -- 172.21.15.66:0/2407987215 learned_addr learned my addr 172.21.15.66:0/2407987215 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa806ffd640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa8081e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa807fff640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808162070 0x7fa808153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 -- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 msgr2=0x7fa8081e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa8081e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 -- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808162070 msgr2=0x7fa808153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808162070 0x7fa808153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 -- 172.21.15.66:0/2407987215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8081e7fc0 con 0x7fa8080afd60 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.294+0000 7fa8077fe640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 0x7fa808143300 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e890 tx=0x7fa7f800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.295+0000 7fa7effff640 1 -- 172.21.15.66:0/2407987215 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8017070 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.295+0000 7fa7effff640 1 -- 172.21.15.66:0/2407987215 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f800c940 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.295+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8081e82a0 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.295+0000 7fa7effff640 1 -- 172.21.15.66:0/2407987215 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800cd10 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.295+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8081e8860 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.296+0000 7fa7effff640 1 -- 172.21.15.66:0/2407987215 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa7f801a070 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.297+0000 7fa7effff640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7f406c530 0x7fa7f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.297+0000 7fa806ffd640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7f406c530 0x7fa7f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.297+0000 7fa7effff640 1 -- 172.21.15.66:0/2407987215 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa7f808bb50 con 0x7fa8080afd60 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.298+0000 7fa806ffd640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7f406c530 0x7fa7f406e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc036000 tx=0x7fa7fc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.298+0000 7fa8109c2740 1 Processor -- start 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.298+0000 7fa8109c2740 1 -- start start 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec47f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 0x558a7ec484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8109c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558a7eb82880 con 0x558a7ec186e0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558a7eb2a8f0 con 0x558a7ec455e0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558a7eb7e830 con 0x558a7ec45200 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 0x558a7ec484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 0x558a7ec484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52966/0 (socket says 172.21.15.66:52966) 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 -- 172.21.15.66:0/717235193 learned_addr learned my addr 172.21.15.66:0/717235193 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa806ffd640 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec47f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa8077fe640 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 -- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 msgr2=0x558a7ec181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 -- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 msgr2=0x558a7ec47f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec47f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 -- 172.21.15.66:0/717235193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7eb35140 con 0x558a7ec186e0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa806ffd640 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec47f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.299+0000 7fa807fff640 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 0x558a7ec484d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa8000076b0 tx=0x7fa800009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00be7807aea6e6a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.300+0000 7fa7ed7fa640 1 -- 172.21.15.66:0/717235193 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800019020 con 0x558a7ec186e0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.300+0000 7fa7ed7fa640 1 -- 172.21.15.66:0/717235193 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa800015600 con 0x558a7ec186e0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.300+0000 7fa7ed7fa640 1 -- 172.21.15.66:0/717235193 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000159d0 con 0x558a7ec186e0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 -- 172.21.15.66:0/717235193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 msgr2=0x558a7ec484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 0x558a7ec484d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa8000076b0 tx=0x7fa800009750 comp rx=0 tx=0).stop 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 -- 172.21.15.66:0/717235193 shutdown_connections 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec186e0 0x558a7ec484d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec47f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 --2- 172.21.15.66:0/717235193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 -- 172.21.15.66:0/717235193 >> 172.21.15.66:0/717235193 conn(0x558a7ec1a410 msgr2=0x558a7ec62f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 -- 172.21.15.66:0/717235193 shutdown_connections 2024-03-30T13:12:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 -- 172.21.15.66:0/717235193 wait complete. 2024-03-30T13:12:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 -- start start 2024-03-30T13:12:38.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.301+0000 7fa8109c2740 1 Processor -- start 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ec49e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec72e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec38880 0x558a7ec4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558a7ec1cab0 con 0x558a7ec45200 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558a7eb3dcf0 con 0x558a7ec455e0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558a7ec1f2f0 con 0x558a7ec38880 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8109c2740 10 client.0 _ll_register_callbacks cb 0x558a7eb2ff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ec49e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ec49e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52968/0 (socket says 172.21.15.66:52968) 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8077fe640 1 -- 172.21.15.66:0/3307830526 learned_addr learned my addr 172.21.15.66:0/3307830526 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa806ffd640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec72e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa807fff640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec38880 0x558a7ec4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.302+0000 7fa8077fe640 1 -- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec38880 msgr2=0x558a7ec4ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8077fe640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec38880 0x558a7ec4ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8077fe640 1 -- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 msgr2=0x558a7ec72e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8077fe640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec72e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8077fe640 1 -- 172.21.15.66:0/3307830526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7e9607d0 con 0x558a7ec45200 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8077fe640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ec49e50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8089f60 tx=0x7fa7f805a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5a4d2dd39a3c93 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa7e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa7e67fc640 1 -- 172.21.15.66:0/3307830526 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8088060 con 0x558a7ec45200 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa7e67fc640 1 -- 172.21.15.66:0/3307830526 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa7f8089a60 con 0x558a7ec45200 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa800015600 con 0x558a7ec45200 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa7e67fc640 1 -- 172.21.15.66:0/3307830526 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa7f8076050 con 0x558a7ec45200 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 10 client.6165 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:38.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x558a7e8963f0 con 0x558a7ec45200 2024-03-30T13:12:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa7e67fc640 1 -- 172.21.15.66:0/3307830526 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa800015600 con 0x558a7ec45200 2024-03-30T13:12:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa7e67fc640 1 -- 172.21.15.66:0/3307830526 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fa7f808b040 con 0x558a7ec45200 2024-03-30T13:12:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 10 client.6165 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 10 client.6165 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 20 client.6165 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec76f70 0x558a7ec773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 4 client.6165 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a7eb8fdc0 con 0x558a7ec76f70 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.303+0000 7fa806ffd640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec76f70 0x558a7ec773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.304+0000 7fa806ffd640 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec76f70 0x558a7ec773b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6807a1d9fa3c121 server_cookie=710f5b260a168733 in_seq=0 out_seq=0 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.304+0000 7fa7e67fc640 10 client.6165 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa7e67fc640 1 -- 172.21.15.66:0/3307830526 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x558a7eb8fdc0 con 0x558a7ec76f70 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa7e67fc640 10 client.6165 handle_command_reply: tid=0 2024-03-30T13:12:38.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 client.6165 shutdown 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 msgr2=0x558a7ec49e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ec49e50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8089f60 tx=0x7fa7f805a110 comp rx=0 tx=0).stop 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 shutdown_connections 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec76f70 0x558a7ec773b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec38880 0x558a7ec4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec72e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa7e67fc640 0 client.6165 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 --2- 172.21.15.66:0/3307830526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ec49e50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 >> 172.21.15.66:0/3307830526 conn(0x558a7ec186e0 msgr2=0x558a7ec1c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 shutdown_connections 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 1 -- 172.21.15.66:0/3307830526 wait complete. 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.305+0000 7fa8109c2740 20 client.6165 trim_cache size 0 max 16384 2024-03-30T13:12:38.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.323+0000 7fa8109c2740 1 Processor -- start 2024-03-30T13:12:38.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.323+0000 7fa8109c2740 1 -- start start 2024-03-30T13:12:38.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 0x558a7ec1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec1c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec42d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8109c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa808201210 con 0x558a7ec38880 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8081f8eb0 con 0x558a7ec455e0 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa808155290 con 0x558a7ec45200 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 0x558a7ec1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 0x558a7ec1bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52980/0 (socket says 172.21.15.66:52980) 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 -- 172.21.15.66:0/2309745333 learned_addr learned my addr 172.21.15.66:0/2309745333 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 -- 172.21.15.66:0/2309745333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 msgr2=0x558a7ec1c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec1c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 -- 172.21.15.66:0/2309745333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 msgr2=0x558a7ec42d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec42d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 -- 172.21.15.66:0/2309745333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7e8560f0 con 0x558a7ec38880 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.324+0000 7fa8077fe640 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 0x558a7ec1bd00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8055db0 tx=0x7fa7f806f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2fc50875a4bcc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa7ecff9640 1 -- 172.21.15.66:0/2309745333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f806f070 con 0x558a7ec38880 2024-03-30T13:12:38.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa7ecff9640 1 -- 172.21.15.66:0/2309745333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f807c070 con 0x558a7ec38880 2024-03-30T13:12:38.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa7ecff9640 1 -- 172.21.15.66:0/2309745333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8021040 con 0x558a7ec38880 2024-03-30T13:12:38.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 -- 172.21.15.66:0/2309745333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 msgr2=0x558a7ec1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 0x558a7ec1bd00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8055db0 tx=0x7fa7f806f000 comp rx=0 tx=0).stop 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 -- 172.21.15.66:0/2309745333 shutdown_connections 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec455e0 0x558a7ec42d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec45200 0x558a7ec1c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 --2- 172.21.15.66:0/2309745333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec38880 0x558a7ec1bd00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 -- 172.21.15.66:0/2309745333 >> 172.21.15.66:0/2309745333 conn(0x558a7ec186e0 msgr2=0x558a7ec4e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.325+0000 7fa8109c2740 1 -- 172.21.15.66:0/2309745333 shutdown_connections 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.326+0000 7fa8109c2740 1 -- 172.21.15.66:0/2309745333 wait complete. 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.326+0000 7fa8109c2740 1 -- start start 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.326+0000 7fa8109c2740 1 Processor -- start 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec38880 0x558a7ecc5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ecdfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec455e0 0x558a7ec1bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558a7ec1ef10 con 0x558a7ec45200 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8081f8860 con 0x558a7ec38880 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558a7eb8fdc0 con 0x558a7ec455e0 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8109c2740 10 client.0 _ll_register_callbacks cb 0x7fa7f800ceb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ecdfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ecdfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52990/0 (socket says 172.21.15.66:52990) 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 -- 172.21.15.66:0/4190738761 learned_addr learned my addr 172.21.15.66:0/4190738761 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8077fe640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec38880 0x558a7ecc5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa807fff640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec455e0 0x558a7ec1bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 -- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec455e0 msgr2=0x558a7ec1bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec455e0 0x558a7ec1bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 -- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec38880 msgr2=0x558a7ecc5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec38880 0x558a7ecc5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 -- 172.21.15.66:0/4190738761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7ecdf3f0 con 0x558a7ec45200 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa8077fe640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec38880 0x558a7ecc5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa806ffd640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ecdfe90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc000c00 tx=0x7fa7fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa687f84bb888db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.327+0000 7fa7e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa7e67fc640 1 -- 172.21.15.66:0/4190738761 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc035040 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa7e67fc640 1 -- 172.21.15.66:0/4190738761 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fa7fc03ac00 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f807c070 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa7e67fc640 1 -- 172.21.15.66:0/4190738761 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa7fc062050 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 10 client.6171 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x558a7ec17af0 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa7e67fc640 1 -- 172.21.15.66:0/4190738761 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f807c070 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa7e67fc640 1 -- 172.21.15.66:0/4190738761 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fa7fc065020 con 0x558a7ec45200 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 10 client.6171 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 10 client.6171 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 20 client.6171 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec71370 0x558a7ec71750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 4 client.6171 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a7eb8a110 con 0x558a7ec71370 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.328+0000 7fa8077fe640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec71370 0x558a7ec71750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa7e67fc640 10 client.6171 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8077fe640 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec71370 0x558a7ec71750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f838d940d79ee08a server_cookie=54f2d073cedcfb44 in_seq=0 out_seq=0 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa7e67fc640 1 -- 172.21.15.66:0/4190738761 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+662 (crc 0 0 0) 0x558a7eb8a110 con 0x558a7ec71370 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa7e67fc640 10 client.6171 handle_command_reply: tid=0 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 client.6171 shutdown 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 msgr2=0x558a7ecdfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ecdfe90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc000c00 tx=0x7fa7fc004730 comp rx=0 tx=0).stop 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 shutdown_connections 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558a7ec71370 0x558a7ec71750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558a7ec455e0 0x558a7ec1bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a7ec45200 0x558a7ecdfe90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa7e67fc640 0 client.6171 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 --2- 172.21.15.66:0/4190738761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a7ec38880 0x558a7ecc5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 >> 172.21.15.66:0/4190738761 conn(0x558a7ec186e0 msgr2=0x558a7ec76090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 shutdown_connections 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 1 -- 172.21.15.66:0/4190738761 wait complete. 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.329+0000 7fa8109c2740 20 client.6171 trim_cache size 0 max 16384 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {}, 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_stats": { 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "93fb2aec-7718-42c4-9900-db6335963981": { 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "start_time": "2024-03-30T13:12:07.901806+0000", 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 74, 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_passed": 74, 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_failed": 0, 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_skipped": 0 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "c7ed1273-d477-4170-a25d-5b1410281d2c": { 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "start_time": "2024-03-30T13:11:25.735190+0000", 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_started": 75, 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_passed": 75, 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_failed": 0, 2024-03-30T13:12:38.341 INFO:teuthology.orchestra.run.smithi066.stdout: "uninline_skipped": 0 2024-03-30T13:12:38.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:38.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:38.341 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7f406c530 msgr2=0x7fa7f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7f406c530 0x7fa7f406e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc036000 tx=0x7fa7fc03c040 comp rx=0 tx=0).stop 2024-03-30T13:12:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 msgr2=0x7fa808143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 0x7fa808143300 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e890 tx=0x7fa7f800ed50 comp rx=0 tx=0).stop 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 shutdown_connections 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa7f406c530 0x7fa7f406e9f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808162070 0x7fa808153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8080b3c90 0x7fa8081e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 --2- 172.21.15.66:0/2407987215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8080afd60 0x7fa808143300 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 >> 172.21.15.66:0/2407987215 conn(0x7fa8080d0840 msgr2=0x7fa80806eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 shutdown_connections 2024-03-30T13:12:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:38.330+0000 7fa80dd00640 1 -- 172.21.15.66:0/2407987215 wait complete. 2024-03-30T13:12:38.349 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T13:12:38.349 DEBUG:tasks.cephfs.test_uninlining:joining snapshotter thread 2024-03-30T13:12:38.350 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_0 2024-03-30T13:12:38.376 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_1 2024-03-30T13:12:38.441 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_2 2024-03-30T13:12:38.509 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_3 2024-03-30T13:12:38.574 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_4 2024-03-30T13:12:38.637 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_5 2024-03-30T13:12:38.697 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_6 2024-03-30T13:12:38.758 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_7 2024-03-30T13:12:38.819 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_8 2024-03-30T13:12:38.879 DEBUG:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_9 2024-03-30T13:12:38.939 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-30T13:12:38.939 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:12:38.939 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-30T13:12:38.954 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628358 linux-5.4/fs/ceph/Kconfig 2024-03-30T13:12:38.954 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-30T13:12:38.955 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-30T13:12:39.010 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628359 linux-5.4/fs/ceph/Makefile 2024-03-30T13:12:39.010 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:12:39.010 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:12:39.064 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628365 linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:12:39.065 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-30T13:12:39.065 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-30T13:12:39.120 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628372 linux-5.4/fs/ceph/io.h 2024-03-30T13:12:39.120 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:12:39.120 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:12:39.175 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628374 linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:12:39.175 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:12:39.176 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-30T13:12:39.230 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628386 linux-5.4/fs/ext2/Kconfig 2024-03-30T13:12:39.230 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:12:39.231 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:12:39.285 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628404 linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:12:39.285 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:12:39.286 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:12:39.340 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628402 linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:12:39.341 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-30T13:12:39.341 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-30T13:12:39.395 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628387 linux-5.4/fs/ext2/Makefile 2024-03-30T13:12:39.396 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:12:39.396 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-30T13:12:39.451 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628399 linux-5.4/fs/ext2/symlink.c 2024-03-30T13:12:39.452 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-30T13:12:39.452 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-30T13:12:39.506 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628389 linux-5.4/fs/ext2/acl.h 2024-03-30T13:12:39.507 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:12:39.507 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:12:39.561 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628403 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:12:39.561 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:12:39.562 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-30T13:12:39.616 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628401 linux-5.4/fs/ext2/xattr.h 2024-03-30T13:12:39.616 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:12:39.617 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:12:39.671 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628406 linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:12:39.671 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:12:39.671 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:12:39.725 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628419 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:12:39.726 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:12:39.726 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:12:39.780 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628426 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:12:39.781 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:12:39.781 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:12:39.835 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628409 linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:12:39.836 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:12:39.836 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:12:39.890 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628417 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:12:39.891 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:12:39.891 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:12:39.945 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628421 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:12:39.946 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:12:39.946 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:12:40.002 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628422 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:12:40.003 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:12:40.003 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-30T13:12:40.058 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628407 linux-5.4/fs/orangefs/Makefile 2024-03-30T13:12:40.058 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:12:40.058 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:12:40.113 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628430 linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:12:40.113 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:12:40.113 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:12:40.168 INFO:teuthology.orchestra.run.smithi066.stdout:1099511628412 linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:12:40.169 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-30T13:12:40.169 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511628358 2024-03-30T13:12:40.169 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628358 2024-03-30T13:12:40.169 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628358 2024-03-30T13:12:40.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.316+0000 7f0d9039d640 1 Processor -- start 2024-03-30T13:12:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.317+0000 7f0d9039d640 1 -- start start 2024-03-30T13:12:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.317+0000 7f0d9039d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.317+0000 7f0d9039d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d880ba090 0x7f0d880d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.317+0000 7f0d9039d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d880b66e0 con 0x7f0d8806c620 2024-03-30T13:12:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.317+0000 7f0d9039d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f0d880d2ff0 con 0x7f0d880ba090 2024-03-30T13:12:40.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.317+0000 7f0d9039d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0d880d32d0 con 0x7f0d880b5dc0 2024-03-30T13:12:40.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8d938640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d880ba090 0x7f0d880d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8e93a640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0d8806c620 0x7f0d8806cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:43152/0 (socket says 172.21.15.66:43152) 2024-03-30T13:12:40.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8e93a640 1 -- 172.21.15.66:0/3340905137 learned_addr learned my addr 172.21.15.66:0/3340905137 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8d938640 1 -- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 msgr2=0x7f0d880b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8d938640 1 --2- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8d938640 1 -- 172.21.15.66:0/3340905137 >> v1:172.21.15.117:6789/0 conn(0x7f0d8806c620 legacy=0x7f0d8806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8d938640 1 -- 172.21.15.66:0/3340905137 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d880d35b0 con 0x7f0d880ba090 2024-03-30T13:12:40.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.318+0000 7f0d8e139640 1 --2- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.319+0000 7f0d8d938640 1 --2- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880d2680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097a0 tx=0x7f0d7802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afc0033e0f35a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.319+0000 7f0d8d137640 1 -- 172.21.15.66:0/3340905137 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78035020 con 0x7f0d880ba090 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.319+0000 7f0d8d137640 1 -- 172.21.15.66:0/3340905137 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d78031810 con 0x7f0d880ba090 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.319+0000 7f0d8d137640 1 -- 172.21.15.66:0/3340905137 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78031be0 con 0x7f0d880ba090 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 -- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 msgr2=0x7f0d880d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 --2- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880d2680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097a0 tx=0x7f0d7802b8b0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 -- 172.21.15.66:0/3340905137 shutdown_connections 2024-03-30T13:12:40.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 --2- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880d2680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 --2- 172.21.15.66:0/3340905137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 -- 172.21.15.66:0/3340905137 >> 172.21.15.66:0/3340905137 conn(0x7f0d880678f0 msgr2=0x7f0d880d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 -- 172.21.15.66:0/3340905137 shutdown_connections 2024-03-30T13:12:40.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.320+0000 7f0d9039d640 1 -- 172.21.15.66:0/3340905137 wait complete. 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.321+0000 7f0d9039d640 1 Processor -- start 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.321+0000 7f0d9039d640 1 -- start start 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d9039d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 0x7f0d88151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d9039d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d9039d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d9039d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880d4090 con 0x7f0d8806c620 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d9039d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880d3b70 con 0x7f0d880ba090 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d9039d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0d880b83e0 con 0x7f0d880b5dc0 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 0x7f0d88151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 0x7f0d88151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52996/0 (socket says 172.21.15.66:52996) 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 -- 172.21.15.66:0/427586895 learned_addr learned my addr 172.21.15.66:0/427586895 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8d938640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 -- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 msgr2=0x7f0d880aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 -- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 msgr2=0x7f0d880cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 -- 172.21.15.66:0/427586895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8815f980 con 0x7f0d8806c620 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8d938640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:40.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.322+0000 7f0d8e139640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 0x7f0d88151820 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ec10 tx=0x7f0d8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.323+0000 7f0d7effd640 1 -- 172.21.15.66:0/427586895 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84017070 con 0x7f0d8806c620 2024-03-30T13:12:40.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.323+0000 7f0d7effd640 1 -- 172.21.15.66:0/427586895 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8400cc30 con 0x7f0d8806c620 2024-03-30T13:12:40.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.323+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8815f3c0 con 0x7f0d8806c620 2024-03-30T13:12:40.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.323+0000 7f0d7effd640 1 -- 172.21.15.66:0/427586895 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84013470 con 0x7f0d8806c620 2024-03-30T13:12:40.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.323+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8815fcb0 con 0x7f0d8806c620 2024-03-30T13:12:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.324+0000 7f0d7effd640 1 -- 172.21.15.66:0/427586895 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0d8401a070 con 0x7f0d8806c620 2024-03-30T13:12:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.325+0000 7f0d7effd640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0d6c06c530 0x7f0d6c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.325+0000 7f0d7effd640 1 -- 172.21.15.66:0/427586895 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0d8408b760 con 0x7f0d8806c620 2024-03-30T13:12:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.325+0000 7f0d8d938640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0d6c06c530 0x7f0d6c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.325+0000 7f0d8d938640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0d6c06c530 0x7f0d6c06e9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d78036000 tx=0x7f0d7803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.326+0000 7f0d92f88740 1 Processor -- start 2024-03-30T13:12:40.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.327+0000 7f0d92f88740 1 -- start start 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832570 0x55ebf7805510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7835300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 0x55ebf7835840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d92f88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ebf776f590 con 0x55ebf7805a50 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ebf770dd30 con 0x55ebf7832950 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55ebf776b540 con 0x55ebf7832570 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 0x55ebf7835840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 0x55ebf7835840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53006/0 (socket says 172.21.15.66:53006) 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 -- 172.21.15.66:0/581722636 learned_addr learned my addr 172.21.15.66:0/581722636 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8d938640 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7835300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e139640 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832570 0x55ebf7805510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 -- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832570 msgr2=0x55ebf7805510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832570 0x55ebf7805510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 -- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 msgr2=0x55ebf7835300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7835300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8e93a640 1 -- 172.21.15.66:0/581722636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf754f920 con 0x55ebf7805a50 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.328+0000 7f0d8d938640 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7835300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.329+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 0x55ebf7835840 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0d800020b0 tx=0x7f0d8000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b195a336e7e237d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.329+0000 7f0d6bfff640 1 -- 172.21.15.66:0/581722636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80019070 con 0x55ebf7805a50 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.329+0000 7f0d6bfff640 1 -- 172.21.15.66:0/581722636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8000edf0 con 0x55ebf7805a50 2024-03-30T13:12:40.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.329+0000 7f0d6bfff640 1 -- 172.21.15.66:0/581722636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80014680 con 0x55ebf7805a50 2024-03-30T13:12:40.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 -- 172.21.15.66:0/581722636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 msgr2=0x55ebf7835840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 0x55ebf7835840 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0d800020b0 tx=0x7f0d8000e530 comp rx=0 tx=0).stop 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 -- 172.21.15.66:0/581722636 shutdown_connections 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7805a50 0x55ebf7835840 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7835300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 --2- 172.21.15.66:0/581722636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832570 0x55ebf7805510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 -- 172.21.15.66:0/581722636 >> 172.21.15.66:0/581722636 conn(0x55ebf7807780 msgr2=0x55ebf7850300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 -- 172.21.15.66:0/581722636 shutdown_connections 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.330+0000 7f0d92f88740 1 -- 172.21.15.66:0/581722636 wait complete. 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.331+0000 7f0d92f88740 1 -- start start 2024-03-30T13:12:40.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.331+0000 7f0d92f88740 1 Processor -- start 2024-03-30T13:12:40.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 0x55ebf7850960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7838f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf7837790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ebf7809e20 con 0x55ebf7832570 2024-03-30T13:12:40.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ebf770b9f0 con 0x55ebf7832950 2024-03-30T13:12:40.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55ebf780c660 con 0x55ebf781f5a0 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 10 client.0 _ll_register_callbacks cb 0x55ebf770fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 0x55ebf7850960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 0x55ebf7850960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53020/0 (socket says 172.21.15.66:53020) 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 -- 172.21.15.66:0/2200950468 learned_addr learned my addr 172.21.15.66:0/2200950468 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8d938640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7838f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf7837790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 -- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 msgr2=0x55ebf7837790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf7837790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 -- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 msgr2=0x55ebf7838f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7838f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 -- 172.21.15.66:0/2200950468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf7443210 con 0x55ebf7832570 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d8e139640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 0x55ebf7850960 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0d84061c10 tx=0x7f0d84059d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d59b9b8c5fb755d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d697fa640 1 -- 172.21.15.66:0/2200950468 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84071020 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d697fa640 1 -- 172.21.15.66:0/2200950468 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0d84058bb0 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.333+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8000edf0 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d697fa640 1 -- 172.21.15.66:0/2200950468 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0d84021040 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 10 client.6180 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55ebf75b00b0 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d697fa640 1 -- 172.21.15.66:0/2200950468 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8000edf0 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d697fa640 1 -- 172.21.15.66:0/2200950468 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f0d84024060 con 0x55ebf7832570 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 10 client.6180 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 10 client.6180 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 20 client.6180 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78609c0 0x55ebf7863ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 4 client.6180 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:40.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf777cad0 con 0x55ebf78609c0 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d8d938640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78609c0 0x55ebf7863ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d8d938640 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78609c0 0x55ebf7863ae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=661d7765989f24d4 server_cookie=ea6bfe34064fd9a0 in_seq=0 out_seq=0 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.334+0000 7f0d697fa640 10 client.6180 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.335+0000 7f0d697fa640 1 -- 172.21.15.66:0/2200950468 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55ebf777cad0 con 0x55ebf78609c0 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.335+0000 7f0d697fa640 10 client.6180 handle_command_reply: tid=0 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.335+0000 7f0d92f88740 1 client.6180 shutdown 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.335+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 msgr2=0x55ebf7850960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.335+0000 7f0d92f88740 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 0x55ebf7850960 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0d84061c10 tx=0x7f0d84059d20 comp rx=0 tx=0).stop 2024-03-30T13:12:40.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 shutdown_connections 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78609c0 0x55ebf7863ae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf7837790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832950 0x55ebf7838f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d697fa640 0 client.6180 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 --2- 172.21.15.66:0/2200950468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832570 0x55ebf7850960 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 >> 172.21.15.66:0/2200950468 conn(0x55ebf7805a50 msgr2=0x55ebf7853780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 shutdown_connections 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 1 -- 172.21.15.66:0/2200950468 wait complete. 2024-03-30T13:12:40.344 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.336+0000 7f0d92f88740 20 client.6180 trim_cache size 0 max 16384 2024-03-30T13:12:40.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.350+0000 7f0d92f88740 1 Processor -- start 2024-03-30T13:12:40.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.350+0000 7f0d92f88740 1 -- start start 2024-03-30T13:12:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 0x55ebf7869d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf786a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832950 0x55ebf7838740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d92f88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88201130 con 0x55ebf781f5a0 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88200150 con 0x55ebf7832570 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0d881ff4e0 con 0x55ebf7832950 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 0x55ebf7869d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 0x55ebf7869d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53032/0 (socket says 172.21.15.66:53032) 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 -- 172.21.15.66:0/407396088 learned_addr learned my addr 172.21.15.66:0/407396088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8d938640 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf786a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 -- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832950 msgr2=0x55ebf7838740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832950 0x55ebf7838740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 -- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 msgr2=0x55ebf786a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf786a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.351+0000 7f0d8e139640 1 -- 172.21.15.66:0/407396088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf741c2d0 con 0x55ebf781f5a0 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d8d938640 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf786a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d8e139640 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 0x55ebf7869d10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ef30 tx=0x7f0d8401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53af1adf4b5f7f8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d6b7fe640 1 -- 172.21.15.66:0/407396088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8407d070 con 0x55ebf781f5a0 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d6b7fe640 1 -- 172.21.15.66:0/407396088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d84071ba0 con 0x55ebf781f5a0 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d6b7fe640 1 -- 172.21.15.66:0/407396088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84071070 con 0x55ebf781f5a0 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 -- 172.21.15.66:0/407396088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 msgr2=0x55ebf7869d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 0x55ebf7869d10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ef30 tx=0x7f0d8401e000 comp rx=0 tx=0).stop 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 -- 172.21.15.66:0/407396088 shutdown_connections 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf7832950 0x55ebf7838740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf786a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 --2- 172.21.15.66:0/407396088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf781f5a0 0x55ebf7869d10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 -- 172.21.15.66:0/407396088 >> 172.21.15.66:0/407396088 conn(0x55ebf7805a50 msgr2=0x55ebf780c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 -- 172.21.15.66:0/407396088 shutdown_connections 2024-03-30T13:12:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.352+0000 7f0d92f88740 1 -- 172.21.15.66:0/407396088 wait complete. 2024-03-30T13:12:40.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.353+0000 7f0d92f88740 1 -- start start 2024-03-30T13:12:40.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.353+0000 7f0d92f88740 1 Processor -- start 2024-03-30T13:12:40.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf78b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf7837700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 0x55ebf7837c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ebf78747d0 con 0x55ebf7832950 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ffb30 con 0x55ebf7832570 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55ebf7804ad0 con 0x55ebf781f5a0 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d92f88740 10 client.0 _ll_register_callbacks cb 0x7f0d8400ef60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 0x55ebf7837c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 0x55ebf7837c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53044/0 (socket says 172.21.15.66:53044) 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 -- 172.21.15.66:0/1098152104 learned_addr learned my addr 172.21.15.66:0/1098152104 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8d938640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf7837700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e139640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf78b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 -- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 msgr2=0x55ebf78b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf78b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 -- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 msgr2=0x55ebf7837700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf7837700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 -- 172.21.15.66:0/1098152104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf7839670 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d8e93a640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 0x55ebf7837c40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0d80017040 tx=0x7f0d80001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f57be27f5acd7f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.354+0000 7f0d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d697fa640 1 -- 172.21.15.66:0/1098152104 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80023070 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d697fa640 1 -- 172.21.15.66:0/1098152104 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f0d80004d60 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d84071ba0 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d697fa640 1 -- 172.21.15.66:0/1098152104 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0d8002d050 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 10 client.6186 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55ebf78043a0 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d697fa640 1 -- 172.21.15.66:0/1098152104 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d84071ba0 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d697fa640 1 -- 172.21.15.66:0/1098152104 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f0d80030020 con 0x55ebf7832950 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 10 client.6186 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 10 client.6186 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 20 client.6186 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78b5700 0x55ebf78d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 4 client.6186 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628358}] 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628358}) -- 0x55ebf7776e20 con 0x55ebf78b5700 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.355+0000 7f0d8e139640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78b5700 0x55ebf78d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d697fa640 10 client.6186 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d8e139640 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78b5700 0x55ebf78d0020 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4cd046b2caf45e server_cookie=80950259eb48c9ce in_seq=0 out_seq=0 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d697fa640 1 -- 172.21.15.66:0/1098152104 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3974 (crc 0 0 0) 0x55ebf7776e20 con 0x55ebf78b5700 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d697fa640 10 client.6186 handle_command_reply: tid=0 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 client.6186 shutdown 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 msgr2=0x55ebf7837c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 0x55ebf7837c40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0d80017040 tx=0x7f0d80001250 comp rx=0 tx=0).stop 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 shutdown_connections 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55ebf78b5700 0x55ebf78d0020 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ebf7832950 0x55ebf7837c40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ebf7832570 0x55ebf7837700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d697fa640 0 client.6186 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 --2- 172.21.15.66:0/1098152104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ebf781f5a0 0x55ebf78b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 >> 172.21.15.66:0/1098152104 conn(0x55ebf7805a50 msgr2=0x55ebf7835180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 shutdown_connections 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 1 -- 172.21.15.66:0/1098152104 wait complete. 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.356+0000 7f0d92f88740 20 client.6186 trim_cache size 0 max 16384 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/Kconfig", 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628358, 2024-03-30T13:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:07.923734+0000", 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.486819+0000", 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1481, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.486819+0000", 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:40.368 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:40.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1481, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.923734+0000" 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1481, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.923734+0000" 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 254, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 2, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:07.963509+0000", 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 56, 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:40.375 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:40.375 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:40.375 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:40.375 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.379 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:40.380 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0d6c06c530 msgr2=0x7f0d6c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0d6c06c530 0x7f0d6c06e9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d78036000 tx=0x7f0d7803c040 comp rx=0 tx=0).stop 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 msgr2=0x7f0d88151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 0x7f0d88151820 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ec10 tx=0x7f0d8400c520 comp rx=0 tx=0).stop 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 shutdown_connections 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0d6c06c530 0x7f0d6c06e9f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d880ba090 0x7f0d880cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d880b5dc0 0x7f0d880aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 --2- 172.21.15.66:0/427586895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d8806c620 0x7f0d88151820 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 >> 172.21.15.66:0/427586895 conn(0x7f0d880678f0 msgr2=0x7f0d880cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 shutdown_connections 2024-03-30T13:12:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.357+0000 7f0d9039d640 1 -- 172.21.15.66:0/427586895 wait complete. 2024-03-30T13:12:40.383 DEBUG:tasks.cephfs.test_uninlining:getting inode info #2 of 23:1099511628359 2024-03-30T13:12:40.383 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628359 2024-03-30T13:12:40.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628359 2024-03-30T13:12:40.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.490+0000 7f2206a9a640 1 Processor -- start 2024-03-30T13:12:40.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.490+0000 7f2206a9a640 1 -- start start 2024-03-30T13:12:40.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.491+0000 7f2206a9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f220006db00 0x7f22000cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.491+0000 7f2206a9a640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22001476b0 con 0x7f22001472d0 2024-03-30T13:12:40.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.491+0000 7f2206a9a640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220006cb00 con 0x7f220006c690 2024-03-30T13:12:40.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.491+0000 7f2206a9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2200148140 con 0x7f220006db00 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.491+0000 7f2204836640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f22001472d0 0x7f220006d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44060/0 (socket says 172.21.15.66:44060) 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.491+0000 7f2204836640 1 -- 172.21.15.66:0/1626496963 learned_addr learned my addr 172.21.15.66:0/1626496963 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21fffff640 1 --2- 172.21.15.66:0/1626496963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f220006db00 0x7f22000cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231103434 0 0) 0x7f22001476b0 con 0x7f22001472d0 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21f0003520 con 0x7f22001472d0 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279892010 0 0) 0x7f220006cb00 con 0x7f220006c690 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22001476b0 con 0x7f220006c690 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1569124664 0 0) 0x7f21f0003520 con 0x7f22001472d0 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21fffff640 1 -- 172.21.15.66:0/1626496963 >> v1:172.21.15.117:6789/0 conn(0x7f220006c690 legacy=0x7f2200162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21fffff640 1 -- 172.21.15.66:0/1626496963 >> v1:172.21.15.66:6789/0 conn(0x7f22001472d0 legacy=0x7f220006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.492+0000 7f21fffff640 1 -- 172.21.15.66:0/1626496963 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2200071290 con 0x7f220006db00 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.493+0000 7f21fffff640 1 --2- 172.21.15.66:0/1626496963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f220006db00 0x7f22000cf410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0097d0 tx=0x7f21ec02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ec6851ddbb847f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.493+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec037070 con 0x7f220006db00 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.493+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec030510 con 0x7f220006db00 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.493+0000 7f21ff7fe640 1 -- 172.21.15.66:0/1626496963 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4001670 con 0x7f220006db00 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 -- 172.21.15.66:0/1626496963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f220006db00 msgr2=0x7f22000cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 --2- 172.21.15.66:0/1626496963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f220006db00 0x7f22000cf410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0097d0 tx=0x7f21ec02b770 comp rx=0 tx=0).stop 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 -- 172.21.15.66:0/1626496963 shutdown_connections 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 --2- 172.21.15.66:0/1626496963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f220006db00 0x7f22000cf410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 -- 172.21.15.66:0/1626496963 >> 172.21.15.66:0/1626496963 conn(0x7f22000b3e70 msgr2=0x7f22000b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 -- 172.21.15.66:0/1626496963 shutdown_connections 2024-03-30T13:12:40.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.494+0000 7f2206a9a640 1 -- 172.21.15.66:0/1626496963 wait complete. 2024-03-30T13:12:40.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.495+0000 7f2206a9a640 1 Processor -- start 2024-03-30T13:12:40.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.495+0000 7f2206a9a640 1 -- start start 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2206a9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f220006c690 0x7f22000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2206a9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 0x7f2200161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2206a9a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22001472d0 0x7f22001407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2206a9a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2200071850 con 0x7f220006db00 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2206a9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2200071040 con 0x7f220006c690 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2206a9a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2200071b30 con 0x7f22001472d0 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 0x7f2200161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 0x7f2200161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53054/0 (socket says 172.21.15.66:53054) 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 -- 172.21.15.66:0/426025386 learned_addr learned my addr 172.21.15.66:0/426025386 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2204836640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f220006c690 0x7f22000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2205037640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22001472d0 0x7f22001407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 -- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22001472d0 msgr2=0x7f22001407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22001472d0 0x7f22001407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 -- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f220006c690 msgr2=0x7f22000b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f220006c690 0x7f22000b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 -- 172.21.15.66:0/426025386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000b69c0 con 0x7f220006db00 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f2205037640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22001472d0 0x7f22001407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.496+0000 7f21fffff640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 0x7f2200161650 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000c00 tx=0x7f21ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.497+0000 7f21fd7fa640 1 -- 172.21.15.66:0/426025386 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec037040 con 0x7f220006db00 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.497+0000 7f21fd7fa640 1 -- 172.21.15.66:0/426025386 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec004250 con 0x7f220006db00 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.497+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22000b6ca0 con 0x7f220006db00 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.497+0000 7f21fd7fa640 1 -- 172.21.15.66:0/426025386 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec006e10 con 0x7f220006db00 2024-03-30T13:12:40.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.497+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22000b7260 con 0x7f220006db00 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.499+0000 7f21fd7fa640 1 -- 172.21.15.66:0/426025386 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21ec03a070 con 0x7f220006db00 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.499+0000 7f21fd7fa640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21e406c530 0x7f21e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.500+0000 7f2204836640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21e406c530 0x7f21e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.500+0000 7f21fd7fa640 1 -- 172.21.15.66:0/426025386 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f21ec0ad1c0 con 0x7f220006db00 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.500+0000 7f2204836640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21e406c530 0x7f21e406e9f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21f4000fd0 tx=0x7f21f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.500+0000 7f2209765740 1 Processor -- start 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.500+0000 7f2209765740 1 -- start start 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec499570 0x5557ec46c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2209765740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec49c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec46ca50 0x5557ec49c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2209765740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5557ec3d6590 con 0x5557ec499950 2024-03-30T13:12:40.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557ec374d30 con 0x5557ec499570 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5557ec3d2540 con 0x5557ec46ca50 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec49c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec49c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53056/0 (socket says 172.21.15.66:53056) 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 -- 172.21.15.66:0/1667480780 learned_addr learned my addr 172.21.15.66:0/1667480780 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2205037640 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec46ca50 0x5557ec49c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f2204836640 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec499570 0x5557ec46c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 -- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec46ca50 msgr2=0x5557ec49c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec46ca50 0x5557ec49c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 -- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec499570 msgr2=0x5557ec46c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec499570 0x5557ec46c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 -- 172.21.15.66:0/1667480780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ec1b6920 con 0x5557ec499950 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.501+0000 7f21fffff640 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec49c300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0835e0 tx=0x7f21ec02bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58eafd49f38b7732 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f21ea7fc640 1 -- 172.21.15.66:0/1667480780 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0a9070 con 0x5557ec499950 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f21ea7fc640 1 -- 172.21.15.66:0/1667480780 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec07b630 con 0x5557ec499950 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f21ea7fc640 1 -- 172.21.15.66:0/1667480780 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec04a040 con 0x5557ec499950 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 -- 172.21.15.66:0/1667480780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 msgr2=0x5557ec49c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec49c300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0835e0 tx=0x7f21ec02bd40 comp rx=0 tx=0).stop 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 -- 172.21.15.66:0/1667480780 shutdown_connections 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec46ca50 0x5557ec49c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec49c300 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 --2- 172.21.15.66:0/1667480780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec499570 0x5557ec46c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 -- 172.21.15.66:0/1667480780 >> 172.21.15.66:0/1667480780 conn(0x5557ec46e780 msgr2=0x5557ec4b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 -- 172.21.15.66:0/1667480780 shutdown_connections 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.502+0000 7f2209765740 1 -- 172.21.15.66:0/1667480780 wait complete. 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.503+0000 7f2209765740 1 -- start start 2024-03-30T13:12:40.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.503+0000 7f2209765740 1 Processor -- start 2024-03-30T13:12:40.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec4b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5557ec470e20 con 0x5557ec499950 2024-03-30T13:12:40.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557ec3729f0 con 0x5557ec49c2f0 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5557ec473660 con 0x5557ec499570 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2209765740 10 client.0 _ll_register_callbacks cb 0x5557ec376c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53068/0 (socket says 172.21.15.66:53068) 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 -- 172.21.15.66:0/3108376954 learned_addr learned my addr 172.21.15.66:0/3108376954 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2205037640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f2204836640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec4b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 -- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 msgr2=0x5557ec4b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec4b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 -- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 msgr2=0x5557ec4b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 -- 172.21.15.66:0/3108376954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ec0aa210 con 0x5557ec499950 2024-03-30T13:12:40.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21fffff640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c85c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f21ec07b7e0 tx=0x7f21ec031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9567faece578344e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.504+0000 7f21df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f21df7fe640 1 -- 172.21.15.66:0/3108376954 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0a9030 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f21df7fe640 1 -- 172.21.15.66:0/3108376954 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f21ec079230 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec07b630 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f21df7fe640 1 -- 172.21.15.66:0/3108376954 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f21ec04a040 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 10 client.6198 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5557ec2170b0 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f21df7fe640 1 -- 172.21.15.66:0/3108376954 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec07b630 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f21df7fe640 1 -- 172.21.15.66:0/3108376954 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f21ec083680 con 0x5557ec499950 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 10 client.6198 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 10 client.6198 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 20 client.6198 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4cb600 0x5557ec4cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 4 client.6198 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557ec3e3ad0 con 0x5557ec4cb600 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.505+0000 7f2204836640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4cb600 0x5557ec4cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f21df7fe640 10 client.6198 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2204836640 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4cb600 0x5557ec4cb9e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c30731dd0ebfc04 server_cookie=cd88c140646f4176 in_seq=0 out_seq=0 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f21df7fe640 1 -- 172.21.15.66:0/3108376954 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5557ec3e3ad0 con 0x5557ec4cb600 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f21df7fe640 10 client.6198 handle_command_reply: tid=0 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 client.6198 shutdown 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 msgr2=0x5557ec4c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c85c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f21ec07b7e0 tx=0x7f21ec031000 comp rx=0 tx=0).stop 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 shutdown_connections 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4cb600 0x5557ec4cb9e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c85c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f21df7fe640 0 client.6198 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 --2- 172.21.15.66:0/3108376954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec4b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.506+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 >> 172.21.15.66:0/3108376954 conn(0x5557ec46ca50 msgr2=0x5557ec4ba810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.507+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 shutdown_connections 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.507+0000 7f2209765740 1 -- 172.21.15.66:0/3108376954 wait complete. 2024-03-30T13:12:40.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.507+0000 7f2209765740 20 client.6198 trim_cache size 0 max 16384 2024-03-30T13:12:40.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.521+0000 7f2209765740 1 Processor -- start 2024-03-30T13:12:40.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.521+0000 7f2209765740 1 -- start start 2024-03-30T13:12:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2209765740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 0x5557ec4a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499950 0x5557ec467930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2209765740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5557ec470cf0 con 0x5557ec499570 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2200201040 con 0x5557ec49c2f0 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f22002000a0 con 0x5557ec499950 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 0x5557ec4a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 0x5557ec4a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53078/0 (socket says 172.21.15.66:53078) 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 -- 172.21.15.66:0/201772424 learned_addr learned my addr 172.21.15.66:0/201772424 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 -- 172.21.15.66:0/201772424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499950 msgr2=0x5557ec467930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499950 0x5557ec467930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 -- 172.21.15.66:0/201772424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 msgr2=0x5557ec4b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 -- 172.21.15.66:0/201772424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ec0832d0 con 0x5557ec499570 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f2204836640 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 0x5557ec4a0140 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f21f40011e0 tx=0x7f21f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435f943c627cd90c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f21e9ffb640 1 -- 172.21.15.66:0/201772424 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f402c070 con 0x5557ec499570 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f21e9ffb640 1 -- 172.21.15.66:0/201772424 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4005a10 con 0x5557ec499570 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.522+0000 7f21e9ffb640 1 -- 172.21.15.66:0/201772424 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f400e730 con 0x5557ec499570 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 -- 172.21.15.66:0/201772424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 msgr2=0x5557ec4a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 0x5557ec4a0140 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f21f40011e0 tx=0x7f21f4012000 comp rx=0 tx=0).stop 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 -- 172.21.15.66:0/201772424 shutdown_connections 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499950 0x5557ec467930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 --2- 172.21.15.66:0/201772424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499570 0x5557ec4a0140 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 -- 172.21.15.66:0/201772424 >> 172.21.15.66:0/201772424 conn(0x5557ec46ca50 msgr2=0x5557ec4950b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 -- 172.21.15.66:0/201772424 shutdown_connections 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 -- 172.21.15.66:0/201772424 wait complete. 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.523+0000 7f2209765740 1 -- start start 2024-03-30T13:12:40.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 Processor -- start 2024-03-30T13:12:40.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec533640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f22001ffa50 con 0x5557ec499950 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2200142c00 con 0x5557ec49c2f0 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.524+0000 7f2209765740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5557ec46cea0 con 0x5557ec499570 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2209765740 10 client.0 _ll_register_callbacks cb 0x7f21ec0a7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53094/0 (socket says 172.21.15.66:53094) 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 -- 172.21.15.66:0/2008718003 learned_addr learned my addr 172.21.15.66:0/2008718003 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2205037640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2204836640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec533640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 -- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 msgr2=0x5557ec533640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec533640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 -- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 msgr2=0x5557ec4c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 -- 172.21.15.66:0/2008718003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ec4b3d10 con 0x5557ec499950 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2204836640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec533640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21fffff640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c8050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0778b0 tx=0x7f21ec0ab410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d546980046339ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21df7fe640 1 -- 172.21.15.66:0/2008718003 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec07aa70 con 0x5557ec499950 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21df7fe640 1 -- 172.21.15.66:0/2008718003 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f21ec0930b0 con 0x5557ec499950 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f4005a10 con 0x5557ec499950 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f21df7fe640 1 -- 172.21.15.66:0/2008718003 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f21ec0a9020 con 0x5557ec499950 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2209765740 10 client.6204 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.525+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5557ec4b3d10 con 0x5557ec499950 2024-03-30T13:12:40.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f21df7fe640 1 -- 172.21.15.66:0/2008718003 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4005a10 con 0x5557ec499950 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f21df7fe640 1 -- 172.21.15.66:0/2008718003 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f21ec07a380 con 0x5557ec499950 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2209765740 10 client.6204 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2209765740 10 client.6204 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2209765740 20 client.6204 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2209765740 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4c4560 0x5557ec4ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2209765740 4 client.6204 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628359}] 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628359}) -- 0x5557ec3dde20 con 0x5557ec4c4560 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2204836640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4c4560 0x5557ec4ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f21df7fe640 10 client.6204 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.526+0000 7f2204836640 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4c4560 0x5557ec4ba810 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed8ad558b58f468 server_cookie=8368cf789e78f4ab in_seq=0 out_seq=0 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f21df7fe640 1 -- 172.21.15.66:0/2008718003 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3972 (crc 0 0 0) 0x5557ec3dde20 con 0x5557ec4c4560 2024-03-30T13:12:40.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f21df7fe640 10 client.6204 handle_command_reply: tid=0 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 client.6204 shutdown 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 msgr2=0x5557ec4c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c8050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0778b0 tx=0x7f21ec0ab410 comp rx=0 tx=0).stop 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 shutdown_connections 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5557ec4c4560 0x5557ec4ba810 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557ec49c2f0 0x5557ec4c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5557ec499950 0x5557ec4c8050 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f21df7fe640 0 client.6204 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 --2- 172.21.15.66:0/2008718003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5557ec499570 0x5557ec533640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 >> 172.21.15.66:0/2008718003 conn(0x5557ec46ca50 msgr2=0x5557ec49a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 shutdown_connections 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 1 -- 172.21.15.66:0/2008718003 wait complete. 2024-03-30T13:12:40.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2209765740 20 client.6204 trim_cache size 0 max 16384 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/Makefile", 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628359, 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:07.965704+0000", 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.487819+0000", 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:40.538 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.539 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 356, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.487819+0000", 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:40.540 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.541 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.965704+0000" 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.965704+0000" 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 194, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:40.542 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 4, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.008534+0000", 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 84, 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.543 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:40.544 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:40.545 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.546 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.547 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.548 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:40.549 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:40.550 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:40.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21e406c530 msgr2=0x7f21e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.527+0000 7f2206a9a640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21e406c530 0x7f21e406e9f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21f4000fd0 tx=0x7f21f4009040 comp rx=0 tx=0).stop 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 msgr2=0x7f2200161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 0x7f2200161650 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f21ec000c00 tx=0x7f21ec0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 shutdown_connections 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f21e406c530 0x7f21e406e9f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22001472d0 0x7f22001407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f220006db00 0x7f2200161650 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 --2- 172.21.15.66:0/426025386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f220006c690 0x7f22000b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 >> 172.21.15.66:0/426025386 conn(0x7f22000b3e70 msgr2=0x7f22000d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 shutdown_connections 2024-03-30T13:12:40.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.528+0000 7f2206a9a640 1 -- 172.21.15.66:0/426025386 wait complete. 2024-03-30T13:12:40.553 DEBUG:tasks.cephfs.test_uninlining:getting inode info #3 of 23:1099511628365 2024-03-30T13:12:40.553 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628365 2024-03-30T13:12:40.553 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628365 2024-03-30T13:12:40.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 Processor -- start 2024-03-30T13:12:40.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 -- start start 2024-03-30T13:12:40.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feff80b3e60 0x7feff815f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff8162070 0x7feff806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff8162610 con 0x7feff80afd60 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7feff806fac0 con 0x7feff80b3e60 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.661+0000 7feffd3c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7feff81628f0 con 0x7feff8162070 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff6ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7feff80afd60 0x7feff80b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44070/0 (socket says 172.21.15.66:44070) 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff6ffd640 1 -- 172.21.15.66:0/582659353 learned_addr learned my addr 172.21.15.66:0/582659353 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff77fe640 1 --2- 172.21.15.66:0/582659353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff8162070 0x7feff806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff67fc640 1 --2- 172.21.15.66:0/582659353 >> v2:172.21.15.117:3300/0 conn(0x7feff80b3e60 0x7feff815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff5ffb640 1 -- 172.21.15.66:0/582659353 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90849872 0 0) 0x7feff8162610 con 0x7feff80afd60 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff5ffb640 1 -- 172.21.15.66:0/582659353 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefe0003520 con 0x7feff80afd60 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff5ffb640 1 -- 172.21.15.66:0/582659353 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3474486266 0 0) 0x7fefe0003520 con 0x7feff80afd60 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff77fe640 1 -- 172.21.15.66:0/582659353 >> v2:172.21.15.117:3300/0 conn(0x7feff80b3e60 msgr2=0x7feff815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff77fe640 1 --2- 172.21.15.66:0/582659353 >> v2:172.21.15.117:3300/0 conn(0x7feff80b3e60 0x7feff815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff77fe640 1 -- 172.21.15.66:0/582659353 >> v1:172.21.15.66:6789/0 conn(0x7feff80afd60 legacy=0x7feff80b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff77fe640 1 -- 172.21.15.66:0/582659353 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff8162bd0 con 0x7feff8162070 2024-03-30T13:12:40.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.662+0000 7feff67fc640 1 --2- 172.21.15.66:0/582659353 >> v2:172.21.15.117:3300/0 conn(0x7feff80b3e60 0x7feff815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feff77fe640 1 --2- 172.21.15.66:0/582659353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff8162070 0x7feff806bb00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feff0009d90 tx=0x7feff002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6acf817b0f67573 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feff5ffb640 1 -- 172.21.15.66:0/582659353 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0039070 con 0x7feff8162070 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feff5ffb640 1 -- 172.21.15.66:0/582659353 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff002c8a0 con 0x7feff8162070 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feff5ffb640 1 -- 172.21.15.66:0/582659353 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8001690 con 0x7feff8162070 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 -- 172.21.15.66:0/582659353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff8162070 msgr2=0x7feff806bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 --2- 172.21.15.66:0/582659353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff8162070 0x7feff806bb00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feff0009d90 tx=0x7feff002bd70 comp rx=0 tx=0).stop 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 -- 172.21.15.66:0/582659353 shutdown_connections 2024-03-30T13:12:40.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 --2- 172.21.15.66:0/582659353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff8162070 0x7feff806bb00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 --2- 172.21.15.66:0/582659353 >> v2:172.21.15.117:3300/0 conn(0x7feff80b3e60 0x7feff815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 -- 172.21.15.66:0/582659353 >> 172.21.15.66:0/582659353 conn(0x7feff80d0840 msgr2=0x7feff80d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 -- 172.21.15.66:0/582659353 shutdown_connections 2024-03-30T13:12:40.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.664+0000 7feffd3c8640 1 -- 172.21.15.66:0/582659353 wait complete. 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 Processor -- start 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 -- start start 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feff80afd60 0x7feff80cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff80b3e60 0x7feff81f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 0x7feff81541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feff806c040 con 0x7feff8162070 2024-03-30T13:12:40.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feff8162eb0 con 0x7feff80afd60 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.666+0000 7feffd3c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7feff806c1c0 con 0x7feff80b3e60 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 0x7feff81541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 0x7feff81541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53096/0 (socket says 172.21.15.66:53096) 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 -- 172.21.15.66:0/1984699728 learned_addr learned my addr 172.21.15.66:0/1984699728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff6ffd640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feff80afd60 0x7feff80cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff67fc640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff80b3e60 0x7feff81f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 -- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff80b3e60 msgr2=0x7feff81f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff80b3e60 0x7feff81f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 -- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feff80afd60 msgr2=0x7feff80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feff80afd60 0x7feff80cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 -- 172.21.15.66:0/1984699728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff80ccfd0 con 0x7feff8162070 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feff77fe640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 0x7feff81541b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feff0009d90 tx=0x7feff0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7fefdf7fe640 1 -- 172.21.15.66:0/1984699728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0044020 con 0x7feff8162070 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7fefdf7fe640 1 -- 172.21.15.66:0/1984699728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff0034c40 con 0x7feff8162070 2024-03-30T13:12:40.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7fefdf7fe640 1 -- 172.21.15.66:0/1984699728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0039040 con 0x7feff8162070 2024-03-30T13:12:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff8157400 con 0x7feff8162070 2024-03-30T13:12:40.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.667+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81579c0 con 0x7feff8162070 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.669+0000 7fefdf7fe640 1 -- 172.21.15.66:0/1984699728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feff0047020 con 0x7feff8162070 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.669+0000 7fefdf7fe640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fefe406c560 0x7fefe406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.669+0000 7fefdf7fe640 1 -- 172.21.15.66:0/1984699728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7feff00add00 con 0x7feff8162070 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.669+0000 7feff6ffd640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fefe406c560 0x7fefe406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.670+0000 7feff6ffd640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fefe406c560 0x7fefe406ea20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fefe8001eb0 tx=0x7fefe8001d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.670+0000 7feffffb5740 1 Processor -- start 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 -- start start 2024-03-30T13:12:40.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe6f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe3fa50 0x556b5fe6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556b5fda9590 con 0x556b5fe6c950 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b5fd47d30 con 0x556b5fe6c570 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x556b5fda5540 con 0x556b5fe3fa50 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feff67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe6f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feff67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe6f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53100/0 (socket says 172.21.15.66:53100) 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feff67fc640 1 -- 172.21.15.66:0/2568956180 learned_addr learned my addr 172.21.15.66:0/2568956180 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.671+0000 7feff6ffd640 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff77fe640 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe3fa50 0x556b5fe6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff67fc640 1 -- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe3fa50 msgr2=0x556b5fe6f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff67fc640 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe3fa50 0x556b5fe6f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff67fc640 1 -- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 msgr2=0x556b5fe3f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff67fc640 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe3f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff67fc640 1 -- 172.21.15.66:0/2568956180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5fb89920 con 0x556b5fe6c950 2024-03-30T13:12:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7feff67fc640 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe6f300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fefec009b90 tx=0x7fefec009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e649e96263ce5aae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7fefdcff9640 1 -- 172.21.15.66:0/2568956180 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec019070 con 0x556b5fe6c950 2024-03-30T13:12:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7fefdcff9640 1 -- 172.21.15.66:0/2568956180 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec011c70 con 0x556b5fe6c950 2024-03-30T13:12:40.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.672+0000 7fefdcff9640 1 -- 172.21.15.66:0/2568956180 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec015460 con 0x556b5fe6c950 2024-03-30T13:12:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 -- 172.21.15.66:0/2568956180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 msgr2=0x556b5fe6f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe6f300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fefec009b90 tx=0x7fefec009190 comp rx=0 tx=0).stop 2024-03-30T13:12:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 -- 172.21.15.66:0/2568956180 shutdown_connections 2024-03-30T13:12:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe3fa50 0x556b5fe6f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe6f300 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 --2- 172.21.15.66:0/2568956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe3f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 -- 172.21.15.66:0/2568956180 >> 172.21.15.66:0/2568956180 conn(0x556b5fe41780 msgr2=0x556b5fe8a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 -- 172.21.15.66:0/2568956180 shutdown_connections 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.673+0000 7feffffb5740 1 -- 172.21.15.66:0/2568956180 wait complete. 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.674+0000 7feffffb5740 1 -- start start 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.674+0000 7feffffb5740 1 Processor -- start 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe72170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556b5fe43e20 con 0x556b5fe6c950 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b5fd459f0 con 0x556b5fe6c570 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x556b5fe46660 con 0x556b5fe6df10 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feff67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe72170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feffffb5740 10 client.0 _ll_register_callbacks cb 0x556b5fd49c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feff6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feff67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe72170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53110/0 (socket says 172.21.15.66:53110) 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feff67fc640 1 -- 172.21.15.66:0/3923465103 learned_addr learned my addr 172.21.15.66:0/3923465103 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.675+0000 7feff77fe640 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feff67fc640 1 -- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 msgr2=0x556b5fe6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feff67fc640 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feff67fc640 1 -- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 msgr2=0x556b5fe6a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feff67fc640 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe6a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feff67fc640 1 -- 172.21.15.66:0/3923465103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5fa7d210 con 0x556b5fe6c950 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feff67fc640 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe72170 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fefec009b60 tx=0x7fefec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4175733705e9bca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7fefd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7fefd5ffb640 1 -- 172.21.15.66:0/3923465103 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec02c060 con 0x556b5fe6c950 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7fefd5ffb640 1 -- 172.21.15.66:0/3923465103 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fefec009d40 con 0x556b5fe6c950 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefec011c70 con 0x556b5fe6c950 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7fefd5ffb640 1 -- 172.21.15.66:0/3923465103 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fefec032050 con 0x556b5fe6c950 2024-03-30T13:12:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feffffb5740 10 client.6216 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x556b5fbea0b0 con 0x556b5fe6c950 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.676+0000 7fefd5ffb640 1 -- 172.21.15.66:0/3923465103 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec011c70 con 0x556b5fe6c950 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7fefd5ffb640 1 -- 172.21.15.66:0/3923465103 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fefec035020 con 0x556b5fe6c950 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feffffb5740 10 client.6216 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feffffb5740 10 client.6216 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feffffb5740 20 client.6216 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feffffb5740 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe9b400 0x556b5fe9b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feffffb5740 4 client.6216 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b5fdb6ad0 con 0x556b5fe9b400 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feff6ffd640 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe9b400 0x556b5fe9b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7fefd5ffb640 10 client.6216 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.677+0000 7feff6ffd640 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe9b400 0x556b5fe9b830 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae94476a576ae48d server_cookie=d67e26548e17be6e in_seq=0 out_seq=0 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7fefd5ffb640 1 -- 172.21.15.66:0/3923465103 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x556b5fdb6ad0 con 0x556b5fe9b400 2024-03-30T13:12:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7fefd5ffb640 10 client.6216 handle_command_reply: tid=0 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 client.6216 shutdown 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 msgr2=0x556b5fe72170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe72170 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fefec009b60 tx=0x7fefec004730 comp rx=0 tx=0).stop 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 shutdown_connections 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe9b400 0x556b5fe9b830 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fe72170 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7fefd5ffb640 0 client.6216 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 --2- 172.21.15.66:0/3923465103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5fe6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 >> 172.21.15.66:0/3923465103 conn(0x556b5fe3fa50 msgr2=0x556b5fe434f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 shutdown_connections 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 1 -- 172.21.15.66:0/3923465103 wait complete. 2024-03-30T13:12:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.678+0000 7feffffb5740 20 client.6216 trim_cache size 0 max 16384 2024-03-30T13:12:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.693+0000 7feffffb5740 1 Processor -- start 2024-03-30T13:12:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.693+0000 7feffffb5740 1 -- start start 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6c570 0x556b5fe8b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c950 0x556b5fe9b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feffffb5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 0x556b5fe9bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feffffb5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feff8201100 con 0x556b5fe6df10 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feff8200100 con 0x556b5fe6c950 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7feff8156650 con 0x556b5fe6c570 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 0x556b5fe9bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 0x556b5fe9bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53124/0 (socket says 172.21.15.66:53124) 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff77fe640 1 -- 172.21.15.66:0/490574720 learned_addr learned my addr 172.21.15.66:0/490574720 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff67fc640 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c950 0x556b5fe9b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff6ffd640 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6c570 0x556b5fe8b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff77fe640 1 -- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6c570 msgr2=0x556b5fe8b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.694+0000 7feff77fe640 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6c570 0x556b5fe8b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feff77fe640 1 -- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c950 msgr2=0x556b5fe9b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feff77fe640 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c950 0x556b5fe9b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feff77fe640 1 -- 172.21.15.66:0/490574720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5fa562d0 con 0x556b5fe6df10 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feff67fc640 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c950 0x556b5fe9b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feff77fe640 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 0x556b5fe9bd50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7feff0077e30 tx=0x7feff0009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a711aa8f556532e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7fefd7fff640 1 -- 172.21.15.66:0/490574720 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff007c0d0 con 0x556b5fe6df10 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7fefd7fff640 1 -- 172.21.15.66:0/490574720 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff007b9d0 con 0x556b5fe6df10 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7fefd7fff640 1 -- 172.21.15.66:0/490574720 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff007fd80 con 0x556b5fe6df10 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feffffb5740 1 -- 172.21.15.66:0/490574720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 msgr2=0x556b5fe9bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.695+0000 7feffffb5740 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 0x556b5fe9bd50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7feff0077e30 tx=0x7feff0009af0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 -- 172.21.15.66:0/490574720 shutdown_connections 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6df10 0x556b5fe9bd50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c950 0x556b5fe9b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 --2- 172.21.15.66:0/490574720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6c570 0x556b5fe8b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 -- 172.21.15.66:0/490574720 >> 172.21.15.66:0/490574720 conn(0x556b5fe3fa50 msgr2=0x556b5fe3a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 -- 172.21.15.66:0/490574720 shutdown_connections 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 -- 172.21.15.66:0/490574720 wait complete. 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 -- start start 2024-03-30T13:12:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.696+0000 7feffffb5740 1 Processor -- start 2024-03-30T13:12:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5ff05d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fea4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556b5fe8a520 con 0x556b5fe6c950 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feff81ffb10 con 0x556b5fe6c570 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x556b5fe6c380 con 0x556b5fe6df10 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feffffb5740 10 client.0 _ll_register_callbacks cb 0x7fefe8001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fea4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fea4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53128/0 (socket says 172.21.15.66:53128) 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 -- 172.21.15.66:0/1691924415 learned_addr learned my addr 172.21.15.66:0/1691924415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff77fe640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff6ffd640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5ff05d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 -- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 msgr2=0x556b5fe6b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 -- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 msgr2=0x556b5ff05d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5ff05d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.697+0000 7feff67fc640 1 -- 172.21.15.66:0/1691924415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5fe6af40 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feff67fc640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fea4d70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fefec009b30 tx=0x7fefec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258bc11f5e3959c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7fefd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7fefd5ffb640 1 -- 172.21.15.66:0/1691924415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec03a020 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7fefd5ffb640 1 -- 172.21.15.66:0/1691924415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fefec02c9d0 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff007b9d0 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7fefd5ffb640 1 -- 172.21.15.66:0/1691924415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fefec019050 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 10 client.6222 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x556b5fe9be50 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7fefd5ffb640 1 -- 172.21.15.66:0/1691924415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff007b9d0 con 0x556b5fe6c950 2024-03-30T13:12:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7fefd5ffb640 1 -- 172.21.15.66:0/1691924415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fefec035040 con 0x556b5fe6c950 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 10 client.6222 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 10 client.6222 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 20 client.6222 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe727d0 0x556b5fe68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 4 client.6222 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628365}] 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628365}) -- 0x556b5fdb0e20 con 0x556b5fe727d0 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.698+0000 7feff6ffd640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe727d0 0x556b5fe68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7fefd5ffb640 10 client.6222 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feff6ffd640 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe727d0 0x556b5fe68e60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9603f4e566f9a7 server_cookie=2ebf8aa0874ee725 in_seq=0 out_seq=0 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7fefd5ffb640 1 -- 172.21.15.66:0/1691924415 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3977 (crc 0 0 0) 0x556b5fdb0e20 con 0x556b5fe727d0 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7fefd5ffb640 10 client.6222 handle_command_reply: tid=0 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 client.6222 shutdown 2024-03-30T13:12:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 msgr2=0x556b5fea4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fea4d70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fefec009b30 tx=0x7fefec008040 comp rx=0 tx=0).stop 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 shutdown_connections 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x556b5fe727d0 0x556b5fe68e60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b5fe6df10 0x556b5fe6b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7fefd5ffb640 0 client.6222 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556b5fe6c950 0x556b5fea4d70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 --2- 172.21.15.66:0/1691924415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b5fe6c570 0x556b5ff05d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 >> 172.21.15.66:0/1691924415 conn(0x556b5fe3fa50 msgr2=0x556b5fe709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 shutdown_connections 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 1 -- 172.21.15.66:0/1691924415 wait complete. 2024-03-30T13:12:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.699+0000 7feffffb5740 20 client.6222 trim_cache size 0 max 16384 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/ceph_frag.c", 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628365, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:07.991018+0000", 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.491819+0000", 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:40.710 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 401, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:40.711 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.491819+0000", 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 401, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.991018+0000" 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.713 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 401, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.991018+0000" 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 234, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 16, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:40.714 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.036598+0000", 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 108, 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:40.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:40.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:40.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:40.718 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.719 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:40.720 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:40.721 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:40.722 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:40.722 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fefe406c560 msgr2=0x7fefe406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fefe406c560 0x7fefe406ea20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fefe8001eb0 tx=0x7fefe8001d00 comp rx=0 tx=0).stop 2024-03-30T13:12:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 msgr2=0x7feff81541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 0x7feff81541b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feff0009d90 tx=0x7feff0004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 shutdown_connections 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fefe406c560 0x7fefe406ea20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feff8162070 0x7feff81541b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feff80b3e60 0x7feff81f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 --2- 172.21.15.66:0/1984699728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feff80afd60 0x7feff80cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 >> 172.21.15.66:0/1984699728 conn(0x7feff80d0840 msgr2=0x7feff806e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 shutdown_connections 2024-03-30T13:12:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.700+0000 7feffd3c8640 1 -- 172.21.15.66:0/1984699728 wait complete. 2024-03-30T13:12:40.725 DEBUG:tasks.cephfs.test_uninlining:getting inode info #4 of 23:1099511628372 2024-03-30T13:12:40.725 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628372 2024-03-30T13:12:40.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628372 2024-03-30T13:12:40.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.831+0000 7f9846d90640 1 Processor -- start 2024-03-30T13:12:40.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.831+0000 7f9846d90640 1 -- start start 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.832+0000 7f9846d90640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 0x7f98400cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.832+0000 7f9846d90640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f984006c690 0x7f9840162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.832+0000 7f9846d90640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98401476b0 con 0x7f98401472d0 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.832+0000 7f9846d90640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f9840148140 con 0x7f984006c690 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.832+0000 7f9846d90640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9840071040 con 0x7f984006db00 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f983ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 0x7f98400cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f984532d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f984006c690 0x7f9840162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f983ffff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 0x7f98400cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:37906/0 (socket says 172.21.15.66:37906) 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f983ffff640 1 -- 172.21.15.66:0/2006154405 learned_addr learned my addr 172.21.15.66:0/2006154405 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f984532d640 1 -- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 msgr2=0x7f98400cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f984532d640 1 --2- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 0x7f98400cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f984532d640 1 -- 172.21.15.66:0/2006154405 >> v1:172.21.15.117:6789/0 conn(0x7f98401472d0 legacy=0x7f984006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f984532d640 1 -- 172.21.15.66:0/2006154405 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984006cb00 con 0x7f984006c690 2024-03-30T13:12:40.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.833+0000 7f983ffff640 1 --2- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 0x7f98400cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.834+0000 7f984532d640 1 --2- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006c690 0x7f9840162110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9838009d90 tx=0x7f983802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5d7d59402e79e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.834+0000 7f983f7fe640 1 -- 172.21.15.66:0/2006154405 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838039070 con 0x7f984006c690 2024-03-30T13:12:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f983f7fe640 1 -- 172.21.15.66:0/2006154405 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983802cbc0 con 0x7f984006c690 2024-03-30T13:12:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f983f7fe640 1 -- 172.21.15.66:0/2006154405 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838034440 con 0x7f984006c690 2024-03-30T13:12:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 -- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006c690 msgr2=0x7f9840162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 --2- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006c690 0x7f9840162110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9838009d90 tx=0x7f983802be10 comp rx=0 tx=0).stop 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 -- 172.21.15.66:0/2006154405 shutdown_connections 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 --2- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006c690 0x7f9840162110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 --2- 172.21.15.66:0/2006154405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006db00 0x7f98400cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 -- 172.21.15.66:0/2006154405 >> 172.21.15.66:0/2006154405 conn(0x7f98400b3e70 msgr2=0x7f98400b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 -- 172.21.15.66:0/2006154405 shutdown_connections 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.835+0000 7f9846d90640 1 -- 172.21.15.66:0/2006154405 wait complete. 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.836+0000 7f9846d90640 1 Processor -- start 2024-03-30T13:12:40.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.836+0000 7f9846d90640 1 -- start start 2024-03-30T13:12:40.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f9846d90640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006c690 0x7f984015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f9846d90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006db00 0x7f9840140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f9846d90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 0x7f98401f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f9846d90640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9840071850 con 0x7f98401472d0 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f9846d90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9840071570 con 0x7f984006db00 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f9846d90640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f9840071b30 con 0x7f984006c690 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f984532d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 0x7f98401f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f984532d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 0x7f98401f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53140/0 (socket says 172.21.15.66:53140) 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f984532d640 1 -- 172.21.15.66:0/2783555876 learned_addr learned my addr 172.21.15.66:0/2783555876 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.837+0000 7f983ffff640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006db00 0x7f9840140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f9844b2c640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006c690 0x7f984015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f984532d640 1 -- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006c690 msgr2=0x7f984015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f984532d640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006c690 0x7f984015c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f984532d640 1 -- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006db00 msgr2=0x7f9840140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f984532d640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006db00 0x7f9840140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f984532d640 1 -- 172.21.15.66:0/2783555876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840141140 con 0x7f98401472d0 2024-03-30T13:12:40.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f984532d640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 0x7f98401f7da0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9838009d90 tx=0x7f9838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f983d7fa640 1 -- 172.21.15.66:0/2783555876 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838039050 con 0x7f98401472d0 2024-03-30T13:12:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98401f47d0 con 0x7f98401472d0 2024-03-30T13:12:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f983d7fa640 1 -- 172.21.15.66:0/2783555876 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9838045070 con 0x7f98401472d0 2024-03-30T13:12:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f983d7fa640 1 -- 172.21.15.66:0/2783555876 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838009450 con 0x7f98401472d0 2024-03-30T13:12:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.838+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401f4d90 con 0x7f98401472d0 2024-03-30T13:12:40.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.841+0000 7f983d7fa640 1 -- 172.21.15.66:0/2783555876 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f983803e850 con 0x7f98401472d0 2024-03-30T13:12:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.841+0000 7f983d7fa640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f982406c530 0x7f982406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.841+0000 7f983d7fa640 1 -- 172.21.15.66:0/2783555876 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f98380ae780 con 0x7f98401472d0 2024-03-30T13:12:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.841+0000 7f9844b2c640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f982406c530 0x7f982406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.841+0000 7f984997d740 1 Processor -- start 2024-03-30T13:12:40.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 -- start start 2024-03-30T13:12:40.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f9844b2c640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f982406c530 0x7f982406e9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98340016e0 tx=0x7f9834009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1d8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1a8a50 0x56391c1d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56391c112590 con 0x56391c1d5570 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56391c0b0d30 con 0x56391c1d5950 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x56391c10e540 con 0x56391c1a8a50 2024-03-30T13:12:40.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f9844b2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f9844b2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53146/0 (socket says 172.21.15.66:53146) 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.842+0000 7f9844b2c640 1 -- 172.21.15.66:0/2605552425 learned_addr learned my addr 172.21.15.66:0/2605552425 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f983ffff640 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1d8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f984532d640 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1a8a50 0x56391c1d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f9844b2c640 1 -- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1a8a50 msgr2=0x56391c1d8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f9844b2c640 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1a8a50 0x56391c1d8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f9844b2c640 1 -- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 msgr2=0x56391c1d8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f9844b2c640 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1d8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f9844b2c640 1 -- 172.21.15.66:0/2605552425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391bef2920 con 0x56391c1d5570 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f983ffff640 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1d8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f9844b2c640 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1a8510 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f983400fb70 tx=0x7f9834012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c05f952487984e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f982a7fc640 1 -- 172.21.15.66:0/2605552425 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834026070 con 0x56391c1d5570 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f982a7fc640 1 -- 172.21.15.66:0/2605552425 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98340067d0 con 0x56391c1d5570 2024-03-30T13:12:40.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.843+0000 7f982a7fc640 1 -- 172.21.15.66:0/2605552425 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834006ba0 con 0x56391c1d5570 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 -- 172.21.15.66:0/2605552425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 msgr2=0x56391c1a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1a8510 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f983400fb70 tx=0x7f9834012000 comp rx=0 tx=0).stop 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 -- 172.21.15.66:0/2605552425 shutdown_connections 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1a8a50 0x56391c1d8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1d8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 --2- 172.21.15.66:0/2605552425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1a8510 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 -- 172.21.15.66:0/2605552425 >> 172.21.15.66:0/2605552425 conn(0x56391c1aa780 msgr2=0x56391c1f3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 -- 172.21.15.66:0/2605552425 shutdown_connections 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 -- 172.21.15.66:0/2605552425 wait complete. 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.844+0000 7f984997d740 1 -- start start 2024-03-30T13:12:40.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 Processor -- start 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1db280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d4c30 0x56391c1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56391c1ace20 con 0x56391c1d5570 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56391c0ae9f0 con 0x56391c1d5950 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x56391c1af660 con 0x56391c1d4c30 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f984997d740 10 client.0 _ll_register_callbacks cb 0x56391c0b2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f9844b2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f9844b2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53150/0 (socket says 172.21.15.66:53150) 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.845+0000 7f9844b2c640 1 -- 172.21.15.66:0/117130676 learned_addr learned my addr 172.21.15.66:0/117130676 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f984532d640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d4c30 0x56391c1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f983ffff640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1db280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f9844b2c640 1 -- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d4c30 msgr2=0x56391c1f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f9844b2c640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d4c30 0x56391c1f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f9844b2c640 1 -- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 msgr2=0x56391c1db280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f9844b2c640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1db280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f9844b2c640 1 -- 172.21.15.66:0/117130676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391bde6210 con 0x56391c1d5570 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f9844b2c640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dad40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9834007400 tx=0x7f9834000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914bdc34519ee542 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f981f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f981f7fe640 1 -- 172.21.15.66:0/117130676 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834032060 con 0x56391c1d5570 2024-03-30T13:12:40.855 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f981f7fe640 1 -- 172.21.15.66:0/117130676 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f98340208b0 con 0x56391c1d5570 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98340067d0 con 0x56391c1d5570 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f981f7fe640 1 -- 172.21.15.66:0/117130676 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9834038050 con 0x56391c1d5570 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f984997d740 10 client.6231 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x56391bf530b0 con 0x56391c1d5570 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.846+0000 7f981f7fe640 1 -- 172.21.15.66:0/117130676 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98340067d0 con 0x56391c1d5570 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f981f7fe640 1 -- 172.21.15.66:0/117130676 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f983403b020 con 0x56391c1d5570 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f984997d740 10 client.6231 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f984997d740 10 client.6231 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f984997d740 20 client.6231 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f984997d740 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1d9970 0x56391c1d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f984997d740 4 client.6231 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56391c11fad0 con 0x56391c1d9970 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f983ffff640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1d9970 0x56391c1d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f981f7fe640 10 client.6231 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.847+0000 7f983ffff640 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1d9970 0x56391c1d9db0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9c65ba0b45f49e server_cookie=de4641d0b4006de7 in_seq=0 out_seq=0 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f981f7fe640 1 -- 172.21.15.66:0/117130676 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x56391c11fad0 con 0x56391c1d9970 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f981f7fe640 10 client.6231 handle_command_reply: tid=0 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 client.6231 shutdown 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 msgr2=0x56391c1dad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dad40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9834007400 tx=0x7f9834000ed0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 shutdown_connections 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1d9970 0x56391c1d9db0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d4c30 0x56391c1f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d5950 0x56391c1db280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 --2- 172.21.15.66:0/117130676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dad40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 >> 172.21.15.66:0/117130676 conn(0x56391c1a8a50 msgr2=0x56391c1ac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f981f7fe640 0 client.6231 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 shutdown_connections 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 1 -- 172.21.15.66:0/117130676 wait complete. 2024-03-30T13:12:40.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.848+0000 7f984997d740 20 client.6231 trim_cache size 0 max 16384 2024-03-30T13:12:40.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.865+0000 7f984997d740 1 Processor -- start 2024-03-30T13:12:40.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.865+0000 7f984997d740 1 -- start start 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c1ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f984997d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5950 0x56391c216340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f984997d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98402012e0 con 0x56391c1d5570 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98402002e0 con 0x56391c1d4c30 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f984015f950 con 0x56391c1d5950 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53152/0 (socket says 172.21.15.66:53152) 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 -- 172.21.15.66:0/1524419300 learned_addr learned my addr 172.21.15.66:0/1524419300 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 -- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5950 msgr2=0x56391c216340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5950 0x56391c216340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 -- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 msgr2=0x56391c1ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f9844b2c640 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c1ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.866+0000 7f983ffff640 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c1ddb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.867+0000 7f983ffff640 1 -- 172.21.15.66:0/1524419300 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391bdbf2d0 con 0x56391c1d5570 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.867+0000 7f983ffff640 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dce20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f98400b1cb0 tx=0x7f982c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb5c2164edd03b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.867+0000 7f9829ffb640 1 -- 172.21.15.66:0/1524419300 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c018190 con 0x56391c1d5570 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.867+0000 7f9829ffb640 1 -- 172.21.15.66:0/1524419300 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982c018a00 con 0x56391c1d5570 2024-03-30T13:12:40.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.867+0000 7f9829ffb640 1 -- 172.21.15.66:0/1524419300 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c018db0 con 0x56391c1d5570 2024-03-30T13:12:40.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 -- 172.21.15.66:0/1524419300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 msgr2=0x56391c1dce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dce20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f98400b1cb0 tx=0x7f982c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:40.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 -- 172.21.15.66:0/1524419300 shutdown_connections 2024-03-30T13:12:40.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5950 0x56391c216340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5570 0x56391c1dce20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 --2- 172.21.15.66:0/1524419300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c1ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 -- 172.21.15.66:0/1524419300 >> 172.21.15.66:0/1524419300 conn(0x56391c1a8a50 msgr2=0x56391c217560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 -- 172.21.15.66:0/1524419300 shutdown_connections 2024-03-30T13:12:40.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.868+0000 7f984997d740 1 -- 172.21.15.66:0/1524419300 wait complete. 2024-03-30T13:12:40.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.869+0000 7f984997d740 1 -- start start 2024-03-30T13:12:40.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.869+0000 7f984997d740 1 Processor -- start 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c20ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5570 0x56391c262810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 0x56391c209410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98401ffcc0 con 0x56391c1d5950 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56391c11fad0 con 0x56391c1d4c30 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x56391c20c0b0 con 0x56391c1d5570 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984997d740 10 client.0 _ll_register_callbacks cb 0x7f9834024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.870+0000 7f984532d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 0x56391c209410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 0x56391c209410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53154/0 (socket says 172.21.15.66:53154) 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 -- 172.21.15.66:0/1103716038 learned_addr learned my addr 172.21.15.66:0/1103716038 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:40.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f9844b2c640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c20ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f983ffff640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5570 0x56391c262810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 -- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5570 msgr2=0x56391c262810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5570 0x56391c262810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 -- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 msgr2=0x56391c20ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c20ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 -- 172.21.15.66:0/1103716038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391c20dd90 con 0x56391c1d5950 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984532d640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 0x56391c209410 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f98380acb00 tx=0x7f983807c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c823e4f6b88db089 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f981f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f981f7fe640 1 -- 172.21.15.66:0/1103716038 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838085170 con 0x56391c1d5950 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f981f7fe640 1 -- 172.21.15.66:0/1103716038 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f983803eb40 con 0x56391c1d5950 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f982c018a00 con 0x56391c1d5950 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f981f7fe640 1 -- 172.21.15.66:0/1103716038 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f983809a2b0 con 0x56391c1d5950 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984997d740 10 client.6237 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:40.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x56391c1f0f90 con 0x56391c1d5950 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.871+0000 7f981f7fe640 1 -- 172.21.15.66:0/1103716038 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982c018a00 con 0x56391c1d5950 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f981f7fe640 1 -- 172.21.15.66:0/1103716038 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f9838084ce0 con 0x56391c1d5950 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f984997d740 10 client.6237 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f984997d740 10 client.6237 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f984997d740 20 client.6237 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f984997d740 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1f2310 0x56391c205220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f984997d740 4 client.6237 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628372}] 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628372}) -- 0x56391c119e20 con 0x56391c1f2310 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f9844b2c640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1f2310 0x56391c205220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f981f7fe640 10 client.6237 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f9844b2c640 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1f2310 0x56391c205220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ec1bd0887d0a17 server_cookie=d5b73c5a8add2d5e in_seq=0 out_seq=0 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f981f7fe640 1 -- 172.21.15.66:0/1103716038 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3970 (crc 0 0 0) 0x56391c119e20 con 0x56391c1f2310 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.872+0000 7f981f7fe640 10 client.6237 handle_command_reply: tid=0 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 client.6237 shutdown 2024-03-30T13:12:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 msgr2=0x56391c209410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 0x56391c209410 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f98380acb00 tx=0x7f983807c060 comp rx=0 tx=0).stop 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 shutdown_connections 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x56391c1f2310 0x56391c205220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56391c1d5950 0x56391c209410 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56391c1d5570 0x56391c262810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 --2- 172.21.15.66:0/1103716038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56391c1d4c30 0x56391c20ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f981f7fe640 0 client.6237 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 >> 172.21.15.66:0/1103716038 conn(0x56391c1a8a50 msgr2=0x56391c1f1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 shutdown_connections 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 1 -- 172.21.15.66:0/1103716038 wait complete. 2024-03-30T13:12:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f984997d740 20 client.6237 trim_cache size 0 max 16384 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/io.h", 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628372, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:07.995356+0000", 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.495819+0000", 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:40.882 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 387, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:40.883 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.495819+0000", 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:40.884 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 387, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.995356+0000" 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:40.885 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 387, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.995356+0000" 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 178, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 30, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:40.886 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.038087+0000", 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 112, 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:40.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.888 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:40.889 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:40.890 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:40.891 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:40.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:40.893 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f982406c530 msgr2=0x7f982406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f9846d90640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f982406c530 0x7f982406e9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98340016e0 tx=0x7f9834009040 comp rx=0 tx=0).stop 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 msgr2=0x7f98401f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.873+0000 7f9846d90640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 0x7f98401f7da0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9838009d90 tx=0x7f9838004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 shutdown_connections 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f982406c530 0x7f982406e9f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98401472d0 0x7f98401f7da0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984006db00 0x7f9840140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 --2- 172.21.15.66:0/2783555876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f984006c690 0x7f984015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 >> 172.21.15.66:0/2783555876 conn(0x7f98400b3e70 msgr2=0x7f98400b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 shutdown_connections 2024-03-30T13:12:40.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:40.874+0000 7f9846d90640 1 -- 172.21.15.66:0/2783555876 wait complete. 2024-03-30T13:12:40.896 DEBUG:tasks.cephfs.test_uninlining:getting inode info #5 of 23:1099511628374 2024-03-30T13:12:40.896 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628374 2024-03-30T13:12:40.896 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628374 2024-03-30T13:12:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.002+0000 7fc351ec8640 1 Processor -- start 2024-03-30T13:12:41.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.002+0000 7fc351ec8640 1 -- start start 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc351ec8640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc34c0b5dc0 0x7fc34c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc351ec8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc351ec8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc34c0b66e0 con 0x7fc34c06c620 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc351ec8640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc34c0d2ff0 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc351ec8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c0d32d0 con 0x7fc34c0ba090 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc34b7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc34c0b5dc0 0x7fc34c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc34b7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc34c0b5dc0 0x7fc34c0b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53162/0 (socket says 172.21.15.66:53162) 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc34bfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc34c06c620 0x7fc34c06cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44078/0 (socket says 172.21.15.66:44078) 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc34b7fe640 1 -- 172.21.15.66:0/3862630433 learned_addr learned my addr 172.21.15.66:0/3862630433 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.003+0000 7fc34affd640 1 --2- 172.21.15.66:0/3862630433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.004+0000 7fc34b7fe640 1 -- 172.21.15.66:0/3862630433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 msgr2=0x7fc34c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.004+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/3862630433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c0d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.004+0000 7fc34b7fe640 1 -- 172.21.15.66:0/3862630433 >> v1:172.21.15.66:6789/0 conn(0x7fc34c06c620 legacy=0x7fc34c06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.004+0000 7fc34a7fc640 1 -- 172.21.15.66:0/3862630433 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014689870 0 0) 0x7fc34c0b66e0 con 0x7fc34c06c620 2024-03-30T13:12:41.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.004+0000 7fc34b7fe640 1 -- 172.21.15.66:0/3862630433 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c0d35b0 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.004+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/3862630433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c0b61a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009870 tx=0x7fc33c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4386a4b3eb34c23f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc34a7fc640 1 -- 172.21.15.66:0/3862630433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c034070 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc34a7fc640 1 -- 172.21.15.66:0/3862630433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c030840 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc34a7fc640 1 -- 172.21.15.66:0/3862630433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c030c10 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 -- 172.21.15.66:0/3862630433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 msgr2=0x7fc34c0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 --2- 172.21.15.66:0/3862630433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c0b61a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009870 tx=0x7fc33c02b980 comp rx=0 tx=0).stop 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 -- 172.21.15.66:0/3862630433 shutdown_connections 2024-03-30T13:12:41.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 --2- 172.21.15.66:0/3862630433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c0d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 --2- 172.21.15.66:0/3862630433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c0b61a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 -- 172.21.15.66:0/3862630433 >> 172.21.15.66:0/3862630433 conn(0x7fc34c0678f0 msgr2=0x7fc34c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 -- 172.21.15.66:0/3862630433 shutdown_connections 2024-03-30T13:12:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.005+0000 7fc351ec8640 1 -- 172.21.15.66:0/3862630433 wait complete. 2024-03-30T13:12:41.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.006+0000 7fc351ec8640 1 Processor -- start 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 -- start start 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34c06c620 0x7fc34c15fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c0d4090 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c0d3b70 con 0x7fc34c06c620 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc351ec8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c0b83e0 con 0x7fc34c0ba090 2024-03-30T13:12:41.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc34affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc34affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c15c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53174/0 (socket says 172.21.15.66:53174) 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.007+0000 7fc34affd640 1 -- 172.21.15.66:0/1102144180 learned_addr learned my addr 172.21.15.66:0/1102144180 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34c06c620 0x7fc34c15fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34bfff640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34affd640 1 -- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 msgr2=0x7fc34c161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34affd640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34affd640 1 -- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34c06c620 msgr2=0x7fc34c15fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34affd640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34c06c620 0x7fc34c15fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34affd640 1 -- 172.21.15.66:0/1102144180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c15ec60 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34bfff640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc34affd640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c15c200 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc33400e840 tx=0x7fc33400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc343fff640 1 -- 172.21.15.66:0/1102144180 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334017070 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc343fff640 1 -- 172.21.15.66:0/1102144180 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33400c940 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c15ef40 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc343fff640 1 -- 172.21.15.66:0/1102144180 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33400cd10 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.008+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c15f500 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.010+0000 7fc343fff640 1 -- 172.21.15.66:0/1102144180 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc33401a070 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.010+0000 7fc343fff640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc32c06c530 0x7fc32c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.010+0000 7fc343fff640 1 -- 172.21.15.66:0/1102144180 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc33408b990 con 0x7fc34c0b5dc0 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.010+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc32c06c530 0x7fc32c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.011+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc32c06c530 0x7fc32c06e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc33c036000 tx=0x7fc33c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.011+0000 7fc354b91740 1 Processor -- start 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.011+0000 7fc354b91740 1 -- start start 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41570 0x55a3d4d14510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc354b91740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d44300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d14a50 0x55a3d4d44840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc354b91740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4c7e590 con 0x55a3d4d41950 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc354b91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4c1cd30 con 0x55a3d4d41570 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc354b91740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55a3d4c7a540 con 0x55a3d4d14a50 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d44300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d44300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53190/0 (socket says 172.21.15.66:53190) 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 -- 172.21.15.66:0/403236580 learned_addr learned my addr 172.21.15.66:0/403236580 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34bfff640 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d14a50 0x55a3d4d44840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41570 0x55a3d4d14510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 -- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d14a50 msgr2=0x55a3d4d44840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d14a50 0x55a3d4d44840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 -- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41570 msgr2=0x55a3d4d14510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41570 0x55a3d4d14510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.012+0000 7fc34affd640 1 -- 172.21.15.66:0/403236580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d4a5e920 con 0x55a3d4d41950 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.013+0000 7fc34affd640 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d44300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc334061e40 tx=0x7fc334059260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f977a3ce55857c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.013+0000 7fc3417fa640 1 -- 172.21.15.66:0/403236580 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334021070 con 0x55a3d4d41950 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.013+0000 7fc3417fa640 1 -- 172.21.15.66:0/403236580 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc334062370 con 0x55a3d4d41950 2024-03-30T13:12:41.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.013+0000 7fc3417fa640 1 -- 172.21.15.66:0/403236580 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334055eb0 con 0x55a3d4d41950 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.013+0000 7fc354b91740 1 -- 172.21.15.66:0/403236580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 msgr2=0x55a3d4d44300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.013+0000 7fc354b91740 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d44300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc334061e40 tx=0x7fc334059260 comp rx=0 tx=0).stop 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 -- 172.21.15.66:0/403236580 shutdown_connections 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d14a50 0x55a3d4d44840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d44300 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 --2- 172.21.15.66:0/403236580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41570 0x55a3d4d14510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 -- 172.21.15.66:0/403236580 >> 172.21.15.66:0/403236580 conn(0x55a3d4d16780 msgr2=0x55a3d4d5f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 -- 172.21.15.66:0/403236580 shutdown_connections 2024-03-30T13:12:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 -- 172.21.15.66:0/403236580 wait complete. 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 -- start start 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.014+0000 7fc354b91740 1 Processor -- start 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41950 0x55a3d4d43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 0x55a3d4d47bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4d18e20 con 0x55a3d4d454a0 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4c1a9f0 con 0x55a3d4d41950 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55a3d4d1b660 con 0x55a3d4d41570 2024-03-30T13:12:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc354b91740 10 client.0 _ll_register_callbacks cb 0x55a3d4c1ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 0x55a3d4d47bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 0x55a3d4d47bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53196/0 (socket says 172.21.15.66:53196) 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 -- 172.21.15.66:0/2424195740 learned_addr learned my addr 172.21.15.66:0/2424195740 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34affd640 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41950 0x55a3d4d43f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 -- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 msgr2=0x55a3d4d5f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 -- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41950 msgr2=0x55a3d4d43f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41950 0x55a3d4d43f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.015+0000 7fc34bfff640 1 -- 172.21.15.66:0/2424195740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d4952210 con 0x55a3d4d454a0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc34bfff640 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 0x55a3d4d47bb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc344007a90 tx=0x7fc344001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba735d88ee3b819 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc32a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc32a7fc640 1 -- 172.21.15.66:0/2424195740 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34401b060 con 0x55a3d4d454a0 2024-03-30T13:12:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc32a7fc640 1 -- 172.21.15.66:0/2424195740 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc344016cc0 con 0x55a3d4d454a0 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc334062370 con 0x55a3d4d454a0 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc32a7fc640 1 -- 172.21.15.66:0/2424195740 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc3440124d0 con 0x55a3d4d454a0 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 10 client.6252 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55a3d4abf0b0 con 0x55a3d4d454a0 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc32a7fc640 1 -- 172.21.15.66:0/2424195740 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc334062370 con 0x55a3d4d454a0 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc32a7fc640 1 -- 172.21.15.66:0/2424195740 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fc344012670 con 0x55a3d4d454a0 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 10 client.6252 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 10 client.6252 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 20 client.6252 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d5e260 0x55a3d4d5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 4 client.6252 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d4c8bad0 con 0x55a3d4d5e260 2024-03-30T13:12:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.016+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d5e260 0x55a3d4d5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d5e260 0x55a3d4d5e670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc0675bda84913 server_cookie=1ea2f0cc3c52a1b1 in_seq=0 out_seq=0 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc32a7fc640 10 client.6252 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc32a7fc640 1 -- 172.21.15.66:0/2424195740 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55a3d4c8bad0 con 0x55a3d4d5e260 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc32a7fc640 10 client.6252 handle_command_reply: tid=0 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 client.6252 shutdown 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 msgr2=0x55a3d4d47bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 0x55a3d4d47bb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc344007a90 tx=0x7fc344001a00 comp rx=0 tx=0).stop 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 shutdown_connections 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d5e260 0x55a3d4d5e670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d454a0 0x55a3d4d47bb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d41950 0x55a3d4d43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc32a7fc640 0 client.6252 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 --2- 172.21.15.66:0/2424195740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.017+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 >> 172.21.15.66:0/2424195740 conn(0x55a3d4d14a50 msgr2=0x55a3d4d14200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.018+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 shutdown_connections 2024-03-30T13:12:41.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.018+0000 7fc354b91740 1 -- 172.21.15.66:0/2424195740 wait complete. 2024-03-30T13:12:41.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.018+0000 7fc354b91740 20 client.6252 trim_cache size 0 max 16384 2024-03-30T13:12:41.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.037+0000 7fc354b91740 1 Processor -- start 2024-03-30T13:12:41.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.037+0000 7fc354b91740 1 -- start start 2024-03-30T13:12:41.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.038+0000 7fc354b91740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.038+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.038+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.038+0000 7fc354b91740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c200f70 con 0x55a3d4d41570 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.038+0000 7fc354b91740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ffff0 con 0x55a3d4d454a0 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.038+0000 7fc354b91740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c158b10 con 0x55a3d4d41950 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53208/0 (socket says 172.21.15.66:53208) 2024-03-30T13:12:41.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 -- 172.21.15.66:0/1305782780 learned_addr learned my addr 172.21.15.66:0/1305782780 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34affd640 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34bfff640 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 -- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41950 msgr2=0x55a3d4d60b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 -- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 msgr2=0x55a3d4d610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.039+0000 7fc34b7fe640 1 -- 172.21.15.66:0/1305782780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d4d47f20 con 0x55a3d4d41570 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.040+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0090b0 tx=0x7fc33c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11321519997ac98 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.040+0000 7fc340ff9640 1 -- 172.21.15.66:0/1305782780 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c05b070 con 0x55a3d4d41570 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.040+0000 7fc340ff9640 1 -- 172.21.15.66:0/1305782780 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c044d80 con 0x55a3d4d41570 2024-03-30T13:12:41.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.040+0000 7fc340ff9640 1 -- 172.21.15.66:0/1305782780 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c034040 con 0x55a3d4d41570 2024-03-30T13:12:41.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.041+0000 7fc354b91740 1 -- 172.21.15.66:0/1305782780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 msgr2=0x55a3d4d5f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0090b0 tx=0x7fc33c030000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 -- 172.21.15.66:0/1305782780 shutdown_connections 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 --2- 172.21.15.66:0/1305782780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41570 0x55a3d4d5f8c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 -- 172.21.15.66:0/1305782780 >> 172.21.15.66:0/1305782780 conn(0x55a3d4d14a50 msgr2=0x55a3d4d1b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 -- 172.21.15.66:0/1305782780 shutdown_connections 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.042+0000 7fc354b91740 1 -- 172.21.15.66:0/1305782780 wait complete. 2024-03-30T13:12:41.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.044+0000 7fc354b91740 1 -- start start 2024-03-30T13:12:41.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.044+0000 7fc354b91740 1 Processor -- start 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.045+0000 7fc354b91740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4dd04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.045+0000 7fc354b91740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.045+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4dd04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4dd04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:38034/0 (socket says 172.21.15.66:38034) 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53216/0 (socket says 172.21.15.66:53216) 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34b7fe640 1 -- 172.21.15.66:0/3575547953 learned_addr learned my addr 172.21.15.66:0/3575547953 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4d1a4f0 con 0x55a3d4d41950 2024-03-30T13:12:41.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ff9d0 con 0x55a3d4d454a0 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55a3d4de3f50 con 0x55a3d4d41570 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc354b91740 10 client.0 _ll_register_callbacks cb 0x7fc344005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34bfff640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 -- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 msgr2=0x55a3d4dd04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4dd04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 -- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 msgr2=0x55a3d4d133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d492b2d0 con 0x55a3d4d41950 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34bfff640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.046+0000 7fc34affd640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d60b80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c520 tx=0x7fc334009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fa6aff3f246706 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc32a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc32a7fc640 1 -- 172.21.15.66:0/3575547953 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334055ae0 con 0x55a3d4d41950 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc32a7fc640 1 -- 172.21.15.66:0/3575547953 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7fc334058e90 con 0x55a3d4d41950 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc33c044d80 con 0x55a3d4d41950 2024-03-30T13:12:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc32a7fc640 1 -- 172.21.15.66:0/3575547953 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc334062370 con 0x55a3d4d41950 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc354b91740 10 client.6258 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55a3d4de3c80 con 0x55a3d4d41950 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.047+0000 7fc32a7fc640 1 -- 172.21.15.66:0/3575547953 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c044d80 con 0x55a3d4d41950 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc32a7fc640 1 -- 172.21.15.66:0/3575547953 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fc334055590 con 0x55a3d4d41950 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc354b91740 10 client.6258 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc354b91740 10 client.6258 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc354b91740 20 client.6258 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d138e0 0x55a3d4d82050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d138e0 0x55a3d4d82050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc354b91740 4 client.6258 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628374}] 2024-03-30T13:12:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.048+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628374}) -- 0x55a3d4c85e20 con 0x55a3d4d138e0 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc32a7fc640 10 client.6258 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc34b7fe640 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d138e0 0x55a3d4d82050 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a79102d7a4db1e server_cookie=8b9904ada7a27d63 in_seq=0 out_seq=0 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc32a7fc640 1 -- 172.21.15.66:0/3575547953 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3976 (crc 0 0 0) 0x55a3d4c85e20 con 0x55a3d4d138e0 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc32a7fc640 10 client.6258 handle_command_reply: tid=0 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc354b91740 1 client.6258 shutdown 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 msgr2=0x55a3d4d60b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d60b80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c520 tx=0x7fc334009f20 comp rx=0 tx=0).stop 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 shutdown_connections 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55a3d4d138e0 0x55a3d4d82050 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.049+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3d4d454a0 0x55a3d4d133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3d4d41950 0x55a3d4d60b80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc354b91740 1 --2- 172.21.15.66:0/3575547953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a3d4d41570 0x55a3d4dd04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc32a7fc640 0 client.6258 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 >> 172.21.15.66:0/3575547953 conn(0x55a3d4d14a50 msgr2=0x55a3d4d71680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 shutdown_connections 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc354b91740 1 -- 172.21.15.66:0/3575547953 wait complete. 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc354b91740 20 client.6258 trim_cache size 0 max 16384 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ceph/ioctl.h", 2024-03-30T13:12:41.057 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628374, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:07.997008+0000", 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.496819+0000", 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:41.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 3724, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:41.059 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.496819+0000", 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:41.060 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3724, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.997008+0000" 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3724, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.061 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:07.997008+0000" 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 206, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 34, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:41.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.041819+0000", 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 122, 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.063 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.064 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:41.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:41.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:41.067 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:41.068 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:41.069 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:41.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc32c06c530 msgr2=0x7fc32c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc351ec8640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc32c06c530 0x7fc32c06e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc33c036000 tx=0x7fc33c008030 comp rx=0 tx=0).stop 2024-03-30T13:12:41.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 msgr2=0x7fc34c15c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.050+0000 7fc351ec8640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c15c200 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc33400e840 tx=0x7fc33400ed00 comp rx=0 tx=0).stop 2024-03-30T13:12:41.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 shutdown_connections 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc32c06c530 0x7fc32c06e9f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc34c0ba090 0x7fc34c161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc34c0b5dc0 0x7fc34c15c200 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 --2- 172.21.15.66:0/1102144180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34c06c620 0x7fc34c15fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 >> 172.21.15.66:0/1102144180 conn(0x7fc34c0678f0 msgr2=0x7fc34c0b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 shutdown_connections 2024-03-30T13:12:41.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.051+0000 7fc351ec8640 1 -- 172.21.15.66:0/1102144180 wait complete. 2024-03-30T13:12:41.072 DEBUG:tasks.cephfs.test_uninlining:getting inode info #6 of 23:1099511628386 2024-03-30T13:12:41.072 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628386 2024-03-30T13:12:41.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628386 2024-03-30T13:12:41.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.229+0000 7f51ce8c4640 1 Processor -- start 2024-03-30T13:12:41.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.229+0000 7f51ce8c4640 1 -- start start 2024-03-30T13:12:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.230+0000 7f51ce8c4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51c80ba090 0x7f51c80d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.230+0000 7f51ce8c4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c80d2ff0 con 0x7f51c80b73d0 2024-03-30T13:12:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.230+0000 7f51ce8c4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c80d32d0 con 0x7f51c80b5dc0 2024-03-30T13:12:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.230+0000 7f51ce8c4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f51c80d35b0 con 0x7f51c80ba090 2024-03-30T13:12:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.230+0000 7f51cce61640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f51c80b73d0 0x7f51c80b7820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:44080/0 (socket says 172.21.15.66:44080) 2024-03-30T13:12:41.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.230+0000 7f51cce61640 1 -- 172.21.15.66:0/4198438524 learned_addr learned my addr 172.21.15.66:0/4198438524 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4198438524 >> v2:172.21.15.117:3300/0 conn(0x7f51c80ba090 0x7f51c80d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452581478 0 0) 0x7f51c80d2ff0 con 0x7f51c80b73d0 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51b0003520 con 0x7f51c80b73d0 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1723457945 0 0) 0x7f51b0003520 con 0x7f51c80b73d0 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 >> v2:172.21.15.117:3300/0 conn(0x7f51c80ba090 msgr2=0x7f51c80d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 --2- 172.21.15.66:0/4198438524 >> v2:172.21.15.117:3300/0 conn(0x7f51c80ba090 0x7f51c80d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 >> v1:172.21.15.117:6789/0 conn(0x7f51c80b5dc0 legacy=0x7f51c80b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c80d3890 con 0x7f51c80b73d0 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f51c00019f0 con 0x7f51c80b73d0 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 >> v1:172.21.15.66:6789/0 conn(0x7f51c80b73d0 legacy=0x7f51c80b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0004800 0x7f51b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b00051c0 0x7f51b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80d35b0 con 0x7f51b00039e0 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51b000c090 con 0x7f51b00051c0 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f51b000c290 con 0x7f51b0004800 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51cce61640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b00051c0 0x7f51b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0004800 0x7f51b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c7fff640 1 -- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0004800 msgr2=0x7f51b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0004800 0x7f51b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c7fff640 1 -- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b00051c0 msgr2=0x7f51b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b00051c0 0x7f51b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51cce61640 1 -- 172.21.15.66:0/4198438524 reap_dead start 2024-03-30T13:12:41.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.231+0000 7f51c7fff640 1 -- 172.21.15.66:0/4198438524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b0003520 con 0x7f51b00039e0 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51b0003e00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f51b8001750 tx=0x7f51b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c39e8d74ae4f9fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8001370 con 0x7f51b00039e0 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b80052d0 con 0x7f51b00039e0 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51c6ffd640 1 -- 172.21.15.66:0/4198438524 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8003830 con 0x7f51b00039e0 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 -- 172.21.15.66:0/4198438524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 msgr2=0x7f51b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51b0003e00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f51b8001750 tx=0x7f51b0005830 comp rx=0 tx=0).stop 2024-03-30T13:12:41.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 -- 172.21.15.66:0/4198438524 shutdown_connections 2024-03-30T13:12:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/4198438524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51b0003e00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 -- 172.21.15.66:0/4198438524 >> 172.21.15.66:0/4198438524 conn(0x7f51c80678f0 msgr2=0x7f51c80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 -- 172.21.15.66:0/4198438524 shutdown_connections 2024-03-30T13:12:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.233+0000 7f51ce8c4640 1 -- 172.21.15.66:0/4198438524 wait complete. 2024-03-30T13:12:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.235+0000 7f51ce8c4640 1 Processor -- start 2024-03-30T13:12:41.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.235+0000 7f51ce8c4640 1 -- start start 2024-03-30T13:12:41.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51ce8c4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51c8160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51ce8c4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c81f7620 0x7f51c814d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51ce8c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51c80ccf10 0x7f51c80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51ce8c4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51c815e880 con 0x7f51b00039e0 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51ce8c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80d4090 con 0x7f51c80ccf10 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51ce8c4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f51c815ea00 con 0x7f51c81f7620 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51c8160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51c8160d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53232/0 (socket says 172.21.15.66:53232) 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.236+0000 7f51c7fff640 1 -- 172.21.15.66:0/807977366 learned_addr learned my addr 172.21.15.66:0/807977366 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c77fe640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c81f7620 0x7f51c814d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51cce61640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51c80ccf10 0x7f51c80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c7fff640 1 -- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c81f7620 msgr2=0x7f51c814d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c7fff640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c81f7620 0x7f51c814d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c7fff640 1 -- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51c80ccf10 msgr2=0x7f51c80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c7fff640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51c80ccf10 0x7f51c80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c7fff640 1 -- 172.21.15.66:0/807977366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c81f7be0 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51cce61640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51c80ccf10 0x7f51c80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c7fff640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51c8160d20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005040 tx=0x7f51b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c4ff9640 1 -- 172.21.15.66:0/807977366 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8041020 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c4ff9640 1 -- 172.21.15.66:0/807977366 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b8005480 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51c4ff9640 1 -- 172.21.15.66:0/807977366 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8007a30 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c81f7ec0 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.237+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81f8480 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.239+0000 7f51c4ff9640 1 -- 172.21.15.66:0/807977366 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f51b8044020 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.239+0000 7f51c4ff9640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f51b406c530 0x7f51b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.239+0000 7f51c77fe640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f51b406c530 0x7f51b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.239+0000 7f51c4ff9640 1 -- 172.21.15.66:0/807977366 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f51b80ad320 con 0x7f51b00039e0 2024-03-30T13:12:41.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.240+0000 7f51c77fe640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f51b406c530 0x7f51b406e9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51c815db90 tx=0x7f51bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.240+0000 7f51d1582740 1 Processor -- start 2024-03-30T13:12:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.240+0000 7f51d1582740 1 -- start start 2024-03-30T13:12:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51d1582740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84cd3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84cd3a50 0x558c84d03840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51d1582740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c84c3d590 con 0x558c84d00570 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558c84bdbd30 con 0x558c84d00950 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558c84c39540 con 0x558c84cd3a50 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84cd3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84cd3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53234/0 (socket says 172.21.15.66:53234) 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 -- 172.21.15.66:0/4099166121 learned_addr learned my addr 172.21.15.66:0/4099166121 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51cce61640 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84cd3a50 0x558c84d03840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 -- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84cd3a50 msgr2=0x558c84d03840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84cd3a50 0x558c84d03840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 -- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 msgr2=0x558c84d03300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d03300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c7fff640 1 -- 172.21.15.66:0/4099166121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c84a1d920 con 0x558c84d00570 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.241+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d03300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84cd3510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f51b80836f0 tx=0x7f51b807b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c32d82ccb51f2ff server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51adffb640 1 -- 172.21.15.66:0/4099166121 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8093030 con 0x558c84d00570 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51adffb640 1 -- 172.21.15.66:0/4099166121 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b8006e10 con 0x558c84d00570 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51adffb640 1 -- 172.21.15.66:0/4099166121 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b80792b0 con 0x558c84d00570 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 -- 172.21.15.66:0/4099166121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 msgr2=0x558c84cd3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84cd3510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f51b80836f0 tx=0x7f51b807b070 comp rx=0 tx=0).stop 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 -- 172.21.15.66:0/4099166121 shutdown_connections 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84cd3a50 0x558c84d03840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d03300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 --2- 172.21.15.66:0/4099166121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84cd3510 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 -- 172.21.15.66:0/4099166121 >> 172.21.15.66:0/4099166121 conn(0x558c84cd5780 msgr2=0x558c84d1e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.242+0000 7f51d1582740 1 -- 172.21.15.66:0/4099166121 shutdown_connections 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.243+0000 7f51d1582740 1 -- 172.21.15.66:0/4099166121 wait complete. 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.243+0000 7f51d1582740 1 -- start start 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.243+0000 7f51d1582740 1 Processor -- start 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84cf89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d1cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84cf94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c84bdbd30 con 0x558c84cf9050 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558c84bd99f0 con 0x558c84d00950 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558c84c3d590 con 0x558c84d00570 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51d1582740 10 client.0 _ll_register_callbacks cb 0x558c84bddc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51cce61640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84cf94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51cce61640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84cf94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53244/0 (socket says 172.21.15.66:53244) 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51cce61640 1 -- 172.21.15.66:0/2791583380 learned_addr learned my addr 172.21.15.66:0/2791583380 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51c7fff640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84cf89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.244+0000 7f51c77fe640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d1cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51cce61640 1 -- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 msgr2=0x558c84cf89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51cce61640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84cf89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51cce61640 1 -- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 msgr2=0x558c84d1cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51cce61640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d1cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51cce61640 1 -- 172.21.15.66:0/2791583380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c84911210 con 0x558c84cf9050 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51cce61640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84cf94b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f51c80ceb90 tx=0x7f51c0007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53a36f18cf53393 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2791583380 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0019020 con 0x558c84cf9050 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2791583380 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f51c000a860 con 0x558c84cf9050 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b8006e10 con 0x558c84cf9050 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2791583380 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f51c0023050 con 0x558c84cf9050 2024-03-30T13:12:41.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51d1582740 10 client.6270 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x558c84a7e0b0 con 0x558c84cf9050 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2791583380 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b8006e10 con 0x558c84cf9050 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.245+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2791583380 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f51c0026020 con 0x558c84cf9050 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51d1582740 10 client.6270 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51d1582740 10 client.6270 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51d1582740 20 client.6270 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51d1582740 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84ce7f30 0x558c84ce8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51d1582740 4 client.6270 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c84c4aad0 con 0x558c84ce7f30 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51c7fff640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84ce7f30 0x558c84ce8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51a6ffd640 10 client.6270 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.246+0000 7f51c7fff640 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84ce7f30 0x558c84ce8330 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc0d64c5367c7cb server_cookie=87ddfad15d85ca52 in_seq=0 out_seq=0 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2791583380 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x558c84c4aad0 con 0x558c84ce7f30 2024-03-30T13:12:41.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51a6ffd640 10 client.6270 handle_command_reply: tid=0 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 client.6270 shutdown 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 msgr2=0x558c84cf94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84cf94b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f51c80ceb90 tx=0x7f51c0007810 comp rx=0 tx=0).stop 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 shutdown_connections 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84ce7f30 0x558c84ce8330 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84cf94b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d1cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 --2- 172.21.15.66:0/2791583380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84cf89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51a6ffd640 0 client.6270 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 >> 172.21.15.66:0/2791583380 conn(0x558c84cd3a50 msgr2=0x558c84d02e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 shutdown_connections 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.247+0000 7f51d1582740 1 -- 172.21.15.66:0/2791583380 wait complete. 2024-03-30T13:12:41.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.248+0000 7f51d1582740 20 client.6270 trim_cache size 0 max 16384 2024-03-30T13:12:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.262+0000 7f51d1582740 1 Processor -- start 2024-03-30T13:12:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.262+0000 7f51d1582740 1 -- start start 2024-03-30T13:12:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84cf9050 0x558c84cd4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51d1582740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84d03160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00950 0x558c84d42bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51d1582740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8201bb0 con 0x558c84d00570 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8200b90 con 0x558c84cf9050 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f51c81e8270 con 0x558c84d00950 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84d03160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84d03160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53258/0 (socket says 172.21.15.66:53258) 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 -- 172.21.15.66:0/4177113491 learned_addr learned my addr 172.21.15.66:0/4177113491 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c7fff640 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84cf9050 0x558c84cd4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51cce61640 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00950 0x558c84d42bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 -- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00950 msgr2=0x558c84d42bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00950 0x558c84d42bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 -- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84cf9050 msgr2=0x558c84cd4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84cf9050 0x558c84cd4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 -- 172.21.15.66:0/4177113491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c848ea2d0 con 0x558c84d00570 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51c77fe640 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84d03160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f51bc014e20 tx=0x7f51bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05241c36f718399 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51ad7fa640 1 -- 172.21.15.66:0/4177113491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc014e50 con 0x558c84d00570 2024-03-30T13:12:41.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51ad7fa640 1 -- 172.21.15.66:0/4177113491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc029030 con 0x558c84d00570 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.263+0000 7f51ad7fa640 1 -- 172.21.15.66:0/4177113491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc025850 con 0x558c84d00570 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 -- 172.21.15.66:0/4177113491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 msgr2=0x558c84d03160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84d03160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f51bc014e20 tx=0x7f51bc017000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 -- 172.21.15.66:0/4177113491 shutdown_connections 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00950 0x558c84d42bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84d00570 0x558c84d03160 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 --2- 172.21.15.66:0/4177113491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84cf9050 0x558c84cd4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 -- 172.21.15.66:0/4177113491 >> 172.21.15.66:0/4177113491 conn(0x558c84cd3a50 msgr2=0x558c84cf8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 -- 172.21.15.66:0/4177113491 shutdown_connections 2024-03-30T13:12:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.264+0000 7f51d1582740 1 -- 172.21.15.66:0/4177113491 wait complete. 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.265+0000 7f51d1582740 1 -- start start 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.265+0000 7f51d1582740 1 Processor -- start 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84ce6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84ce72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c84ccedc0 con 0x558c84cf9050 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51c82004e0 con 0x558c84d00950 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x558c84cd2ff0 con 0x558c84d00570 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 10 client.0 _ll_register_callbacks cb 0x7f51c0001250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84ce6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84ce6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53262/0 (socket says 172.21.15.66:53262) 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 -- 172.21.15.66:0/2507042096 learned_addr learned my addr 172.21.15.66:0/2507042096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51cce61640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c77fe640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84ce72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 -- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 msgr2=0x558c84ce72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84ce72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 -- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 msgr2=0x558c84d9cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d9cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 -- 172.21.15.66:0/2507042096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c84d99510 con 0x558c84cf9050 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c77fe640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84ce72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51c7fff640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84ce6db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f51b807acb0 tx=0x7f51b80ad050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d19810216640223 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2507042096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8093040 con 0x558c84cf9050 2024-03-30T13:12:41.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2507042096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f51b8079050 con 0x558c84cf9050 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc029030 con 0x558c84cf9050 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2507042096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f51b80ab2b0 con 0x558c84cf9050 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 10 client.6276 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.267+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x558c84d99810 con 0x558c84cf9050 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2507042096 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc029030 con 0x558c84cf9050 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2507042096 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f51c0001870 con 0x558c84cf9050 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51d1582740 10 client.6276 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51d1582740 10 client.6276 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51d1582740 20 client.6276 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51d1582740 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84d210a0 0x558c84d9d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51d1582740 4 client.6276 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628386}] 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628386}) -- 0x558c84c44e20 con 0x558c84d210a0 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51c77fe640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84d210a0 0x558c84d9d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51a6ffd640 10 client.6276 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.268+0000 7f51c77fe640 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84d210a0 0x558c84d9d220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344dfa28858dc228 server_cookie=ea167a66ab259761 in_seq=0 out_seq=0 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51a6ffd640 1 -- 172.21.15.66:0/2507042096 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3974 (crc 0 0 0) 0x558c84c44e20 con 0x558c84d210a0 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51a6ffd640 10 client.6276 handle_command_reply: tid=0 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 client.6276 shutdown 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 msgr2=0x558c84ce6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84ce6db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f51b807acb0 tx=0x7f51b80ad050 comp rx=0 tx=0).stop 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 shutdown_connections 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x558c84d210a0 0x558c84d9d220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c84d00950 0x558c84d9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558c84d00570 0x558c84ce72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51a6ffd640 0 client.6276 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 --2- 172.21.15.66:0/2507042096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c84cf9050 0x558c84ce6db0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 >> 172.21.15.66:0/2507042096 conn(0x558c84cd3a50 msgr2=0x558c84cd7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 shutdown_connections 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 1 -- 172.21.15.66:0/2507042096 wait complete. 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.269+0000 7f51d1582740 20 client.6276 trim_cache size 0 max 16384 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/Kconfig", 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628386, 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:41.279 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.008418+0000", 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.524819+0000", 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:41.280 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1240, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.524819+0000", 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:41.281 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:41.282 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1240, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.008418+0000" 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1240, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.008418+0000" 2024-03-30T13:12:41.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 142, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 2, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.051352+0000", 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 40, 2024-03-30T13:12:41.284 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:41.285 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:41.286 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:41.287 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.288 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.289 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:41.290 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:41.291 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:41.292 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:41.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f51b406c530 msgr2=0x7f51b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f51b406c530 0x7f51b406e9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51c815db90 tx=0x7f51bc008040 comp rx=0 tx=0).stop 2024-03-30T13:12:41.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 msgr2=0x7f51c8160d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51c8160d20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005040 tx=0x7f51b8001120 comp rx=0 tx=0).stop 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 shutdown_connections 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f51b406c530 0x7f51b406e9f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51c80ccf10 0x7f51c80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51c81f7620 0x7f51c814d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 --2- 172.21.15.66:0/807977366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51b00039e0 0x7f51c8160d20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 >> 172.21.15.66:0/807977366 conn(0x7f51c80678f0 msgr2=0x7f51c80b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 shutdown_connections 2024-03-30T13:12:41.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.270+0000 7f51ce8c4640 1 -- 172.21.15.66:0/807977366 wait complete. 2024-03-30T13:12:41.294 DEBUG:tasks.cephfs.test_uninlining:getting inode info #7 of 23:1099511628404 2024-03-30T13:12:41.294 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628404 2024-03-30T13:12:41.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628404 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.401+0000 7fe62cb38640 1 Processor -- start 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.401+0000 7fe62cb38640 1 -- start start 2024-03-30T13:12:41.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62cb38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe628145d70 0x7fe628160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62cb38640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6280ce490 0x7fe6280cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62cb38640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6280cc970 con 0x7fe6280d0290 2024-03-30T13:12:41.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62cb38640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe6280ce8f0 con 0x7fe6280ce490 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62cb38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe628146150 con 0x7fe628145d70 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe625d9b640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6280ce490 0x7fe6280cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe628145d70 0x7fe628160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe626d9d640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe6280d0290 0x7fe62815ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:43194/0 (socket says 172.21.15.66:43194) 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe626d9d640 1 -- 172.21.15.66:0/3517070568 learned_addr learned my addr 172.21.15.66:0/3517070568 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62559a640 1 -- 172.21.15.66:0/3517070568 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972353223 0 0) 0x7fe6280cc970 con 0x7fe6280d0290 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62659c640 1 -- 172.21.15.66:0/3517070568 >> v2:172.21.15.117:3300/0 conn(0x7fe6280ce490 msgr2=0x7fe6280cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62659c640 1 --2- 172.21.15.66:0/3517070568 >> v2:172.21.15.117:3300/0 conn(0x7fe6280ce490 0x7fe6280cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62659c640 1 -- 172.21.15.66:0/3517070568 >> v1:172.21.15.117:6789/0 conn(0x7fe6280d0290 legacy=0x7fe62815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.402+0000 7fe62659c640 1 -- 172.21.15.66:0/3517070568 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6280ccc50 con 0x7fe628145d70 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.403+0000 7fe62659c640 1 --2- 172.21.15.66:0/3517070568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe628145d70 0x7fe628160880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6180097a0 tx=0x7fe61802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f0c76e0ba79d17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.403+0000 7fe62559a640 1 -- 172.21.15.66:0/3517070568 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6180315a0 con 0x7fe628145d70 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.403+0000 7fe62559a640 1 -- 172.21.15.66:0/3517070568 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618037070 con 0x7fe628145d70 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.403+0000 7fe62559a640 1 -- 172.21.15.66:0/3517070568 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618031960 con 0x7fe628145d70 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 -- 172.21.15.66:0/3517070568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe628145d70 msgr2=0x7fe628160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 --2- 172.21.15.66:0/3517070568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe628145d70 0x7fe628160880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6180097a0 tx=0x7fe61802b840 comp rx=0 tx=0).stop 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 -- 172.21.15.66:0/3517070568 shutdown_connections 2024-03-30T13:12:41.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 --2- 172.21.15.66:0/3517070568 >> v2:172.21.15.117:3300/0 conn(0x7fe6280ce490 0x7fe6280cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 --2- 172.21.15.66:0/3517070568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe628145d70 0x7fe628160880 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 -- 172.21.15.66:0/3517070568 >> 172.21.15.66:0/3517070568 conn(0x7fe6280b3e70 msgr2=0x7fe6280b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 -- 172.21.15.66:0/3517070568 shutdown_connections 2024-03-30T13:12:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.404+0000 7fe62cb38640 1 -- 172.21.15.66:0/3517070568 wait complete. 2024-03-30T13:12:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.405+0000 7fe62cb38640 1 Processor -- start 2024-03-30T13:12:41.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.405+0000 7fe62cb38640 1 -- start start 2024-03-30T13:12:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62cb38640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6280ce490 0x7fe62813ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62cb38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 0x7fe62815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62cb38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe628145d70 0x7fe628155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62cb38640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6280ccf30 con 0x7fe6280d0290 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62cb38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe62815ca20 con 0x7fe628145d70 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62cb38640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe62815d5c0 con 0x7fe6280ce490 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 0x7fe62815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 0x7fe62815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53264/0 (socket says 172.21.15.66:53264) 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 -- 172.21.15.66:0/1659431607 learned_addr learned my addr 172.21.15.66:0/1659431607 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe626d9d640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe628145d70 0x7fe628155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe62659c640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6280ce490 0x7fe62813ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 -- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6280ce490 msgr2=0x7fe62813ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6280ce490 0x7fe62813ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 -- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe628145d70 msgr2=0x7fe628155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe628145d70 0x7fe628155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe625d9b640 1 -- 172.21.15.66:0/1659431607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62813f2d0 con 0x7fe6280d0290 2024-03-30T13:12:41.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.406+0000 7fe626d9d640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe628145d70 0x7fe628155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.407+0000 7fe625d9b640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 0x7fe62815f4e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00e9b0 tx=0x7fe61c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.407+0000 7fe616ffd640 1 -- 172.21.15.66:0/1659431607 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c017070 con 0x7fe6280d0290 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.407+0000 7fe616ffd640 1 -- 172.21.15.66:0/1659431607 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61c00c9a0 con 0x7fe6280d0290 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.407+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62813f5b0 con 0x7fe6280d0290 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.407+0000 7fe616ffd640 1 -- 172.21.15.66:0/1659431607 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00cd90 con 0x7fe6280d0290 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.407+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62813fb70 con 0x7fe6280d0290 2024-03-30T13:12:41.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.409+0000 7fe616ffd640 1 -- 172.21.15.66:0/1659431607 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe61c01a070 con 0x7fe6280d0290 2024-03-30T13:12:41.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.409+0000 7fe616ffd640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe60806c530 0x7fe60806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.409+0000 7fe616ffd640 1 -- 172.21.15.66:0/1659431607 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7fe61c08bce0 con 0x7fe6280d0290 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.409+0000 7fe62659c640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe60806c530 0x7fe60806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.410+0000 7fe62659c640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe60806c530 0x7fe60806e9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe618036000 tx=0x7fe61803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.410+0000 7fe62f7f5740 1 Processor -- start 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.410+0000 7fe62f7f5740 1 -- start start 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abda510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac0a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d06ab44590 con 0x55d06ac07570 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d06aae2d30 con 0x55d06abdaa50 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d06ab40540 con 0x55d06ac07950 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abda510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abda510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53278/0 (socket says 172.21.15.66:53278) 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe62659c640 1 -- 172.21.15.66:0/66869005 learned_addr learned my addr 172.21.15.66:0/66869005 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe626d9d640 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.411+0000 7fe625d9b640 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe62659c640 1 -- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 msgr2=0x55d06ac0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe62659c640 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe62659c640 1 -- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 msgr2=0x55d06ac0a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe62659c640 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac0a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe62659c640 1 -- 172.21.15.66:0/66869005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06a924920 con 0x55d06ac07570 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe62659c640 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abda510 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe618034040 tx=0x7fe618030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df813194b7bb09d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe607fff640 1 -- 172.21.15.66:0/66869005 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618032980 con 0x55d06ac07570 2024-03-30T13:12:41.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe607fff640 1 -- 172.21.15.66:0/66869005 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618037040 con 0x55d06ac07570 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.412+0000 7fe607fff640 1 -- 172.21.15.66:0/66869005 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618032d20 con 0x55d06ac07570 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 -- 172.21.15.66:0/66869005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 msgr2=0x55d06abda510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abda510 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe618034040 tx=0x7fe618030000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 -- 172.21.15.66:0/66869005 shutdown_connections 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac0a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/66869005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abda510 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 -- 172.21.15.66:0/66869005 >> 172.21.15.66:0/66869005 conn(0x55d06abdc780 msgr2=0x55d06ac25300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 -- 172.21.15.66:0/66869005 shutdown_connections 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 -- 172.21.15.66:0/66869005 wait complete. 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.413+0000 7fe62f7f5740 1 -- start start 2024-03-30T13:12:41.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.414+0000 7fe62f7f5740 1 Processor -- start 2024-03-30T13:12:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.414+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07570 0x55d06ac0e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.414+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 0x55d06ac0c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.414+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac11940 0x55d06abfb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.414+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d06abdee20 con 0x55d06ac07950 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.414+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d06aae09f0 con 0x55d06ac11940 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d06abe1660 con 0x55d06ac07570 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe62f7f5740 10 client.0 _ll_register_callbacks cb 0x55d06aae4c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 0x55d06ac0c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 0x55d06ac0c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53282/0 (socket says 172.21.15.66:53282) 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 -- 172.21.15.66:0/3246636800 learned_addr learned my addr 172.21.15.66:0/3246636800 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe626d9d640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac11940 0x55d06abfb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe62659c640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07570 0x55d06ac0e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 -- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07570 msgr2=0x55d06ac0e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07570 0x55d06ac0e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 -- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac11940 msgr2=0x55d06abfb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac11940 0x55d06abfb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 -- 172.21.15.66:0/3246636800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06a818210 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe625d9b640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 0x55d06ac0c920 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00e980 tx=0x7fe61c05a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab38cd0c0f5b264a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe6057fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe6057fa640 1 -- 172.21.15.66:0/3246636800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c021040 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe6057fa640 1 -- 172.21.15.66:0/3246636800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7fe61c062360 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.415+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe618037040 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe6057fa640 1 -- 172.21.15.66:0/3246636800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe61c058e60 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 10 client.6285 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d06a9850b0 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe6057fa640 1 -- 172.21.15.66:0/3246636800 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618037040 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe6057fa640 1 -- 172.21.15.66:0/3246636800 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fe61c059000 con 0x55d06ac07950 2024-03-30T13:12:41.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 10 client.6285 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 10 client.6285 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 20 client.6285 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac111c0 0x55d06ac0ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 4 client.6285 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d06ab51ad0 con 0x55d06ac111c0 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62659c640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac111c0 0x55d06ac0ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe6057fa640 10 client.6285 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.416+0000 7fe62659c640 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac111c0 0x55d06ac0ec30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6a5cb0422fee0e server_cookie=fb631994e0c07811 in_seq=0 out_seq=0 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.417+0000 7fe6057fa640 1 -- 172.21.15.66:0/3246636800 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d06ab51ad0 con 0x55d06ac111c0 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.417+0000 7fe6057fa640 10 client.6285 handle_command_reply: tid=0 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.417+0000 7fe62f7f5740 1 client.6285 shutdown 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 msgr2=0x55d06ac0c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 0x55d06ac0c920 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00e980 tx=0x7fe61c05a250 comp rx=0 tx=0).stop 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 shutdown_connections 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac111c0 0x55d06ac0ec30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac11940 0x55d06abfb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07950 0x55d06ac0c920 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe6057fa640 0 client.6285 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3246636800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07570 0x55d06ac0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 >> 172.21.15.66:0/3246636800 conn(0x55d06abdaa50 msgr2=0x55d06abde630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 shutdown_connections 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3246636800 wait complete. 2024-03-30T13:12:41.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.418+0000 7fe62f7f5740 20 client.6285 trim_cache size 0 max 16384 2024-03-30T13:12:41.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.432+0000 7fe62f7f5740 1 Processor -- start 2024-03-30T13:12:41.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 -- start start 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 0x55d06ac37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac07570 0x55d06ac9aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac9afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6282011a0 con 0x55d06abdaa50 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6282001a0 con 0x55d06ac07570 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe62815e9d0 con 0x55d06ac07950 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 0x55d06ac37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 0x55d06ac37c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53296/0 (socket says 172.21.15.66:53296) 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.433+0000 7fe62659c640 1 -- 172.21.15.66:0/1580069383 learned_addr learned my addr 172.21.15.66:0/1580069383 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe625d9b640 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac07570 0x55d06ac9aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe626d9d640 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac9afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62659c640 1 -- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 msgr2=0x55d06ac9afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62659c640 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac9afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62659c640 1 -- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac07570 msgr2=0x55d06ac9aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62659c640 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac07570 0x55d06ac9aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62659c640 1 -- 172.21.15.66:0/1580069383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06a7f12d0 con 0x55d06abdaa50 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62659c640 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 0x55d06ac37c30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe6180098c0 tx=0x7fe618009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722ea670e0a9ad08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe6077fe640 1 -- 172.21.15.66:0/1580069383 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618057070 con 0x55d06abdaa50 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe6077fe640 1 -- 172.21.15.66:0/1580069383 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618037040 con 0x55d06abdaa50 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe6077fe640 1 -- 172.21.15.66:0/1580069383 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618050480 con 0x55d06abdaa50 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62f7f5740 1 -- 172.21.15.66:0/1580069383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 msgr2=0x55d06ac37c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.434+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 0x55d06ac37c30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe6180098c0 tx=0x7fe618009190 comp rx=0 tx=0).stop 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 -- 172.21.15.66:0/1580069383 shutdown_connections 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06ac9afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06ac07570 0x55d06ac9aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/1580069383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06abdaa50 0x55d06ac37c30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 -- 172.21.15.66:0/1580069383 >> 172.21.15.66:0/1580069383 conn(0x55d06abfb250 msgr2=0x55d06abe1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 -- 172.21.15.66:0/1580069383 shutdown_connections 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 -- 172.21.15.66:0/1580069383 wait complete. 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 -- start start 2024-03-30T13:12:41.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.435+0000 7fe62f7f5740 1 Processor -- start 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac8a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abde210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06abde750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d06ac0d900 con 0x55d06ac07570 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6281ffb50 con 0x55d06abdaa50 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d06ac49320 con 0x55d06ac07950 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62f7f5740 10 client.0 _ll_register_callbacks cb 0x7fe61c0598b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe625d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abde210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe62659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac8a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe625d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abde210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53298/0 (socket says 172.21.15.66:53298) 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe625d9b640 1 -- 172.21.15.66:0/3225861947 learned_addr learned my addr 172.21.15.66:0/3225861947 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.436+0000 7fe626d9d640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06abde750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62659c640 1 -- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 msgr2=0x55d06abde750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62659c640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06abde750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62659c640 1 -- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 msgr2=0x55d06abde210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62659c640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abde210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62659c640 1 -- 172.21.15.66:0/3225861947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06ac09de0 con 0x55d06abdaa50 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe625d9b640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abde210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62659c640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac8a0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55d06ac07180 tx=0x7fe618004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd98d166816a482 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe6057fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-03-30T13:12:41.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe6057fa640 1 -- 172.21.15.66:0/3225861947 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618057040 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe6057fa640 1 -- 172.21.15.66:0/3225861947 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7fe6180348e0 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe618037040 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe6057fa640 1 -- 172.21.15.66:0/3225861947 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe618034ae0 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62f7f5740 10 client.6028 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d06ac49120 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe6057fa640 1 -- 172.21.15.66:0/3225861947 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618037040 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe6057fa640 1 -- 172.21.15.66:0/3225861947 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fe618031070 con 0x55d06abdaa50 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62f7f5740 10 client.6028 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.437+0000 7fe62f7f5740 10 client.6028 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 20 client.6028 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac11940 0x55d06abdf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 4 client.6028 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628404}] 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628404}) -- 0x55d06ab4be20 con 0x55d06ac11940 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe625d9b640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac11940 0x55d06abdf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe6057fa640 10 client.6028 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe625d9b640 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac11940 0x55d06abdf2c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dd6ca6ea9e4a29 server_cookie=27991050c59894af in_seq=0 out_seq=0 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe6057fa640 1 -- 172.21.15.66:0/3225861947 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3980 (crc 0 0 0) 0x55d06ab4be20 con 0x55d06ac11940 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe6057fa640 10 client.6028 handle_command_reply: tid=0 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 1 client.6028 shutdown 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 msgr2=0x55d06ac8a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.438+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac8a0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55d06ac07180 tx=0x7fe618004730 comp rx=0 tx=0).stop 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 shutdown_connections 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d06ac11940 0x55d06abdf2c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d06ac07950 0x55d06abde750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d06ac07570 0x55d06abde210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 --2- 172.21.15.66:0/3225861947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d06abdaa50 0x55d06ac8a0a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe6057fa640 0 client.6028 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 >> 172.21.15.66:0/3225861947 conn(0x55d06abfb250 msgr2=0x55d06abfa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 shutdown_connections 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 1 -- 172.21.15.66:0/3225861947 wait complete. 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62f7f5740 20 client.6028 trim_cache size 0 max 16384 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr_user.c", 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628404, 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.010624+0000", 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.535819+0000", 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:41.449 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:41.450 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1232, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.535819+0000", 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:41.451 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1232, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.452 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.010624+0000" 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1232, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.010624+0000" 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 154, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 38, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:41.453 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.051956+0000", 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 42, 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.454 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:41.455 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:41.456 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:41.457 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:41.460 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:41.461 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:41.461 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe60806c530 msgr2=0x7fe60806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe60806c530 0x7fe60806e9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe618036000 tx=0x7fe61803d040 comp rx=0 tx=0).stop 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 msgr2=0x7fe62815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 0x7fe62815f4e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00e9b0 tx=0x7fe61c00ee70 comp rx=0 tx=0).stop 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 shutdown_connections 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe60806c530 0x7fe60806e9f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe628145d70 0x7fe628155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6280d0290 0x7fe62815f4e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 --2- 172.21.15.66:0/1659431607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6280ce490 0x7fe62813ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 >> 172.21.15.66:0/1659431607 conn(0x7fe6280b3e70 msgr2=0x7fe6280b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 shutdown_connections 2024-03-30T13:12:41.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.439+0000 7fe62cb38640 1 -- 172.21.15.66:0/1659431607 wait complete. 2024-03-30T13:12:41.463 DEBUG:tasks.cephfs.test_uninlining:getting inode info #8 of 23:1099511628402 2024-03-30T13:12:41.463 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628402 2024-03-30T13:12:41.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628402 2024-03-30T13:12:41.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.574+0000 7f09f3455640 1 Processor -- start 2024-03-30T13:12:41.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.574+0000 7f09f3455640 1 -- start start 2024-03-30T13:12:41.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f3455640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f3455640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09ec0ba090 0x7f09ec0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f3455640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f09ec06c620 0x7f09ec06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f3455640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec0b66e0 con 0x7f09ec0ba090 2024-03-30T13:12:41.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f3455640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec0d2ff0 con 0x7f09ec06c620 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f3455640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec0d32d0 con 0x7f09ec0b5dc0 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f09f0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09ec0ba090 0x7f09ec0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f09f0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f09ec0ba090 0x7f09ec0d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53304/0 (socket says 172.21.15.66:53304) 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f09f0640 1 -- 172.21.15.66:0/1455769674 learned_addr learned my addr 172.21.15.66:0/1455769674 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f19f2640 1 --2- 172.21.15.66:0/1455769674 >> v2:172.21.15.117:3300/0 conn(0x7f09ec06c620 0x7f09ec06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f11f1640 1 --2- 172.21.15.66:0/1455769674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f09f0640 1 -- 172.21.15.66:0/1455769674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 msgr2=0x7f09ec0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.575+0000 7f09f09f0640 1 --2- 172.21.15.66:0/1455769674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09f09f0640 1 -- 172.21.15.66:0/1455769674 >> v2:172.21.15.117:3300/0 conn(0x7f09ec06c620 msgr2=0x7f09ec06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09f09f0640 1 --2- 172.21.15.66:0/1455769674 >> v2:172.21.15.117:3300/0 conn(0x7f09ec06c620 0x7f09ec06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09f09f0640 1 -- 172.21.15.66:0/1455769674 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec0d35b0 con 0x7f09ec0ba090 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09f19f2640 1 --2- 172.21.15.66:0/1455769674 >> v2:172.21.15.117:3300/0 conn(0x7f09ec06c620 0x7f09ec06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09f09f0640 1 --2- 172.21.15.66:0/1455769674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0d2680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0d2120 tx=0x7f09e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dab5554a0691f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09ebfff640 1 -- 172.21.15.66:0/1455769674 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0035020 con 0x7f09ec0ba090 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.576+0000 7f09ebfff640 1 -- 172.21.15.66:0/1455769674 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0031870 con 0x7f09ec0ba090 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09ebfff640 1 -- 172.21.15.66:0/1455769674 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0031c40 con 0x7f09ec0ba090 2024-03-30T13:12:41.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 -- 172.21.15.66:0/1455769674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 msgr2=0x7f09ec0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 --2- 172.21.15.66:0/1455769674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0d2680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0d2120 tx=0x7f09e002b910 comp rx=0 tx=0).stop 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 -- 172.21.15.66:0/1455769674 shutdown_connections 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 --2- 172.21.15.66:0/1455769674 >> v2:172.21.15.117:3300/0 conn(0x7f09ec06c620 0x7f09ec06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 --2- 172.21.15.66:0/1455769674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0d2680 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 --2- 172.21.15.66:0/1455769674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 -- 172.21.15.66:0/1455769674 >> 172.21.15.66:0/1455769674 conn(0x7f09ec0678f0 msgr2=0x7f09ec0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 -- 172.21.15.66:0/1455769674 shutdown_connections 2024-03-30T13:12:41.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.577+0000 7f09f3455640 1 -- 172.21.15.66:0/1455769674 wait complete. 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.578+0000 7f09f3455640 1 Processor -- start 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.578+0000 7f09f3455640 1 -- start start 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f3455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ec06c620 0x7f09ec0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f3455640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f3455640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f3455640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0d4090 con 0x7f09ec0ba090 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f3455640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0d3b70 con 0x7f09ec06c620 2024-03-30T13:12:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f3455640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec15e880 con 0x7f09ec0b5dc0 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f11f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ec06c620 0x7f09ec0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53306/0 (socket says 172.21.15.66:53306) 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 -- 172.21.15.66:0/1737149267 learned_addr learned my addr 172.21.15.66:0/1737149267 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f09f0640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 -- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 msgr2=0x7f09ec15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 -- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ec06c620 msgr2=0x7f09ec0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ec06c620 0x7f09ec0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.579+0000 7f09f19f2640 1 -- 172.21.15.66:0/1737149267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec1f4a10 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.580+0000 7f09f19f2640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0b84d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00eed0 tx=0x7f09dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.580+0000 7f09e9ffb640 1 -- 172.21.15.66:0/1737149267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc018030 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.580+0000 7f09e9ffb640 1 -- 172.21.15.66:0/1737149267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc00bd20 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.580+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec1f4cf0 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.580+0000 7f09e9ffb640 1 -- 172.21.15.66:0/1737149267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc003bb0 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.580+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec0cc7e0 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.582+0000 7f09e9ffb640 1 -- 172.21.15.66:0/1737149267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f09dc01b070 con 0x7f09ec0ba090 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.582+0000 7f09e9ffb640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d806c530 0x7f09d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.582+0000 7f09e9ffb640 1 -- 172.21.15.66:0/1737149267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f09dc08bde0 con 0x7f09ec0ba090 2024-03-30T13:12:41.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.582+0000 7f09f11f1640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d806c530 0x7f09d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.583+0000 7f09f11f1640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d806c530 0x7f09d806e9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0b4770 tx=0x7f09e4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.584+0000 7f09f6047740 1 Processor -- start 2024-03-30T13:12:41.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.584+0000 7f09f6047740 1 -- start start 2024-03-30T13:12:41.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f6047740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 0x55cf4ee2e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee2ea50 0x55cf4ee5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f6047740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ed98590 con 0x55cf4ee5b570 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ed36d30 con 0x55cf4ee5b950 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55cf4ed94540 con 0x55cf4ee2ea50 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 0x55cf4ee2e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 0x55cf4ee2e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53322/0 (socket says 172.21.15.66:53322) 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 -- 172.21.15.66:0/3967422177 learned_addr learned my addr 172.21.15.66:0/3967422177 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f09f0640 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f19f2640 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee2ea50 0x55cf4ee5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 -- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee2ea50 msgr2=0x55cf4ee5e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee2ea50 0x55cf4ee5e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 -- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 msgr2=0x55cf4ee5e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee5e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.585+0000 7f09f11f1640 1 -- 172.21.15.66:0/3967422177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf4eb78920 con 0x55cf4ee5b570 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.586+0000 7f09f11f1640 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 0x55cf4ee2e510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f09e4011eb0 tx=0x7f09e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec0beca455989b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.586+0000 7f09d2ffd640 1 -- 172.21.15.66:0/3967422177 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e401b070 con 0x55cf4ee5b570 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.586+0000 7f09d2ffd640 1 -- 172.21.15.66:0/3967422177 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e4027070 con 0x55cf4ee5b570 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.586+0000 7f09d2ffd640 1 -- 172.21.15.66:0/3967422177 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e4003a20 con 0x55cf4ee5b570 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 -- 172.21.15.66:0/3967422177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 msgr2=0x55cf4ee2e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 0x55cf4ee2e510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f09e4011eb0 tx=0x7f09e4014000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 -- 172.21.15.66:0/3967422177 shutdown_connections 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee2ea50 0x55cf4ee5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee5e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 --2- 172.21.15.66:0/3967422177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5b570 0x55cf4ee2e510 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 -- 172.21.15.66:0/3967422177 >> 172.21.15.66:0/3967422177 conn(0x55cf4ee30780 msgr2=0x55cf4ee79300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 -- 172.21.15.66:0/3967422177 shutdown_connections 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.587+0000 7f09f6047740 1 -- 172.21.15.66:0/3967422177 wait complete. 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.588+0000 7f09f6047740 1 -- start start 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.588+0000 7f09f6047740 1 Processor -- start 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee53a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee8d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ee32e20 con 0x55cf4ee5db20 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ed349f0 con 0x55cf4ee5b950 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55cf4ee35660 con 0x55cf4ee5b570 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f6047740 10 client.0 _ll_register_callbacks cb 0x55cf4ed38c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee777f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53326/0 (socket says 172.21.15.66:53326) 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.589+0000 7f09f19f2640 1 -- 172.21.15.66:0/624837368 learned_addr learned my addr 172.21.15.66:0/624837368 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f11f1640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee53a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f09f0640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee8d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f19f2640 1 -- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 msgr2=0x55cf4ee53a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f19f2640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee53a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f19f2640 1 -- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 msgr2=0x55cf4ee8d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f19f2640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee8d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f19f2640 1 -- 172.21.15.66:0/624837368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4ea6c210 con 0x55cf4ee5db20 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f19f2640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee777f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f09dc059b40 tx=0x7f09dc059cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2605df03fefab1f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09cbfff640 1 -- 172.21.15.66:0/624837368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc087020 con 0x55cf4ee5db20 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09cbfff640 1 -- 172.21.15.66:0/624837368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f09dc059200 con 0x55cf4ee5db20 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e4027070 con 0x55cf4ee5db20 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09cbfff640 1 -- 172.21.15.66:0/624837368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f09dc0600b0 con 0x55cf4ee5db20 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 10 client.6303 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55cf4ebd90b0 con 0x55cf4ee5db20 2024-03-30T13:12:41.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09cbfff640 1 -- 172.21.15.66:0/624837368 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e4027070 con 0x55cf4ee5db20 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09cbfff640 1 -- 172.21.15.66:0/624837368 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f09dc025060 con 0x55cf4ee5db20 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 10 client.6303 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 10 client.6303 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 20 client.6303 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.590+0000 7f09f6047740 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee61d10 0x55cf4ee5f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.591+0000 7f09f6047740 4 client.6303 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.591+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf4eda5ad0 con 0x55cf4ee61d10 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.591+0000 7f09f11f1640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee61d10 0x55cf4ee5f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.591+0000 7f09cbfff640 10 client.6303 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.591+0000 7f09f11f1640 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee61d10 0x55cf4ee5f4a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd95bf4fee320cc8 server_cookie=8ca75eae4dd705c3 in_seq=0 out_seq=0 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09cbfff640 1 -- 172.21.15.66:0/624837368 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55cf4eda5ad0 con 0x55cf4ee61d10 2024-03-30T13:12:41.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09cbfff640 10 client.6303 handle_command_reply: tid=0 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 client.6303 shutdown 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 msgr2=0x55cf4ee777f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee777f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f09dc059b40 tx=0x7f09dc059cf0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 shutdown_connections 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee61d10 0x55cf4ee5f4a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee777f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee8d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09cbfff640 0 client.6303 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 --2- 172.21.15.66:0/624837368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee53a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 >> 172.21.15.66:0/624837368 conn(0x55cf4ee2ea50 msgr2=0x55cf4ee32560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 shutdown_connections 2024-03-30T13:12:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.592+0000 7f09f6047740 1 -- 172.21.15.66:0/624837368 wait complete. 2024-03-30T13:12:41.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.593+0000 7f09f6047740 20 client.6303 trim_cache size 0 max 16384 2024-03-30T13:12:41.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.623+0000 7f09f6047740 1 Processor -- start 2024-03-30T13:12:41.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.623+0000 7f09f6047740 1 -- start start 2024-03-30T13:12:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f6047740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f6047740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec201360 con 0x55cf4ee5db20 2024-03-30T13:12:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec200360 con 0x55cf4ee5b950 2024-03-30T13:12:41.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec1ff6c0 con 0x55cf4ee5b570 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee77170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53336/0 (socket says 172.21.15.66:53336) 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.624+0000 7f09f19f2640 1 -- 172.21.15.66:0/977699627 learned_addr learned my addr 172.21.15.66:0/977699627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f09f0640 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f11f1640 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f19f2640 1 -- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 msgr2=0x55cf4ee7b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f19f2640 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee7b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f19f2640 1 -- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 msgr2=0x55cf4ee7c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f19f2640 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f19f2640 1 -- 172.21.15.66:0/977699627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf4ea452d0 con 0x55cf4ee5db20 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f09f0640 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09f19f2640 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee77170 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f09dc057e60 tx=0x7f09dc06f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306a184c491ae093 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09d27fc640 1 -- 172.21.15.66:0/977699627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc081020 con 0x55cf4ee5db20 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09d27fc640 1 -- 172.21.15.66:0/977699627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc059000 con 0x55cf4ee5db20 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.625+0000 7f09d27fc640 1 -- 172.21.15.66:0/977699627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc087040 con 0x55cf4ee5db20 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 -- 172.21.15.66:0/977699627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 msgr2=0x55cf4ee77170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee77170 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f09dc057e60 tx=0x7f09dc06f000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 -- 172.21.15.66:0/977699627 shutdown_connections 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee77170 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 --2- 172.21.15.66:0/977699627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee7b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 -- 172.21.15.66:0/977699627 >> 172.21.15.66:0/977699627 conn(0x55cf4ee2ea50 msgr2=0x55cf4ee29740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 -- 172.21.15.66:0/977699627 shutdown_connections 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 -- 172.21.15.66:0/977699627 wait complete. 2024-03-30T13:12:41.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.626+0000 7f09f6047740 1 -- start start 2024-03-30T13:12:41.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 Processor -- start 2024-03-30T13:12:41.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee78dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee7c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec1ffd10 con 0x55cf4ee5db20 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf4ee2e800 con 0x55cf4ee5b950 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55cf4ee79310 con 0x55cf4ee5b570 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.627+0000 7f09f6047740 10 client.0 _ll_register_callbacks cb 0x7f09dc055f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee7c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee7c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53352/0 (socket says 172.21.15.66:53352) 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 -- 172.21.15.66:0/143101041 learned_addr learned my addr 172.21.15.66:0/143101041 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f09f0640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f11f1640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee78dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 -- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 msgr2=0x55cf4ee78dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee78dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 -- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 msgr2=0x55cf4ee7a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 -- 172.21.15.66:0/143101041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4ee56d30 con 0x55cf4ee5db20 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f19f2640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee7c5e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f09dc059b40 tx=0x7f09dc089fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffc8967795010f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09cbfff640 1 -- 172.21.15.66:0/143101041 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc069060 con 0x55cf4ee5db20 2024-03-30T13:12:41.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09cbfff640 1 -- 172.21.15.66:0/143101041 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f09dc058a70 con 0x55cf4ee5db20 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09dc059000 con 0x55cf4ee5db20 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09cbfff640 1 -- 172.21.15.66:0/143101041 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f09dc087040 con 0x55cf4ee5db20 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f6047740 10 client.6309 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.628+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55cf4ee56d30 con 0x55cf4ee5db20 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09cbfff640 1 -- 172.21.15.66:0/143101041 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc059000 con 0x55cf4ee5db20 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09cbfff640 1 -- 172.21.15.66:0/143101041 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f09dc058c90 con 0x55cf4ee5db20 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f6047740 10 client.6309 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f6047740 10 client.6309 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f6047740 20 client.6309 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f6047740 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee79b50 0x55cf4ee7cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f6047740 4 client.6309 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628402}] 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628402}) -- 0x55cf4ed9fe20 con 0x55cf4ee79b50 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f11f1640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee79b50 0x55cf4ee7cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09cbfff640 10 client.6309 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09f11f1640 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee79b50 0x55cf4ee7cb20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b765ea852a7ae467 server_cookie=c904391ac4746fd7 in_seq=0 out_seq=0 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09cbfff640 1 -- 172.21.15.66:0/143101041 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3984 (crc 0 0 0) 0x55cf4ed9fe20 con 0x55cf4ee79b50 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.629+0000 7f09cbfff640 10 client.6309 handle_command_reply: tid=0 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 client.6309 shutdown 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 msgr2=0x55cf4ee7c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee7c5e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f09dc059b40 tx=0x7f09dc089fe0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 shutdown_connections 2024-03-30T13:12:41.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55cf4ee79b50 0x55cf4ee7cb20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf4ee5db20 0x55cf4ee7c5e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf4ee5b950 0x55cf4ee7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09cbfff640 0 client.6309 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 --2- 172.21.15.66:0/143101041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cf4ee5b570 0x55cf4ee78dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 >> 172.21.15.66:0/143101041 conn(0x55cf4ee2ea50 msgr2=0x55cf4ee77170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 shutdown_connections 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 1 -- 172.21.15.66:0/143101041 wait complete. 2024-03-30T13:12:41.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f6047740 20 client.6309 trim_cache size 0 max 16384 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr_security.c", 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628402, 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.018413+0000", 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.534819+0000", 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:41.644 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:41.645 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1473, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.534819+0000", 2024-03-30T13:12:41.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:41.647 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:41.648 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1473, 2024-03-30T13:12:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.649 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.018413+0000" 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1473, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.018413+0000" 2024-03-30T13:12:41.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 178, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 34, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:41.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.061833+0000", 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 46, 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:41.652 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:41.653 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:41.654 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.655 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:41.656 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.658 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:41.660 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:41.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d806c530 msgr2=0x7f09d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f3455640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d806c530 0x7f09d806e9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0b4770 tx=0x7f09e4009190 comp rx=0 tx=0).stop 2024-03-30T13:12:41.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 msgr2=0x7f09ec0b84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.630+0000 7f09f3455640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0b84d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00eed0 tx=0x7f09dc00b480 comp rx=0 tx=0).stop 2024-03-30T13:12:41.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 shutdown_connections 2024-03-30T13:12:41.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f09d806c530 0x7f09d806e9f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09ec0ba090 0x7f09ec0b84d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09ec0b5dc0 0x7f09ec15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 --2- 172.21.15.66:0/1737149267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ec06c620 0x7f09ec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 >> 172.21.15.66:0/1737149267 conn(0x7f09ec0678f0 msgr2=0x7f09ec160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 shutdown_connections 2024-03-30T13:12:41.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.631+0000 7f09f3455640 1 -- 172.21.15.66:0/1737149267 wait complete. 2024-03-30T13:12:41.663 DEBUG:tasks.cephfs.test_uninlining:getting inode info #9 of 23:1099511628387 2024-03-30T13:12:41.663 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628387 2024-03-30T13:12:41.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628387 2024-03-30T13:12:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.771+0000 7f039be63640 1 Processor -- start 2024-03-30T13:12:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 -- start start 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f03940ba090 0x7f03940d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f039406c620 0x7f039406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f03940b66e0 con 0x7f039406c620 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f03940d2ff0 con 0x7f03940ba090 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.772+0000 7f039be63640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f03940d32d0 con 0x7f03940b5dc0 2024-03-30T13:12:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f039406c620 0x7f039406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f039406c620 0x7f039406cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35800/0 (socket says 172.21.15.66:35800) 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 -- 172.21.15.66:0/3375415677 learned_addr learned my addr 172.21.15.66:0/3375415677 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f03993fe640 1 --2- 172.21.15.66:0/3375415677 >> v2:172.21.15.117:3300/0 conn(0x7f03940ba090 0x7f03940d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f0399bff640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 -- 172.21.15.66:0/3375415677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 msgr2=0x7f03940b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 -- 172.21.15.66:0/3375415677 >> v2:172.21.15.117:3300/0 conn(0x7f03940ba090 msgr2=0x7f03940d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 --2- 172.21.15.66:0/3375415677 >> v2:172.21.15.117:3300/0 conn(0x7f03940ba090 0x7f03940d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f039a400640 1 -- 172.21.15.66:0/3375415677 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03940d35b0 con 0x7f039406c620 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f03993fe640 1 --2- 172.21.15.66:0/3375415677 >> v2:172.21.15.117:3300/0 conn(0x7f03940ba090 0x7f03940d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.773+0000 7f0399bff640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:41.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.774+0000 7f039a400640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f039406c620 0x7f039406cad0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f039406d680 tx=0x7f038c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92511ba5ccd50b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.774+0000 7f0398bfd640 1 -- 172.21.15.66:0/3375415677 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c035020 con 0x7f039406c620 2024-03-30T13:12:41.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.774+0000 7f0398bfd640 1 -- 172.21.15.66:0/3375415677 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c031870 con 0x7f039406c620 2024-03-30T13:12:41.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.774+0000 7f0398bfd640 1 -- 172.21.15.66:0/3375415677 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c031c40 con 0x7f039406c620 2024-03-30T13:12:41.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 -- 172.21.15.66:0/3375415677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f039406c620 msgr2=0x7f039406cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f039406c620 0x7f039406cad0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f039406d680 tx=0x7f038c02b910 comp rx=0 tx=0).stop 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 -- 172.21.15.66:0/3375415677 shutdown_connections 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f039406c620 0x7f039406cad0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 --2- 172.21.15.66:0/3375415677 >> v2:172.21.15.117:3300/0 conn(0x7f03940ba090 0x7f03940d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 --2- 172.21.15.66:0/3375415677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 -- 172.21.15.66:0/3375415677 >> 172.21.15.66:0/3375415677 conn(0x7f03940678f0 msgr2=0x7f03940d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 -- 172.21.15.66:0/3375415677 shutdown_connections 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.775+0000 7f039be63640 1 -- 172.21.15.66:0/3375415677 wait complete. 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.776+0000 7f039be63640 1 Processor -- start 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.776+0000 7f039be63640 1 -- start start 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039be63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039406c620 0x7f03941575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039be63640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039be63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 0x7f039414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039be63640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03940d4090 con 0x7f03940ba090 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039be63640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03940d3b70 con 0x7f039406c620 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039be63640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f039415e880 con 0x7f03940b5dc0 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 0x7f039414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f0399bff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039406c620 0x7f03941575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 0x7f039414c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35816/0 (socket says 172.21.15.66:35816) 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 -- 172.21.15.66:0/1057801672 learned_addr learned my addr 172.21.15.66:0/1057801672 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f03993fe640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 -- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 msgr2=0x7f03940cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 -- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039406c620 msgr2=0x7f03941575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.777+0000 7f039a400640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039406c620 0x7f03941575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f039a400640 1 -- 172.21.15.66:0/1057801672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03940ccf40 con 0x7f03940ba090 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f039a400640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 0x7f039414c6b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f038c02b950 tx=0x7f038c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f038a7fc640 1 -- 172.21.15.66:0/1057801672 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c040070 con 0x7f03940ba090 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f038a7fc640 1 -- 172.21.15.66:0/1057801672 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c030d00 con 0x7f03940ba090 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03940cd220 con 0x7f03940ba090 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f038a7fc640 1 -- 172.21.15.66:0/1057801672 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c035040 con 0x7f03940ba090 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.778+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03940cd7e0 con 0x7f03940ba090 2024-03-30T13:12:41.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.780+0000 7f038a7fc640 1 -- 172.21.15.66:0/1057801672 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f038c038020 con 0x7f03940ba090 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.780+0000 7f038a7fc640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c560 0x7f037806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.780+0000 7f038a7fc640 1 -- 172.21.15.66:0/1057801672 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f038c0ad640 con 0x7f03940ba090 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.780+0000 7f0399bff640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c560 0x7f037806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.781+0000 7f0399bff640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c560 0x7f037806ea20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0390003aa0 tx=0x7f0390009810 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.781+0000 7f039ea4f740 1 Processor -- start 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.781+0000 7f039ea4f740 1 -- start start 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e76a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e79a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 0x55c55e79a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039ea4f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c55e6d4590 con 0x55c55e76aa50 2024-03-30T13:12:41.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c55e672d30 con 0x55c55e797570 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c55e6d0540 con 0x55c55e797950 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 0x55c55e79a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 0x55c55e79a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35826/0 (socket says 172.21.15.66:35826) 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 -- 172.21.15.66:0/3554986730 learned_addr learned my addr 172.21.15.66:0/3554986730 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f03993fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e79a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f0399bff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e76a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 -- 172.21.15.66:0/3554986730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 msgr2=0x55c55e79a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e79a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 -- 172.21.15.66:0/3554986730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 msgr2=0x55c55e76a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e76a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.782+0000 7f039a400640 1 -- 172.21.15.66:0/3554986730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55e4b4920 con 0x55c55e76aa50 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.783+0000 7f039a400640 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 0x55c55e79a840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f038c0ab910 tx=0x7f038c0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a95bf9f305a1aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.783+0000 7f03777fe640 1 -- 172.21.15.66:0/3554986730 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c07b410 con 0x55c55e76aa50 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.783+0000 7f03777fe640 1 -- 172.21.15.66:0/3554986730 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c07f710 con 0x55c55e76aa50 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.783+0000 7f03777fe640 1 -- 172.21.15.66:0/3554986730 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c083f40 con 0x55c55e76aa50 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 -- 172.21.15.66:0/3554986730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 msgr2=0x55c55e79a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 0x55c55e79a840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f038c0ab910 tx=0x7f038c0094d0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 -- 172.21.15.66:0/3554986730 shutdown_connections 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e76aa50 0x55c55e79a840 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e79a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3554986730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e76a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 -- 172.21.15.66:0/3554986730 >> 172.21.15.66:0/3554986730 conn(0x55c55e76c780 msgr2=0x55c55e7b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 -- 172.21.15.66:0/3554986730 shutdown_connections 2024-03-30T13:12:41.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 -- 172.21.15.66:0/3554986730 wait complete. 2024-03-30T13:12:41.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 -- start start 2024-03-30T13:12:41.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.784+0000 7f039ea4f740 1 Processor -- start 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797570 0x55c55e7a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 0x55c55e79dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e79b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c55e672d30 con 0x55c55e797950 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c55e6709f0 con 0x55c55e79e000 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c55e6d4590 con 0x55c55e797570 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.785+0000 7f039ea4f740 10 client.0 _ll_register_callbacks cb 0x55c55e674c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 0x55c55e79dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 0x55c55e79dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35832/0 (socket says 172.21.15.66:35832) 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 -- 172.21.15.66:0/400801546 learned_addr learned my addr 172.21.15.66:0/400801546 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f0399bff640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797570 0x55c55e7a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f039a400640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e79b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 -- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797570 msgr2=0x55c55e7a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797570 0x55c55e7a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 -- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 msgr2=0x55c55e79b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e79b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 -- 172.21.15.66:0/400801546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55e3a8210 con 0x55c55e797950 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f03993fe640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 0x55c55e79dac0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0384004630 tx=0x7f038400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a544034193742e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f0374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f0374ff9640 1 -- 172.21.15.66:0/400801546 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03840156f0 con 0x55c55e797950 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f0374ff9640 1 -- 172.21.15.66:0/400801546 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f0384015890 con 0x55c55e797950 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038c07f710 con 0x55c55e797950 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f0374ff9640 1 -- 172.21.15.66:0/400801546 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0384015a90 con 0x55c55e797950 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.786+0000 7f039ea4f740 10 client.6321 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55c55e5150b0 con 0x55c55e797950 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f0374ff9640 1 -- 172.21.15.66:0/400801546 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c07f710 con 0x55c55e797950 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f0374ff9640 1 -- 172.21.15.66:0/400801546 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f0384028080 con 0x55c55e797950 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 10 client.6321 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 10 client.6321 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 20 client.6321 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e786ac0 0x55c55e7d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 4 client.6321 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c55e6e1ad0 con 0x55c55e786ac0 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f0399bff640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e786ac0 0x55c55e7d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f0399bff640 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e786ac0 0x55c55e7d2620 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce1ef84f6e3be81 server_cookie=385db406be365c24 in_seq=0 out_seq=0 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.787+0000 7f0374ff9640 10 client.6321 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f0374ff9640 1 -- 172.21.15.66:0/400801546 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c55e6e1ad0 con 0x55c55e786ac0 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f0374ff9640 10 client.6321 handle_command_reply: tid=0 2024-03-30T13:12:41.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 client.6321 shutdown 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 msgr2=0x55c55e79dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 0x55c55e79dac0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0384004630 tx=0x7f038400b8d0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 shutdown_connections 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e786ac0 0x55c55e7d2620 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e79b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f0374ff9640 0 client.6321 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797950 0x55c55e79dac0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 --2- 172.21.15.66:0/400801546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797570 0x55c55e7a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 >> 172.21.15.66:0/400801546 conn(0x55c55e76aa50 msgr2=0x55c55e7b8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 shutdown_connections 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 1 -- 172.21.15.66:0/400801546 wait complete. 2024-03-30T13:12:41.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.788+0000 7f039ea4f740 20 client.6321 trim_cache size 0 max 16384 2024-03-30T13:12:41.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.819+0000 7f039ea4f740 1 Processor -- start 2024-03-30T13:12:41.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.819+0000 7f039ea4f740 1 -- start start 2024-03-30T13:12:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e7d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039ea4f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03942012f0 con 0x55c55e79e000 2024-03-30T13:12:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03942002d0 con 0x55c55e797570 2024-03-30T13:12:41.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f03941ff620 con 0x55c55e797950 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35838/0 (socket says 172.21.15.66:35838) 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.820+0000 7f039a400640 1 -- 172.21.15.66:0/1608702073 learned_addr learned my addr 172.21.15.66:0/1608702073 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f03993fe640 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f0399bff640 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e7d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f039a400640 1 -- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 msgr2=0x55c55e7c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f039a400640 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f039a400640 1 -- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 msgr2=0x55c55e7d7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f039a400640 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e7d7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f039a400640 1 -- 172.21.15.66:0/1608702073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55e3812d0 con 0x55c55e79e000 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f039a400640 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f038c07b110 tx=0x7f038c07f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc36539ab4a6030 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f0376ffd640 1 -- 172.21.15.66:0/1608702073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c097070 con 0x55c55e79e000 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f0376ffd640 1 -- 172.21.15.66:0/1608702073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c083f40 con 0x55c55e79e000 2024-03-30T13:12:41.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.821+0000 7f0376ffd640 1 -- 172.21.15.66:0/1608702073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c07b410 con 0x55c55e79e000 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 -- 172.21.15.66:0/1608702073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 msgr2=0x55c55e7c3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f038c07b110 tx=0x7f038c07f000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 -- 172.21.15.66:0/1608702073 shutdown_connections 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 --2- 172.21.15.66:0/1608702073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e797570 0x55c55e7d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 -- 172.21.15.66:0/1608702073 >> 172.21.15.66:0/1608702073 conn(0x55c55e76aa50 msgr2=0x55c55e7a07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 -- 172.21.15.66:0/1608702073 shutdown_connections 2024-03-30T13:12:41.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.822+0000 7f039ea4f740 1 -- 172.21.15.66:0/1608702073 wait complete. 2024-03-30T13:12:41.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.823+0000 7f039ea4f740 1 -- start start 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.823+0000 7f039ea4f740 1 Processor -- start 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 0x55c55e819de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03941ffc40 con 0x55c55e797570 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c55e7b90a0 con 0x55c55e79e000 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55c55e834d30 con 0x55c55e797950 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039ea4f740 10 client.0 _ll_register_callbacks cb 0x7f03840042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f0399bff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 0x55c55e819de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f0399bff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 0x55c55e819de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35840/0 (socket says 172.21.15.66:35840) 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f0399bff640 1 -- 172.21.15.66:0/3546249124 learned_addr learned my addr 172.21.15.66:0/3546249124 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f039a400640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.824+0000 7f03993fe640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0399bff640 1 -- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 msgr2=0x55c55e7c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0399bff640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0399bff640 1 -- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 msgr2=0x55c55e7c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0399bff640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0399bff640 1 -- 172.21.15.66:0/3546249124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55e834120 con 0x55c55e797570 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0399bff640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 0x55c55e819de0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55c55e79fe70 tx=0x7f0390004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c257a88508aaa13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0374ff9640 1 -- 172.21.15.66:0/3546249124 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390031060 con 0x55c55e797570 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0374ff9640 1 -- 172.21.15.66:0/3546249124 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f039002b2d0 con 0x55c55e797570 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038c083f40 con 0x55c55e797570 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0374ff9640 1 -- 172.21.15.66:0/3546249124 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f039001c040 con 0x55c55e797570 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f039ea4f740 10 client.6327 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55c55e7b4790 con 0x55c55e797570 2024-03-30T13:12:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0374ff9640 1 -- 172.21.15.66:0/3546249124 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c083f40 con 0x55c55e797570 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.825+0000 7f0374ff9640 1 -- 172.21.15.66:0/3546249124 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f039003a080 con 0x55c55e797570 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 10 client.6327 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 10 client.6327 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 20 client.6327 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e7c44a0 0x55c55e7693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 4 client.6327 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628387}] 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628387}) -- 0x55c55e7a22e0 con 0x55c55e7c44a0 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f03993fe640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e7c44a0 0x55c55e7693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f0374ff9640 10 client.6327 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f03993fe640 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e7c44a0 0x55c55e7693a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f845675d486aea server_cookie=58b223e04bdba1f9 in_seq=0 out_seq=0 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f0374ff9640 1 -- 172.21.15.66:0/3546249124 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3972 (crc 0 0 0) 0x55c55e7a22e0 con 0x55c55e7c44a0 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f0374ff9640 10 client.6327 handle_command_reply: tid=0 2024-03-30T13:12:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 1 client.6327 shutdown 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.826+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 msgr2=0x55c55e819de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 0x55c55e819de0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55c55e79fe70 tx=0x7f0390004ab0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 shutdown_connections 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55c55e7c44a0 0x55c55e7693a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c55e79e000 0x55c55e7c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c55e797950 0x55c55e7c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f0374ff9640 0 client.6327 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 --2- 172.21.15.66:0/3546249124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c55e797570 0x55c55e819de0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 >> 172.21.15.66:0/3546249124 conn(0x55c55e76aa50 msgr2=0x55c55e79a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 shutdown_connections 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 1 -- 172.21.15.66:0/3546249124 wait complete. 2024-03-30T13:12:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039ea4f740 20 client.6327 trim_cache size 0 max 16384 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/Makefile", 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628387, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.047455+0000", 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.525819+0000", 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:41.837 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 378, 2024-03-30T13:12:41.838 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.525819+0000", 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:41.839 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 378, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.047455+0000" 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.840 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 378, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.047455+0000" 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 150, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 4, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:41.841 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.091000+0000", 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 62, 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:41.844 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.846 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:41.847 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:41.848 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:41.849 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c560 msgr2=0x7f037806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039be63640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c560 0x7f037806ea20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0390003aa0 tx=0x7f0390009810 comp rx=0 tx=0).stop 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 msgr2=0x7f039414c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039be63640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 0x7f039414c6b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f038c02b950 tx=0x7f038c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.827+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 shutdown_connections 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f037806c560 0x7f037806ea20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03940ba090 0x7f039414c6b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03940b5dc0 0x7f03940cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 --2- 172.21.15.66:0/1057801672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039406c620 0x7f03941575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 >> 172.21.15.66:0/1057801672 conn(0x7f03940678f0 msgr2=0x7f0394160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 shutdown_connections 2024-03-30T13:12:41.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.828+0000 7f039be63640 1 -- 172.21.15.66:0/1057801672 wait complete. 2024-03-30T13:12:41.852 DEBUG:tasks.cephfs.test_uninlining:getting inode info #10 of 23:1099511628399 2024-03-30T13:12:41.852 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628399 2024-03-30T13:12:41.852 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628399 2024-03-30T13:12:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.959+0000 7f151da3f640 1 Processor -- start 2024-03-30T13:12:41.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.959+0000 7f151da3f640 1 -- start start 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f151da3f640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1518144cb0 0x7f15180b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f151da3f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 0x7f15180ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f151da3f640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15180b35e0 con 0x7f151815f800 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f151da3f640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f15180b3e80 con 0x7f1518144cb0 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f151da3f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f15180b38c0 con 0x7f1518072d50 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f15167fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f151815f800 0x7f1518072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37736/0 (socket says 172.21.15.66:37736) 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f15167fc640 1 -- 172.21.15.66:0/1470928173 learned_addr learned my addr 172.21.15.66:0/1470928173 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f15177fe640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 0x7f15180ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1470928173 >> v2:172.21.15.117:3300/0 conn(0x7f1518144cb0 0x7f15180b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4275154129 0 0) 0x7f15180b35e0 con 0x7f151815f800 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1500003520 con 0x7f151815f800 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3232226088 0 0) 0x7f1500003520 con 0x7f151815f800 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 msgr2=0x7f15180ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 0x7f15180ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 >> v2:172.21.15.117:3300/0 conn(0x7f1518144cb0 msgr2=0x7f15180b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 --2- 172.21.15.66:0/1470928173 >> v2:172.21.15.117:3300/0 conn(0x7f1518144cb0 0x7f15180b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180b3ba0 con 0x7f151815f800 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f1508001670 con 0x7f151815f800 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 >> v1:172.21.15.66:6789/0 conn(0x7f151815f800 legacy=0x7f1518072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.960+0000 7f1515ffb640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15000039b0 0x7f1500003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1515ffb640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 0x7f1500004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1515ffb640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1500004d40 0x7f15000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15180b3e80 con 0x7f15000039b0 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15180b38c0 con 0x7f1500004d40 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f150000b9c0 con 0x7f1518072d50 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15000039b0 0x7f1500003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f15177fe640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1500004d40 0x7f15000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f15167fc640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 0x7f1500004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1516ffd640 1 -- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 msgr2=0x7f1500004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1518072d50 0x7f1500004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1516ffd640 1 -- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1500004d40 msgr2=0x7f15000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1500004d40 0x7f15000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.961+0000 7f1516ffd640 1 -- 172.21.15.66:0/1470928173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1500003520 con 0x7f15000039b0 2024-03-30T13:12:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15000039b0 0x7f1500003dd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f150c009af0 tx=0x7f1500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397257781dd0b08f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c039070 con 0x7f15000039b0 2024-03-30T13:12:41.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c02c710 con 0x7f15000039b0 2024-03-30T13:12:41.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f1515ffb640 1 -- 172.21.15.66:0/1470928173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c009690 con 0x7f15000039b0 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f151da3f640 1 -- 172.21.15.66:0/1470928173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15000039b0 msgr2=0x7f1500003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f151da3f640 1 --2- 172.21.15.66:0/1470928173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15000039b0 0x7f1500003dd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f150c009af0 tx=0x7f1500005830 comp rx=0 tx=0).stop 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f15177fe640 1 -- 172.21.15.66:0/1470928173 reap_dead start 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f151da3f640 1 -- 172.21.15.66:0/1470928173 shutdown_connections 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.962+0000 7f151da3f640 1 -- 172.21.15.66:0/1470928173 >> 172.21.15.66:0/1470928173 conn(0x7f15180b5b00 msgr2=0x7f151806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.963+0000 7f151da3f640 1 -- 172.21.15.66:0/1470928173 shutdown_connections 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.963+0000 7f151da3f640 1 -- 172.21.15.66:0/1470928173 wait complete. 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.964+0000 7f151da3f640 1 Processor -- start 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.964+0000 7f151da3f640 1 -- start start 2024-03-30T13:12:41.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f151da3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000039b0 0x7f1518143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f151da3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 0x7f1518157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f151da3f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b0e20 0x7f15180b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f151da3f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15180ce050 con 0x7f1518155e10 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f151da3f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f151815fc40 con 0x7f15000039b0 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f151da3f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f15180ce1d0 con 0x7f15180b0e20 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 0x7f1518157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000039b0 0x7f1518143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 0x7f1518157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35872/0 (socket says 172.21.15.66:35872) 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 -- 172.21.15.66:0/3809116371 learned_addr learned my addr 172.21.15.66:0/3809116371 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15177fe640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b0e20 0x7f15180b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 -- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b0e20 msgr2=0x7f15180b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b0e20 0x7f15180b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 -- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000039b0 msgr2=0x7f1518143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000039b0 0x7f1518143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 -- 172.21.15.66:0/3809116371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1518142750 con 0x7f1518155e10 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f15167fc640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 0x7f1518157e50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1508007340 tx=0x7f1508007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.965+0000 7f14fffff640 1 -- 172.21.15.66:0/3809116371 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150801a070 con 0x7f1518155e10 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.966+0000 7f14fffff640 1 -- 172.21.15.66:0/3809116371 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150800f8c0 con 0x7f1518155e10 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.966+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1518141cb0 con 0x7f1518155e10 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.966+0000 7f14fffff640 1 -- 172.21.15.66:0/3809116371 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150800fc90 con 0x7f1518155e10 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.966+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1518141f50 con 0x7f1518155e10 2024-03-30T13:12:41.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.967+0000 7f14fffff640 1 -- 172.21.15.66:0/3809116371 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f150801d070 con 0x7f1518155e10 2024-03-30T13:12:41.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.967+0000 7f14fffff640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f150406c530 0x7f150406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.968+0000 7f14fffff640 1 -- 172.21.15.66:0/3809116371 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f150808b100 con 0x7f1518155e10 2024-03-30T13:12:41.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.968+0000 7f1516ffd640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f150406c530 0x7f150406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.968+0000 7f1516ffd640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f150406c530 0x7f150406e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f150c036000 tx=0x7f150c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.968+0000 7f1520621740 1 Processor -- start 2024-03-30T13:12:41.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 -- start start 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 0x55fca664f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c950 0x55fca667f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca664fa50 0x55fca667f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fca65b9590 con 0x55fca667c570 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fca6557d30 con 0x55fca667c950 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.969+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fca65b5540 con 0x55fca664fa50 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 0x55fca664f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 0x55fca664f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35888/0 (socket says 172.21.15.66:35888) 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 -- 172.21.15.66:0/157300498 learned_addr learned my addr 172.21.15.66:0/157300498 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f15177fe640 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca664fa50 0x55fca667f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f15167fc640 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c950 0x55fca667f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 -- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca664fa50 msgr2=0x55fca667f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca664fa50 0x55fca667f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 -- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c950 msgr2=0x55fca667f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c950 0x55fca667f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 -- 172.21.15.66:0/157300498 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca6399920 con 0x55fca667c570 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f1516ffd640 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 0x55fca664f510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f150c02c380 tx=0x7f150c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8e5741c3cfd239 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f14fd7fa640 1 -- 172.21.15.66:0/157300498 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c043040 con 0x55fca667c570 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f14fd7fa640 1 -- 172.21.15.66:0/157300498 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c03a040 con 0x55fca667c570 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.970+0000 7f14fd7fa640 1 -- 172.21.15.66:0/157300498 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c039050 con 0x55fca667c570 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 -- 172.21.15.66:0/157300498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 msgr2=0x55fca664f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 0x55fca664f510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f150c02c380 tx=0x7f150c02e000 comp rx=0 tx=0).stop 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 -- 172.21.15.66:0/157300498 shutdown_connections 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca664fa50 0x55fca667f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c950 0x55fca667f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 --2- 172.21.15.66:0/157300498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c570 0x55fca664f510 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 -- 172.21.15.66:0/157300498 >> 172.21.15.66:0/157300498 conn(0x55fca6651780 msgr2=0x55fca669a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 -- 172.21.15.66:0/157300498 shutdown_connections 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.971+0000 7f1520621740 1 -- 172.21.15.66:0/157300498 wait complete. 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.972+0000 7f1520621740 1 -- start start 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.972+0000 7f1520621740 1 Processor -- start 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.972+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c570 0x55fca66976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca666eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca666f720 0x55fca666f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fca6653e20 con 0x55fca667c950 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fca65559f0 con 0x55fca666f720 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fca6656660 con 0x55fca667c570 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 10 client.0 _ll_register_callbacks cb 0x55fca6559c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca666eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca666eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35904/0 (socket says 172.21.15.66:35904) 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 -- 172.21.15.66:0/1395563715 learned_addr learned my addr 172.21.15.66:0/1395563715 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c570 0x55fca66976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15177fe640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca666f720 0x55fca666f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 -- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c570 msgr2=0x55fca66976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c570 0x55fca66976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 -- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca666f720 msgr2=0x55fca666f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca666f720 0x55fca666f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 -- 172.21.15.66:0/1395563715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca628d210 con 0x55fca667c950 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f15167fc640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca666eb90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f15080553c0 tx=0x7f150805d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6483824e5d2e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f14f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f14f67fc640 1 -- 172.21.15.66:0/1395563715 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508087060 con 0x55fca667c950 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f14f67fc640 1 -- 172.21.15.66:0/1395563715 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f1508057260 con 0x55fca667c950 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c03a040 con 0x55fca667c950 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f14f67fc640 1 -- 172.21.15.66:0/1395563715 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1508076050 con 0x55fca667c950 2024-03-30T13:12:41.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.973+0000 7f1520621740 10 client.6339 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55fca63fa0b0 con 0x55fca667c950 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f14f67fc640 1 -- 172.21.15.66:0/1395563715 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c03a040 con 0x55fca667c950 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f14f67fc640 1 -- 172.21.15.66:0/1395563715 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f15080583b0 con 0x55fca667c950 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 10 client.6339 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 10 client.6339 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 20 client.6339 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca6669c50 0x55fca666a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 4 client.6339 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca65c6ad0 con 0x55fca6669c50 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca6669c50 0x55fca666a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f1516ffd640 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca6669c50 0x55fca666a030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c59954d094bedfd server_cookie=790afa5a9b4a7210 in_seq=0 out_seq=0 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.974+0000 7f14f67fc640 10 client.6339 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f14f67fc640 1 -- 172.21.15.66:0/1395563715 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55fca65c6ad0 con 0x55fca6669c50 2024-03-30T13:12:41.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f14f67fc640 10 client.6339 handle_command_reply: tid=0 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 client.6339 shutdown 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 msgr2=0x55fca666eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca666eb90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f15080553c0 tx=0x7f150805d2e0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 shutdown_connections 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca6669c50 0x55fca666a030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca666f720 0x55fca666f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca666eb90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f14f67fc640 0 client.6339 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 --2- 172.21.15.66:0/1395563715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c570 0x55fca66976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.975+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 >> 172.21.15.66:0/1395563715 conn(0x55fca664fa50 msgr2=0x55fca6653420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.976+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 shutdown_connections 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.976+0000 7f1520621740 1 -- 172.21.15.66:0/1395563715 wait complete. 2024-03-30T13:12:41.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.976+0000 7f1520621740 20 client.6339 trim_cache size 0 max 16384 2024-03-30T13:12:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.990+0000 7f1520621740 1 Processor -- start 2024-03-30T13:12:41.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.990+0000 7f1520621740 1 -- start start 2024-03-30T13:12:41.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca666f720 0x55fca667f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca666a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1520621740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca6699b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1520621740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15181ff6a0 con 0x55fca667c950 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15181fe680 con 0x55fca667c570 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f15181fda10 con 0x55fca666f720 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca6699b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca6699b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35906/0 (socket says 172.21.15.66:35906) 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 -- 172.21.15.66:0/2586888836 learned_addr learned my addr 172.21.15.66:0/2586888836 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15167fc640 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca666a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f1516ffd640 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca666f720 0x55fca667f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 -- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca666f720 msgr2=0x55fca667f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca666f720 0x55fca667f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 -- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 msgr2=0x55fca666a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca666a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 -- 172.21.15.66:0/2586888836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca62662d0 con 0x55fca667c950 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.991+0000 7f15177fe640 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca6699b90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f151806c4e0 tx=0x7f1510009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8d97d7e1a438fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f14fcff9640 1 -- 172.21.15.66:0/2586888836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15100157e0 con 0x55fca667c950 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f14fcff9640 1 -- 172.21.15.66:0/2586888836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1510015980 con 0x55fca667c950 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f14fcff9640 1 -- 172.21.15.66:0/2586888836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510015d50 con 0x55fca667c950 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 -- 172.21.15.66:0/2586888836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 msgr2=0x55fca6699b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca6699b90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f151806c4e0 tx=0x7f1510009750 comp rx=0 tx=0).stop 2024-03-30T13:12:41.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 -- 172.21.15.66:0/2586888836 shutdown_connections 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca667c950 0x55fca6699b90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca666a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 --2- 172.21.15.66:0/2586888836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca666f720 0x55fca667f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 -- 172.21.15.66:0/2586888836 >> 172.21.15.66:0/2586888836 conn(0x55fca664fa50 msgr2=0x55fca664a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 -- 172.21.15.66:0/2586888836 shutdown_connections 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.992+0000 7f1520621740 1 -- 172.21.15.66:0/2586888836 wait complete. 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.993+0000 7f1520621740 1 -- start start 2024-03-30T13:12:41.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.993+0000 7f1520621740 1 Processor -- start 2024-03-30T13:12:42.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 0x55fca6682c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca6718650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c950 0x55fca6718b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15181fe030 con 0x55fca666f720 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fca6719410 con 0x55fca667c570 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fca6668fe0 con 0x55fca667c950 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1520621740 10 client.0 _ll_register_callbacks cb 0x7f150808b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 0x55fca6682c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 0x55fca6682c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35908/0 (socket says 172.21.15.66:35908) 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 -- 172.21.15.66:0/2355480171 learned_addr learned my addr 172.21.15.66:0/2355480171 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f15177fe640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c950 0x55fca6718b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f15167fc640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca6718650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 -- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c950 msgr2=0x55fca6718b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c950 0x55fca6718b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 -- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 msgr2=0x55fca6718650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca6718650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 -- 172.21.15.66:0/2355480171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca66add70 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f1516ffd640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 0x55fca6682c90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f150c006f80 tx=0x7f150c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef26c0d575e4c5a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f14f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.994+0000 7f14f67fc640 1 -- 172.21.15.66:0/2355480171 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c043040 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f14f67fc640 1 -- 172.21.15.66:0/2355480171 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6218+0+0 (secure 0 0 0) 0x7f150c005c00 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1510015980 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f14f67fc640 1 -- 172.21.15.66:0/2355480171 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f150c062050 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 10 client.6345 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55fca6719150 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f14f67fc640 1 -- 172.21.15.66:0/2355480171 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1510015980 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f14f67fc640 1 -- 172.21.15.66:0/2355480171 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f150c065020 con 0x55fca666f720 2024-03-30T13:12:42.005 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 10 client.6345 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 10 client.6345 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 20 client.6345 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca66fdd40 0x55fca6699b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 4 client.6345 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628399}] 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628399}) -- 0x55fca65c0e20 con 0x55fca66fdd40 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.995+0000 7f15167fc640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca66fdd40 0x55fca6699b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f15167fc640 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca66fdd40 0x55fca6699b90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4afed22a9335d34f server_cookie=ae75a1bc1fc6b47c in_seq=0 out_seq=0 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f14f67fc640 10 client.6345 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f14f67fc640 1 -- 172.21.15.66:0/2355480171 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3974 (crc 0 0 0) 0x55fca65c0e20 con 0x55fca66fdd40 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f14f67fc640 10 client.6345 handle_command_reply: tid=0 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 client.6345 shutdown 2024-03-30T13:12:42.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 msgr2=0x55fca6682c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 0x55fca6682c90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f150c006f80 tx=0x7f150c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 shutdown_connections 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fca66fdd40 0x55fca6699b90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fca667c950 0x55fca6718b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fca667c570 0x55fca6718650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f14f67fc640 0 client.6345 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 --2- 172.21.15.66:0/2355480171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fca666f720 0x55fca6682c90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 >> 172.21.15.66:0/2355480171 conn(0x55fca664fa50 msgr2=0x55fca6653210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 shutdown_connections 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 1 -- 172.21.15.66:0/2355480171 wait complete. 2024-03-30T13:12:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.996+0000 7f1520621740 20 client.6345 trim_cache size 0 max 16384 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/symlink.c", 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628399, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.050272+0000", 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.532819+0000", 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:42.008 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 916, 2024-03-30T13:12:42.009 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.532819+0000", 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:42.010 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 916, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.050272+0000" 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.011 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 916, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.050272+0000" 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 190, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 28, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.093255+0000", 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 64, 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:42.013 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:42.014 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:42.015 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.016 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.017 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.018 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:42.019 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.020 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.020 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:42.020 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:42.020 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:42.020 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:42.020 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f150406c530 msgr2=0x7f150406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f150406c530 0x7f150406e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f150c036000 tx=0x7f150c030000 comp rx=0 tx=0).stop 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 msgr2=0x7f1518157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 0x7f1518157e50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1508007340 tx=0x7f1508007800 comp rx=0 tx=0).stop 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 shutdown_connections 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f150406c530 0x7f150406e9f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b0e20 0x7f15180b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1518155e10 0x7f1518157e50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 --2- 172.21.15.66:0/3809116371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15000039b0 0x7f1518143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 >> 172.21.15.66:0/3809116371 conn(0x7f15180b5b00 msgr2=0x7f151813e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 shutdown_connections 2024-03-30T13:12:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:41.997+0000 7f151da3f640 1 -- 172.21.15.66:0/3809116371 wait complete. 2024-03-30T13:12:42.023 DEBUG:tasks.cephfs.test_uninlining:getting inode info #11 of 23:1099511628389 2024-03-30T13:12:42.023 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628389 2024-03-30T13:12:42.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628389 2024-03-30T13:12:42.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.136+0000 7f4eb4747640 1 Processor -- start 2024-03-30T13:12:42.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.136+0000 7f4eb4747640 1 -- start start 2024-03-30T13:12:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.137+0000 7f4eb4747640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4eac144cb0 0x7f4eac0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.137+0000 7f4eb4747640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac072d50 0x7f4eac0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.137+0000 7f4eb4747640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eac0b35e0 con 0x7f4eac15f800 2024-03-30T13:12:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb4747640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4eac0b3e80 con 0x7f4eac144cb0 2024-03-30T13:12:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb4747640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac0b38c0 con 0x7f4eac072d50 2024-03-30T13:12:42.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb24e3640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4eac144cb0 0x7f4eac0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb1ce2640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f4eac15f800 0x7f4eac072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37738/0 (socket says 172.21.15.66:37738) 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb1ce2640 1 -- 172.21.15.66:0/216971917 learned_addr learned my addr 172.21.15.66:0/216971917 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/216971917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac072d50 0x7f4eac0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb24e3640 1 -- 172.21.15.66:0/216971917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac072d50 msgr2=0x7f4eac0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/216971917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac072d50 0x7f4eac0ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb24e3640 1 -- 172.21.15.66:0/216971917 >> v1:172.21.15.66:6789/0 conn(0x7f4eac15f800 legacy=0x7f4eac072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb14e1640 1 -- 172.21.15.66:0/216971917 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559565851 0 0) 0x7f4eac0b35e0 con 0x7f4eac15f800 2024-03-30T13:12:42.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.138+0000 7f4eb24e3640 1 -- 172.21.15.66:0/216971917 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac0b3ba0 con 0x7f4eac144cb0 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.139+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/216971917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac144cb0 0x7f4eac0b2f20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80097a0 tx=0x7f4ea802dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70d9b1ec0522adc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.139+0000 7f4eb14e1640 1 -- 172.21.15.66:0/216971917 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea802f8a0 con 0x7f4eac144cb0 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.139+0000 7f4eb14e1640 1 -- 172.21.15.66:0/216971917 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea802fa40 con 0x7f4eac144cb0 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.139+0000 7f4eb14e1640 1 -- 172.21.15.66:0/216971917 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea802fe10 con 0x7f4eac144cb0 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.139+0000 7f4eb4747640 1 -- 172.21.15.66:0/216971917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac144cb0 msgr2=0x7f4eac0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.139+0000 7f4eb4747640 1 --2- 172.21.15.66:0/216971917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac144cb0 0x7f4eac0b2f20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80097a0 tx=0x7f4ea802dae0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.140+0000 7f4eb4747640 1 -- 172.21.15.66:0/216971917 shutdown_connections 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.140+0000 7f4eb4747640 1 --2- 172.21.15.66:0/216971917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac072d50 0x7f4eac0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.140+0000 7f4eb4747640 1 --2- 172.21.15.66:0/216971917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac144cb0 0x7f4eac0b2f20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.140+0000 7f4eb4747640 1 -- 172.21.15.66:0/216971917 >> 172.21.15.66:0/216971917 conn(0x7f4eac0b5b00 msgr2=0x7f4eac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.140+0000 7f4eb4747640 1 -- 172.21.15.66:0/216971917 shutdown_connections 2024-03-30T13:12:42.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.140+0000 7f4eb4747640 1 -- 172.21.15.66:0/216971917 wait complete. 2024-03-30T13:12:42.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.141+0000 7f4eb4747640 1 Processor -- start 2024-03-30T13:12:42.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.141+0000 7f4eb4747640 1 -- start start 2024-03-30T13:12:42.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.141+0000 7f4eb4747640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 0x7f4eac134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.141+0000 7f4eb4747640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eac144cb0 0x7f4eac135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb4747640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac15f800 0x7f4eac13f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb4747640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac0ce590 con 0x7f4eac072d50 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb4747640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac15fc40 con 0x7f4eac144cb0 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb4747640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac0ce710 con 0x7f4eac15f800 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 0x7f4eac134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 0x7f4eac134b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35926/0 (socket says 172.21.15.66:35926) 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 -- 172.21.15.66:0/3642144705 learned_addr learned my addr 172.21.15.66:0/3642144705 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac15f800 0x7f4eac13f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eac144cb0 0x7f4eac135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 -- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac15f800 msgr2=0x7f4eac13f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac15f800 0x7f4eac13f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 -- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eac144cb0 msgr2=0x7f4eac135080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eac144cb0 0x7f4eac135080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 -- 172.21.15.66:0/3642144705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac133ba0 con 0x7f4eac072d50 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac15f800 0x7f4eac13f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 0x7f4eac134b40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098c0 tx=0x7f4ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4ea2ffd640 1 -- 172.21.15.66:0/3642144705 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8047070 con 0x7f4eac072d50 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4ea2ffd640 1 -- 172.21.15.66:0/3642144705 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8030b70 con 0x7f4eac072d50 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac133e80 con 0x7f4eac072d50 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4ea2ffd640 1 -- 172.21.15.66:0/3642144705 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8042410 con 0x7f4eac072d50 2024-03-30T13:12:42.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.142+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac134440 con 0x7f4eac072d50 2024-03-30T13:12:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.144+0000 7f4ea2ffd640 1 -- 172.21.15.66:0/3642144705 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4ea8039400 con 0x7f4eac072d50 2024-03-30T13:12:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.144+0000 7f4ea2ffd640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4e9006c530 0x7f4e9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.144+0000 7f4ea2ffd640 1 -- 172.21.15.66:0/3642144705 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f4ea80ad280 con 0x7f4eac072d50 2024-03-30T13:12:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.144+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4e9006c530 0x7f4e9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.145+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4e9006c530 0x7f4e9006e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4eac135c40 tx=0x7f4e9c007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.145+0000 7f4eb740d740 1 Processor -- start 2024-03-30T13:12:42.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.145+0000 7f4eb740d740 1 -- start start 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af34510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af64300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af34a50 0x55739af64840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb740d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55739ae9e590 con 0x55739af61950 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55739ae3cd30 con 0x55739af34a50 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55739ae9a540 con 0x55739af61570 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb1ce2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af64300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb1ce2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af64300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35932/0 (socket says 172.21.15.66:35932) 2024-03-30T13:12:42.154 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb1ce2640 1 -- 172.21.15.66:0/601026762 learned_addr learned my addr 172.21.15.66:0/601026762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af34a50 0x55739af64840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.146+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af34510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb1ce2640 1 -- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 msgr2=0x55739af34510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af34510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb1ce2640 1 -- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af34a50 msgr2=0x55739af64840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af34a50 0x55739af64840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb1ce2640 1 -- 172.21.15.66:0/601026762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739ac7e920 con 0x55739af61950 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af64300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005d80 tx=0x7f4e9c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07c20d388b079db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4e8ffff640 1 -- 172.21.15.66:0/601026762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c015110 con 0x55739af61950 2024-03-30T13:12:42.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4e8ffff640 1 -- 172.21.15.66:0/601026762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0159c0 con 0x55739af61950 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4e8ffff640 1 -- 172.21.15.66:0/601026762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c015d70 con 0x55739af61950 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb740d740 1 -- 172.21.15.66:0/601026762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 msgr2=0x55739af64300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.147+0000 7f4eb740d740 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af64300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005d80 tx=0x7f4e9c012000 comp rx=0 tx=0).stop 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 -- 172.21.15.66:0/601026762 shutdown_connections 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af34a50 0x55739af64840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af64300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 --2- 172.21.15.66:0/601026762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af34510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 -- 172.21.15.66:0/601026762 >> 172.21.15.66:0/601026762 conn(0x55739af36780 msgr2=0x55739af7f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 -- 172.21.15.66:0/601026762 shutdown_connections 2024-03-30T13:12:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 -- 172.21.15.66:0/601026762 wait complete. 2024-03-30T13:12:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 -- start start 2024-03-30T13:12:42.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.148+0000 7f4eb740d740 1 Processor -- start 2024-03-30T13:12:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af5a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61950 0x55739af59cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55739af38e20 con 0x55739af5ae20 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55739ae3a9f0 con 0x55739af61950 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55739af3b660 con 0x55739af61570 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb740d740 10 client.0 _ll_register_callbacks cb 0x55739ae3ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb2ce4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb2ce4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35944/0 (socket says 172.21.15.66:35944) 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/1321165485 learned_addr learned my addr 172.21.15.66:0/1321165485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61950 0x55739af59cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.149+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af5a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 msgr2=0x55739af5a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af5a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61950 msgr2=0x55739af59cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61950 0x55739af59cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/1321165485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739ab72210 con 0x55739af5ae20 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c910 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4007e10 tx=0x7f4ea4012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aea737a6100e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4e8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/1321165485 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea401c020 con 0x55739af5ae20 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/1321165485 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f4ea4018670 con 0x55739af5ae20 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c0159c0 con 0x55739af5ae20 2024-03-30T13:12:42.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/1321165485 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4ea4026050 con 0x55739af5ae20 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb740d740 10 client.6360 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55739acdf0b0 con 0x55739af5ae20 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.150+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/1321165485 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0159c0 con 0x55739af5ae20 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/1321165485 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f4ea4029020 con 0x55739af5ae20 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb740d740 10 client.6360 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb740d740 10 client.6360 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb740d740 20 client.6360 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb740d740 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af66630 0x55739af66a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb740d740 4 client.6360 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55739aeabad0 con 0x55739af66630 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af66630 0x55739af66a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af66630 0x55739af66a40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a477ec88eab59 server_cookie=ce75accd6574802b in_seq=0 out_seq=0 2024-03-30T13:12:42.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.151+0000 7f4e8d7fa640 10 client.6360 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/1321165485 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55739aeabad0 con 0x55739af66630 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4e8d7fa640 10 client.6360 handle_command_reply: tid=0 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 client.6360 shutdown 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 msgr2=0x55739af5c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c910 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4007e10 tx=0x7f4ea4012610 comp rx=0 tx=0).stop 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 shutdown_connections 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af66630 0x55739af66a40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c910 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61950 0x55739af59cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 --2- 172.21.15.66:0/1321165485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af5a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4e8d7fa640 0 client.6360 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 >> 172.21.15.66:0/1321165485 conn(0x55739af34a50 msgr2=0x55739af38420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 shutdown_connections 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 1 -- 172.21.15.66:0/1321165485 wait complete. 2024-03-30T13:12:42.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.152+0000 7f4eb740d740 20 client.6360 trim_cache size 0 max 16384 2024-03-30T13:12:42.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.166+0000 7f4eb740d740 1 Processor -- start 2024-03-30T13:12:42.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 -- start start 2024-03-30T13:12:42.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61570 0x55739af33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61950 0x55739af5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac201160 con 0x55739af5ae20 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac2001c0 con 0x55739af61570 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac0cc3a0 con 0x55739af61950 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb24e3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb24e3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35950/0 (socket says 172.21.15.66:35950) 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.167+0000 7f4eb24e3640 1 -- 172.21.15.66:0/222395972 learned_addr learned my addr 172.21.15.66:0/222395972 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb24e3640 1 -- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61950 msgr2=0x55739af5f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61950 0x55739af5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb24e3640 1 -- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61570 msgr2=0x55739af33720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61570 0x55739af33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61570 0x55739af33720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb24e3640 1 -- 172.21.15.66:0/222395972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739ab4b2d0 con 0x55739af5ae20 2024-03-30T13:12:42.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61570 0x55739af33720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c860 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8006ce0 tx=0x7f4ea8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aa7161454b0a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/222395972 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8083bd0 con 0x55739af5ae20 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/222395972 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8079090 con 0x55739af5ae20 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4e8f7fe640 1 -- 172.21.15.66:0/222395972 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80792f0 con 0x55739af5ae20 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 -- 172.21.15.66:0/222395972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 msgr2=0x55739af5c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c860 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8006ce0 tx=0x7f4ea8009520 comp rx=0 tx=0).stop 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 -- 172.21.15.66:0/222395972 shutdown_connections 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61950 0x55739af5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af61570 0x55739af33720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 --2- 172.21.15.66:0/222395972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af5ae20 0x55739af5c860 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 -- 172.21.15.66:0/222395972 >> 172.21.15.66:0/222395972 conn(0x55739af34a50 msgr2=0x55739af59130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.168+0000 7f4eb740d740 1 -- 172.21.15.66:0/222395972 shutdown_connections 2024-03-30T13:12:42.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.169+0000 7f4eb740d740 1 -- 172.21.15.66:0/222395972 wait complete. 2024-03-30T13:12:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.169+0000 7f4eb740d740 1 -- start start 2024-03-30T13:12:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.169+0000 7f4eb740d740 1 Processor -- start 2024-03-30T13:12:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af5ae20 0x55739af562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ffba0 con 0x55739af61950 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55739af35ee0 con 0x55739af5ae20 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55739affe510 con 0x55739af61570 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb740d740 10 client.0 _ll_register_callbacks cb 0x7f4ea4007a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af8da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35952/0 (socket says 172.21.15.66:35952) 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/2839686931 learned_addr learned my addr 172.21.15.66:0/2839686931 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af5ae20 0x55739af562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb1ce2640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 msgr2=0x55739af567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af5ae20 msgr2=0x55739af562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af5ae20 0x55739af562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb2ce4640 1 -- 172.21.15.66:0/2839686931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739af62fe0 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.170+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af5ae20 0x55739af562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb2ce4640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af8da20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4000f80 tx=0x7f4ea4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceda1833e570ffb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4e8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/2839686931 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea402f060 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/2839686931 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f4ea4021820 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea8079090 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/2839686931 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4ea4035050 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 10 client.6366 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55739af5c0b0 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/2839686931 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8079090 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/2839686931 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f4ea4038020 con 0x55739af61950 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 10 client.6366 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 10 client.6366 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 20 client.6366 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af8ec50 0x55739af8df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 4 client.6366 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628389}] 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628389}) -- 0x55739aea5e20 con 0x55739af8ec50 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.171+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af8ec50 0x55739af8df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb24e3640 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af8ec50 0x55739af8df60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7063da1995b44b4 server_cookie=2856f115a6805d64 in_seq=0 out_seq=0 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4e8d7fa640 10 client.6366 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4e8d7fa640 1 -- 172.21.15.66:0/2839686931 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3972 (crc 0 0 0) 0x55739aea5e20 con 0x55739af8ec50 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4e8d7fa640 10 client.6366 handle_command_reply: tid=0 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 client.6366 shutdown 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 msgr2=0x55739af8da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af8da20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4000f80 tx=0x7f4ea4004ab0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 shutdown_connections 2024-03-30T13:12:42.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55739af8ec50 0x55739af8df60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739af61950 0x55739af8da20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4e8d7fa640 0 client.6366 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55739af61570 0x55739af567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 --2- 172.21.15.66:0/2839686931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55739af5ae20 0x55739af562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 >> 172.21.15.66:0/2839686931 conn(0x55739af34a50 msgr2=0x55739af829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 shutdown_connections 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 1 -- 172.21.15.66:0/2839686931 wait complete. 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.172+0000 7f4eb740d740 20 client.6366 trim_cache size 0 max 16384 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:42.182 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/acl.h", 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628389, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.057999+0000", 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.526819+0000", 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:42.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1502, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:42.184 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.526819+0000", 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:42.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1502, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.057999+0000" 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1502, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.186 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.057999+0000" 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 146, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 8, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.187 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.100711+0000", 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 72, 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:42.188 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.189 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.190 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:42.191 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:42.192 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:42.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:42.194 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:42.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4e9006c530 msgr2=0x7f4e9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4e9006c530 0x7f4e9006e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4eac135c40 tx=0x7f4e9c007c00 comp rx=0 tx=0).stop 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 msgr2=0x7f4eac134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 0x7f4eac134b40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098c0 tx=0x7f4ea80046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 shutdown_connections 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4e9006c530 0x7f4e9006e9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4eac15f800 0x7f4eac13f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4eac144cb0 0x7f4eac135080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 --2- 172.21.15.66:0/3642144705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4eac072d50 0x7f4eac134b40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 >> 172.21.15.66:0/3642144705 conn(0x7f4eac0b5b00 msgr2=0x7f4eac15ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 shutdown_connections 2024-03-30T13:12:42.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.173+0000 7f4eb4747640 1 -- 172.21.15.66:0/3642144705 wait complete. 2024-03-30T13:12:42.197 DEBUG:tasks.cephfs.test_uninlining:getting inode info #12 of 23:1099511628403 2024-03-30T13:12:42.197 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628403 2024-03-30T13:12:42.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628403 2024-03-30T13:12:42.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.304+0000 7fa3b0275640 1 Processor -- start 2024-03-30T13:12:42.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.304+0000 7fa3b0275640 1 -- start start 2024-03-30T13:12:42.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3b0275640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa3a80d1370 0x7fa3a80d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3b0275640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a80d3fe0 0x7fa3a80d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3b0275640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a80d4470 con 0x7fa3a80d3a50 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3b0275640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a80d5080 con 0x7fa3a80d1370 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3b0275640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a80d4750 con 0x7fa3a80d3fe0 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3ae011640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa3a80d1370 0x7fa3a80d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3ae011640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa3a80d1370 0x7fa3a80d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35964/0 (socket says 172.21.15.66:35964) 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.305+0000 7fa3ae011640 1 -- 172.21.15.66:0/1505390645 learned_addr learned my addr 172.21.15.66:0/1505390645 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae812640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a80d3fe0 0x7fa3a80d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae011640 1 -- 172.21.15.66:0/1505390645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a80d3fe0 msgr2=0x7fa3a80d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae011640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a80d3fe0 0x7fa3a80d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae011640 1 -- 172.21.15.66:0/1505390645 >> v1:172.21.15.117:6789/0 conn(0x7fa3a80d3a50 legacy=0x7fa3a80d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae011640 1 -- 172.21.15.66:0/1505390645 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a80d4a30 con 0x7fa3a80d1370 2024-03-30T13:12:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae812640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a80d3fe0 0x7fa3a80d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.306+0000 7fa3ae011640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a80d07d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d081a1fe9b742dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3ad00f640 1 -- 172.21.15.66:0/1505390645 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4035020 con 0x7fa3a80d1370 2024-03-30T13:12:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3ad00f640 1 -- 172.21.15.66:0/1505390645 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4031640 con 0x7fa3a80d1370 2024-03-30T13:12:42.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3ad00f640 1 -- 172.21.15.66:0/1505390645 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a402d040 con 0x7fa3a80d1370 2024-03-30T13:12:42.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 -- 172.21.15.66:0/1505390645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 msgr2=0x7fa3a80d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a80d07d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a402b8f0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 -- 172.21.15.66:0/1505390645 shutdown_connections 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a80d3fe0 0x7fa3a80d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 --2- 172.21.15.66:0/1505390645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a80d07d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 -- 172.21.15.66:0/1505390645 >> 172.21.15.66:0/1505390645 conn(0x7fa3a80676e0 msgr2=0x7fa3a8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 -- 172.21.15.66:0/1505390645 shutdown_connections 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.307+0000 7fa3b0275640 1 -- 172.21.15.66:0/1505390645 wait complete. 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 Processor -- start 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 -- start start 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a8164490 0x7fa3a8164910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a8166710 0x7fa3a8166b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8162d30 con 0x7fa3a80d1370 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8162bb0 con 0x7fa3a8166710 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3b0275640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a80d2620 con 0x7fa3a8164490 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3ae011640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3ae011640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35980/0 (socket says 172.21.15.66:35980) 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.309+0000 7fa3ae011640 1 -- 172.21.15.66:0/3889054143 learned_addr learned my addr 172.21.15.66:0/3889054143 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae812640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a8166710 0x7fa3a8166b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae011640 1 -- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a8164490 msgr2=0x7fa3a8164910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ad810640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a8164490 0x7fa3a8164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a8164490 0x7fa3a8164910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae011640 1 -- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a8166710 msgr2=0x7fa3a8166b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a8166710 0x7fa3a8166b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae011640 1 -- 172.21.15.66:0/3889054143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a81fd8f0 con 0x7fa3a80d1370 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae812640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a8166710 0x7fa3a8166b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a815a910 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000c00 tx=0x7fa3a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa39effd640 1 -- 172.21.15.66:0/3889054143 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4040070 con 0x7fa3a80d1370 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa39effd640 1 -- 172.21.15.66:0/3889054143 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a40399f0 con 0x7fa3a80d1370 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a81fd610 con 0x7fa3a80d1370 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa39effd640 1 -- 172.21.15.66:0/3889054143 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4035040 con 0x7fa3a80d1370 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.310+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a80c8000 con 0x7fa3a80d1370 2024-03-30T13:12:42.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.312+0000 7fa39effd640 1 -- 172.21.15.66:0/3889054143 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa3a404b060 con 0x7fa3a80d1370 2024-03-30T13:12:42.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.312+0000 7fa39effd640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa38c06c560 0x7fa38c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.312+0000 7fa39effd640 1 -- 172.21.15.66:0/3889054143 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fa3a40ad130 con 0x7fa3a80d1370 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.312+0000 7fa3ad810640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa38c06c560 0x7fa38c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.313+0000 7fa3ad810640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa38c06c560 0x7fa38c06ea20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8166360 tx=0x7fa398009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.313+0000 7fa3b2e5e740 1 Processor -- start 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 -- start start 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab128e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab125ea50 0x563ab128e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563ab11c8590 con 0x563ab128b570 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563ab1166d30 con 0x563ab128b950 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x563ab11c4540 con 0x563ab125ea50 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3ae011640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3ae011640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35992/0 (socket says 172.21.15.66:35992) 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.314+0000 7fa3ae011640 1 -- 172.21.15.66:0/3624485906 learned_addr learned my addr 172.21.15.66:0/3624485906 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ad810640 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab128e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae812640 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab125ea50 0x563ab128e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae011640 1 -- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab125ea50 msgr2=0x563ab128e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab125ea50 0x563ab128e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae011640 1 -- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 msgr2=0x563ab128e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab128e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae011640 1 -- 172.21.15.66:0/3624485906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab0fa8920 con 0x563ab128b570 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae812640 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab125ea50 0x563ab128e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125e510 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40835a0 tx=0x7fa3a407aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaca6765716d9038 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa38bfff640 1 -- 172.21.15.66:0/3624485906 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40a4e60 con 0x563ab128b570 2024-03-30T13:12:42.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa38bfff640 1 -- 172.21.15.66:0/3624485906 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4078f90 con 0x563ab128b570 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.315+0000 7fa38bfff640 1 -- 172.21.15.66:0/3624485906 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4039dc0 con 0x563ab128b570 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3624485906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 msgr2=0x563ab125e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125e510 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40835a0 tx=0x7fa3a407aef0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3624485906 shutdown_connections 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab125ea50 0x563ab128e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab128e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3624485906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125e510 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3624485906 >> 172.21.15.66:0/3624485906 conn(0x563ab1260780 msgr2=0x563ab12a9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3624485906 shutdown_connections 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3624485906 wait complete. 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.316+0000 7fa3b2e5e740 1 -- start start 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.317+0000 7fa3b2e5e740 1 Processor -- start 2024-03-30T13:12:42.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab128f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 0x563ab128c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128c920 0x563ab1271e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563ab11c8590 con 0x563ab128b950 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563ab11649f0 con 0x563ab128c920 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x563ab1262e20 con 0x563ab128b570 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 10 client.0 _ll_register_callbacks cb 0x563ab1168c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 0x563ab128c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 0x563ab128c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36000/0 (socket says 172.21.15.66:36000) 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 -- 172.21.15.66:0/560181813 learned_addr learned my addr 172.21.15.66:0/560181813 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ae812640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128c920 0x563ab1271e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ae011640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab128f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 -- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 msgr2=0x563ab128f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab128f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 -- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128c920 msgr2=0x563ab1271e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128c920 0x563ab1271e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 -- 172.21.15.66:0/560181813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab0e9c210 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3ad810640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 0x563ab128c3e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa3980039f0 tx=0x7fa398000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139cf6107d49e80a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3897fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3897fa640 1 -- 172.21.15.66:0/560181813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800ddb0 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3897fa640 1 -- 172.21.15.66:0/560181813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fa398022c60 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a4078f90 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.318+0000 7fa3897fa640 1 -- 172.21.15.66:0/560181813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa398022e60 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 10 client.6378 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x563ab10090b0 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3897fa640 1 -- 172.21.15.66:0/560181813 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4078f90 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3897fa640 1 -- 172.21.15.66:0/560181813 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fa398022410 con 0x563ab128b950 2024-03-30T13:12:42.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 10 client.6378 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 10 client.6378 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 20 client.6378 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1274880 0x563ab1274c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 4 client.6378 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ab11d5ad0 con 0x563ab1274880 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3ae011640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1274880 0x563ab1274c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3897fa640 10 client.6378 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.319+0000 7fa3ae011640 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1274880 0x563ab1274c90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660f0d70b43c24f3 server_cookie=813e11a51efad8cf in_seq=0 out_seq=0 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3897fa640 1 -- 172.21.15.66:0/560181813 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x563ab11d5ad0 con 0x563ab1274880 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3897fa640 10 client.6378 handle_command_reply: tid=0 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 client.6378 shutdown 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 msgr2=0x563ab128c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 0x563ab128c3e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa3980039f0 tx=0x7fa398000bc0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 shutdown_connections 2024-03-30T13:12:42.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1274880 0x563ab1274c90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128c920 0x563ab1271e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b950 0x563ab128c3e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3897fa640 0 client.6378 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/560181813 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab128f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 >> 172.21.15.66:0/560181813 conn(0x563ab125ea50 msgr2=0x563ab1265340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 shutdown_connections 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/560181813 wait complete. 2024-03-30T13:12:42.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.320+0000 7fa3b2e5e740 20 client.6378 trim_cache size 0 max 16384 2024-03-30T13:12:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.345+0000 7fa3b2e5e740 1 Processor -- start 2024-03-30T13:12:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.345+0000 7fa3b2e5e740 1 -- start start 2024-03-30T13:12:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1273a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128c920 0x563ab128bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8205850 con 0x563ab128b570 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80b8bb0 con 0x563ab128b950 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a80b7eb0 con 0x563ab128c920 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3ae011640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3ae011640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36002/0 (socket says 172.21.15.66:36002) 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3ae011640 1 -- 172.21.15.66:0/3267688948 learned_addr learned my addr 172.21.15.66:0/3267688948 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.346+0000 7fa3ad810640 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1273a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae812640 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128c920 0x563ab128bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae011640 1 -- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128c920 msgr2=0x563ab128bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128c920 0x563ab128bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae011640 1 -- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 msgr2=0x563ab1273a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1273a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae011640 1 -- 172.21.15.66:0/3267688948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab0e752d0 con 0x563ab128b570 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae812640 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128c920 0x563ab128bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa3ae011640 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125dd40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40ab4d0 tx=0x7fa3a403c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9901f01e38aceb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa38b7fe640 1 -- 172.21.15.66:0/3267688948 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40a4e60 con 0x563ab128b570 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa38b7fe640 1 -- 172.21.15.66:0/3267688948 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4078f90 con 0x563ab128b570 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.347+0000 7fa38b7fe640 1 -- 172.21.15.66:0/3267688948 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4039dc0 con 0x563ab128b570 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3267688948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 msgr2=0x563ab125dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125dd40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40ab4d0 tx=0x7fa3a403c070 comp rx=0 tx=0).stop 2024-03-30T13:12:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3267688948 shutdown_connections 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128c920 0x563ab128bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1273a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/3267688948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128b570 0x563ab125dd40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3267688948 >> 172.21.15.66:0/3267688948 conn(0x563ab125ea50 msgr2=0x563ab12a93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3267688948 shutdown_connections 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/3267688948 wait complete. 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 -- start start 2024-03-30T13:12:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.348+0000 7fa3b2e5e740 1 Processor -- start 2024-03-30T13:12:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab1324510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1328020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 0x563ab1328560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563ab1273fa0 con 0x563ab128c920 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80b8500 con 0x563ab128b950 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x563ab128ace0 con 0x563ab128b570 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3b2e5e740 10 client.0 _ll_register_callbacks cb 0x7fa398003610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3ae812640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 0x563ab1328560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3ae812640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 0x563ab1328560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36010/0 (socket says 172.21.15.66:36010) 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3ae812640 1 -- 172.21.15.66:0/4027559487 learned_addr learned my addr 172.21.15.66:0/4027559487 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3ad810640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1328020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.349+0000 7fa3ae011640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab1324510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3ae812640 1 -- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 msgr2=0x563ab1324510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3ae812640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab1324510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3ae812640 1 -- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 msgr2=0x563ab1328020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3ae812640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1328020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3ae812640 1 -- 172.21.15.66:0/4027559487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab12ca140 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3ae812640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 0x563ab1328560 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0007a40 tx=0x7fa3a0007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebfd7572ea88496 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3897fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3897fa640 1 -- 172.21.15.66:0/4027559487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a001d070 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3897fa640 1 -- 172.21.15.66:0/4027559487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fa3a0014e10 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a4078f90 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3897fa640 1 -- 172.21.15.66:0/4027559487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa3a0017440 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3b2e5e740 10 client.6384 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x563ab12b8d50 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3897fa640 1 -- 172.21.15.66:0/4027559487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4078f90 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.350+0000 7fa3897fa640 1 -- 172.21.15.66:0/4027559487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fa3a002b080 con 0x563ab128c920 2024-03-30T13:12:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3b2e5e740 10 client.6384 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3b2e5e740 10 client.6384 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3b2e5e740 20 client.6384 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1328aa0 0x563ab1291650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3b2e5e740 4 client.6384 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628403}] 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628403}) -- 0x563ab11cfe20 con 0x563ab1328aa0 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3ae011640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1328aa0 0x563ab1291650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3897fa640 10 client.6384 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.351+0000 7fa3ae011640 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1328aa0 0x563ab1291650 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e36f11b59442bb0 server_cookie=7295b053b881b210 in_seq=0 out_seq=0 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3897fa640 1 -- 172.21.15.66:0/4027559487 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3983 (crc 0 0 0) 0x563ab11cfe20 con 0x563ab1328aa0 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3897fa640 10 client.6384 handle_command_reply: tid=0 2024-03-30T13:12:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 client.6384 shutdown 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 msgr2=0x563ab1328560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 0x563ab1328560 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0007a40 tx=0x7fa3a0007f00 comp rx=0 tx=0).stop 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 shutdown_connections 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x563ab1328aa0 0x563ab1291650 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ab128c920 0x563ab1328560 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ab128b950 0x563ab1328020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3897fa640 0 client.6384 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 --2- 172.21.15.66:0/4027559487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563ab128b570 0x563ab1324510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 >> 172.21.15.66:0/4027559487 conn(0x563ab125ea50 msgr2=0x563ab128f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 shutdown_connections 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 1 -- 172.21.15.66:0/4027559487 wait complete. 2024-03-30T13:12:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.352+0000 7fa3b2e5e740 20 client.6384 trim_cache size 0 max 16384 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr_trusted.c", 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628403, 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.064059+0000", 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.535819+0000", 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:42.364 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:42.365 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 1094, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.535819+0000", 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:42.366 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1094, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.367 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.064059+0000" 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 1094, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.064059+0000" 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 122, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:42.368 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 36, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.108339+0000", 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 82, 2024-03-30T13:12:42.369 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:42.370 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:42.371 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.372 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:42.373 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.374 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:42.376 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:42.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa38c06c560 msgr2=0x7fa38c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa38c06c560 0x7fa38c06ea20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8166360 tx=0x7fa398009040 comp rx=0 tx=0).stop 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 msgr2=0x7fa3a815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a815a910 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000c00 tx=0x7fa3a40046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 shutdown_connections 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa38c06c560 0x7fa38c06ea20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a8166710 0x7fa3a8166b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3a8164490 0x7fa3a8164910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 --2- 172.21.15.66:0/3889054143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3a80d1370 0x7fa3a815a910 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 >> 172.21.15.66:0/3889054143 conn(0x7fa3a80676e0 msgr2=0x7fa3a8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 shutdown_connections 2024-03-30T13:12:42.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.353+0000 7fa3b0275640 1 -- 172.21.15.66:0/3889054143 wait complete. 2024-03-30T13:12:42.379 DEBUG:tasks.cephfs.test_uninlining:getting inode info #13 of 23:1099511628401 2024-03-30T13:12:42.379 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628401 2024-03-30T13:12:42.379 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628401 2024-03-30T13:12:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.486+0000 7f66c6159640 1 Processor -- start 2024-03-30T13:12:42.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.486+0000 7f66c6159640 1 -- start start 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66c6159640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c0167120 0x7f66c0169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66c6159640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c00d2ff0 con 0x7f66c00ba090 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66c6159640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c00d32d0 con 0x7f66c00b5dc0 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66c6159640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f66c00d2d90 con 0x7f66c0167120 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66b7fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f66c00ba090 0x7f66c00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37752/0 (socket says 172.21.15.66:37752) 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66b7fff640 1 -- 172.21.15.66:0/3081221460 learned_addr learned my addr 172.21.15.66:0/3081221460 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66bffff640 1 --2- 172.21.15.66:0/3081221460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c0167120 0x7f66c0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608923344 0 0) 0x7f66c00d2ff0 con 0x7f66c00ba090 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66ac003520 con 0x7f66c00ba090 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190123689 0 0) 0x7f66c00d32d0 con 0x7f66c00b5dc0 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66c00d2ff0 con 0x7f66c00b5dc0 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66bffff640 1 -- 172.21.15.66:0/3081221460 >> v1:172.21.15.117:6789/0 conn(0x7f66c00b5dc0 legacy=0x7f66c00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66bffff640 1 -- 172.21.15.66:0/3081221460 >> v1:172.21.15.66:6789/0 conn(0x7f66c00ba090 legacy=0x7f66c00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 540360361 0 0) 0x7f66ac003520 con 0x7f66c00ba090 2024-03-30T13:12:42.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.487+0000 7f66bffff640 1 -- 172.21.15.66:0/3081221460 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c00d35b0 con 0x7f66c0167120 2024-03-30T13:12:42.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.489+0000 7f66bffff640 1 --2- 172.21.15.66:0/3081221460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c0167120 0x7f66c0169510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009df0 tx=0x7f66b802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb2cd5a30c4bfde5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.489+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8039070 con 0x7f66c0167120 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.489+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b802c910 con 0x7f66c0167120 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.489+0000 7f66beffd640 1 -- 172.21.15.66:0/3081221460 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8001670 con 0x7f66c0167120 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 -- 172.21.15.66:0/3081221460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c0167120 msgr2=0x7f66c0169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 --2- 172.21.15.66:0/3081221460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c0167120 0x7f66c0169510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009df0 tx=0x7f66b802bde0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 -- 172.21.15.66:0/3081221460 shutdown_connections 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 --2- 172.21.15.66:0/3081221460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c0167120 0x7f66c0169510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 -- 172.21.15.66:0/3081221460 >> 172.21.15.66:0/3081221460 conn(0x7f66c00678f0 msgr2=0x7f66c00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 -- 172.21.15.66:0/3081221460 shutdown_connections 2024-03-30T13:12:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.490+0000 7f66c6159640 1 -- 172.21.15.66:0/3081221460 wait complete. 2024-03-30T13:12:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.491+0000 7f66c6159640 1 Processor -- start 2024-03-30T13:12:42.496 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.491+0000 7f66c6159640 1 -- start start 2024-03-30T13:12:42.496 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66c6159640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c00b5dc0 0x7f66c0161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66c6159640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c00ba090 0x7f66c0156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66c6159640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 0x7f66c01f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66c6159640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00d4090 con 0x7f66c0167120 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66c6159640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66c00d3b70 con 0x7f66c00b5dc0 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66c6159640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f66c00b83e0 con 0x7f66c00ba090 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 0x7f66c01f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 0x7f66c01f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36026/0 (socket says 172.21.15.66:36026) 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 -- 172.21.15.66:0/3754474516 learned_addr learned my addr 172.21.15.66:0/3754474516 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c00b5dc0 0x7f66c0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66b7fff640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c00ba090 0x7f66c0156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 -- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c00ba090 msgr2=0x7f66c0156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c00ba090 0x7f66c0156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 -- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c00b5dc0 msgr2=0x7f66c0161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c00b5dc0 0x7f66c0161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 -- 172.21.15.66:0/3754474516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c00b4eb0 con 0x7f66c0167120 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c00b5dc0 0x7f66c0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.492+0000 7f66bffff640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 0x7f66c01f4290 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009df0 tx=0x7f66b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.493+0000 7f66bcff9640 1 -- 172.21.15.66:0/3754474516 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8039050 con 0x7f66c0167120 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.493+0000 7f66bcff9640 1 -- 172.21.15.66:0/3754474516 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b803a040 con 0x7f66c0167120 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.493+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c00b5190 con 0x7f66c0167120 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.493+0000 7f66bcff9640 1 -- 172.21.15.66:0/3754474516 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8009450 con 0x7f66c0167120 2024-03-30T13:12:42.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.493+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c00cd360 con 0x7f66c0167120 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.494+0000 7f66bcff9640 1 -- 172.21.15.66:0/3754474516 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f66b8047020 con 0x7f66c0167120 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.495+0000 7f66bcff9640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f66a006c530 0x7f66a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.495+0000 7f66bcff9640 1 -- 172.21.15.66:0/3754474516 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f66b80ad970 con 0x7f66c0167120 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.495+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f66a006c530 0x7f66a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.495+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f66a006c530 0x7f66a006e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66b00011e0 tx=0x7f66b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.496+0000 7f66c8e23740 1 Processor -- start 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.496+0000 7f66c8e23740 1 -- start start 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 0x559351b76510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351ba6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351b76a50 0x559351ba6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66c8e23740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559351ae0590 con 0x559351ba3570 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559351a7ed30 con 0x559351ba3950 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x559351adc540 con 0x559351b76a50 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 0x559351b76510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 0x559351b76510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36040/0 (socket says 172.21.15.66:36040) 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 -- 172.21.15.66:0/3787188693 learned_addr learned my addr 172.21.15.66:0/3787188693 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bffff640 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351b76a50 0x559351ba6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66b7fff640 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351ba6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 -- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351b76a50 msgr2=0x559351ba6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351b76a50 0x559351ba6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 -- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 msgr2=0x559351ba6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351ba6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 -- 172.21.15.66:0/3787188693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593518c0920 con 0x559351ba3570 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.497+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 0x559351b76510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005200 tx=0x7f66b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bab72bffb493299 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66b57fa640 1 -- 172.21.15.66:0/3787188693 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00035f0 con 0x559351ba3570 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66b57fa640 1 -- 172.21.15.66:0/3787188693 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0003790 con 0x559351ba3570 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66b57fa640 1 -- 172.21.15.66:0/3787188693 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003b40 con 0x559351ba3570 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 -- 172.21.15.66:0/3787188693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 msgr2=0x559351b76510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 0x559351b76510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005200 tx=0x7f66b0011000 comp rx=0 tx=0).stop 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 -- 172.21.15.66:0/3787188693 shutdown_connections 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351b76a50 0x559351ba6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351ba6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3787188693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3570 0x559351b76510 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 -- 172.21.15.66:0/3787188693 >> 172.21.15.66:0/3787188693 conn(0x559351b78780 msgr2=0x559351bc1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 -- 172.21.15.66:0/3787188693 shutdown_connections 2024-03-30T13:12:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.498+0000 7f66c8e23740 1 -- 172.21.15.66:0/3787188693 wait complete. 2024-03-30T13:12:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.499+0000 7f66c8e23740 1 -- start start 2024-03-30T13:12:42.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.499+0000 7f66c8e23740 1 Processor -- start 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.500+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3570 0x559351b94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.500+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3950 0x559351ba57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.500+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351ba81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66c8e23740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559351b7ae20 con 0x559351ba7d20 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559351a7c9f0 con 0x559351ba3570 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x559351b7d660 con 0x559351ba3950 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351ba81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351ba81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36042/0 (socket says 172.21.15.66:36042) 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 -- 172.21.15.66:0/1165673893 learned_addr learned my addr 172.21.15.66:0/1165673893 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66c8e23740 10 client.0 _ll_register_callbacks cb 0x559351a80c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3570 0x559351b94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66b7fff640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3950 0x559351ba57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 -- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3950 msgr2=0x559351ba57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3950 0x559351ba57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 -- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3570 msgr2=0x559351b94e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3570 0x559351b94e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 -- 172.21.15.66:0/1165673893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593517b4210 con 0x559351ba7d20 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66bffff640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351ba81a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009d90 tx=0x7f66b807b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc86adfd09fa246a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f669e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f669e7fc640 1 -- 172.21.15.66:0/1165673893 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b80840a0 con 0x559351ba7d20 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f669e7fc640 1 -- 172.21.15.66:0/1165673893 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f66b807aba0 con 0x559351ba7d20 2024-03-30T13:12:42.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.501+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b0003790 con 0x559351ba7d20 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f669e7fc640 1 -- 172.21.15.66:0/1165673893 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f66b809a050 con 0x559351ba7d20 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 10 client.6396 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5593519210b0 con 0x559351ba7d20 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f669e7fc640 1 -- 172.21.15.66:0/1165673893 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0003790 con 0x559351ba7d20 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f669e7fc640 1 -- 172.21.15.66:0/1165673893 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f66b80774a0 con 0x559351ba7d20 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 10 client.6396 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 10 client.6396 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 20 client.6396 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bd5500 0x559351bd5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 4 client.6396 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559351aedad0 con 0x559351bd5500 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bd5500 0x559351bd5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f669e7fc640 10 client.6396 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.502+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bd5500 0x559351bd5930 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15aef8aace0d6e server_cookie=c7c6c84af7f7a417 in_seq=0 out_seq=0 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.503+0000 7f669e7fc640 1 -- 172.21.15.66:0/1165673893 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x559351aedad0 con 0x559351bd5500 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.503+0000 7f669e7fc640 10 client.6396 handle_command_reply: tid=0 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.503+0000 7f66c8e23740 1 client.6396 shutdown 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 msgr2=0x559351ba81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351ba81a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009d90 tx=0x7f66b807b750 comp rx=0 tx=0).stop 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 shutdown_connections 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bd5500 0x559351bd5930 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351ba81a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3950 0x559351ba57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f669e7fc640 0 client.6396 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 --2- 172.21.15.66:0/1165673893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3570 0x559351b94e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 >> 172.21.15.66:0/1165673893 conn(0x559351b76a50 msgr2=0x559351ba1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 shutdown_connections 2024-03-30T13:12:42.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 1 -- 172.21.15.66:0/1165673893 wait complete. 2024-03-30T13:12:42.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.504+0000 7f66c8e23740 20 client.6396 trim_cache size 0 max 16384 2024-03-30T13:12:42.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.519+0000 7f66c8e23740 1 Processor -- start 2024-03-30T13:12:42.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.519+0000 7f66c8e23740 1 -- start start 2024-03-30T13:12:42.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351b94280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351bc35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351be4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66c8e23740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66c02011e0 con 0x559351ba7d20 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66c02001e0 con 0x559351ba3950 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f66c015fe00 con 0x559351ba3570 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351be4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351be4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36044/0 (socket says 172.21.15.66:36044) 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 -- 172.21.15.66:0/793336658 learned_addr learned my addr 172.21.15.66:0/793336658 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66b7fff640 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351bc35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351b94280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 -- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 msgr2=0x559351b94280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351b94280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 -- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 msgr2=0x559351bc35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351bc35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 -- 172.21.15.66:0/793336658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935178d2d0 con 0x559351ba7d20 2024-03-30T13:12:42.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.520+0000 7f66bffff640 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351be4540 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f66b80780d0 tx=0x7f66b807f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ada7cb0111b658 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.521+0000 7f66b4ff9640 1 -- 172.21.15.66:0/793336658 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8084380 con 0x559351ba7d20 2024-03-30T13:12:42.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.521+0000 7f66b4ff9640 1 -- 172.21.15.66:0/793336658 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b80aa070 con 0x559351ba7d20 2024-03-30T13:12:42.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.521+0000 7f66b4ff9640 1 -- 172.21.15.66:0/793336658 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b80840c0 con 0x559351ba7d20 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.521+0000 7f66c8e23740 1 -- 172.21.15.66:0/793336658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 msgr2=0x559351be4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.521+0000 7f66c8e23740 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351be4540 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f66b80780d0 tx=0x7f66b807f000 comp rx=0 tx=0).stop 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 -- 172.21.15.66:0/793336658 shutdown_connections 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba7d20 0x559351be4540 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba3950 0x559351bc35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 --2- 172.21.15.66:0/793336658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351b94280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 -- 172.21.15.66:0/793336658 >> 172.21.15.66:0/793336658 conn(0x559351b76a50 msgr2=0x559351bd58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 -- 172.21.15.66:0/793336658 shutdown_connections 2024-03-30T13:12:42.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 -- 172.21.15.66:0/793336658 wait complete. 2024-03-30T13:12:42.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.522+0000 7f66c8e23740 1 -- start start 2024-03-30T13:12:42.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.523+0000 7f66c8e23740 1 Processor -- start 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351bc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 0x559351c3a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba7d20 0x559351c3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559351bbf6a0 con 0x559351ba3950 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01ffb90 con 0x559351ba7d20 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x559351bc5020 con 0x559351ba3570 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66c8e23740 10 client.0 _ll_register_callbacks cb 0x7f66b80779f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 0x559351c3a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 0x559351c3a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36058/0 (socket says 172.21.15.66:36058) 2024-03-30T13:12:42.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 -- 172.21.15.66:0/3314832042 learned_addr learned my addr 172.21.15.66:0/3314832042 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351bc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 -- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 msgr2=0x559351bc4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351bc4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 -- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba7d20 msgr2=0x559351c3a8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba7d20 0x559351c3a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.524+0000 7f66b7fff640 1 -- 172.21.15.66:0/3314832042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559351bc49c0 con 0x559351ba3950 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66b7fff640 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 0x559351c3a380 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005880 tx=0x7f66a8016ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0d8227038247d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f669e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f669e7fc640 1 -- 172.21.15.66:0/3314832042 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80147d0 con 0x559351ba3950 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f669e7fc640 1 -- 172.21.15.66:0/3314832042 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f66a8021d10 con 0x559351ba3950 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b80aa070 con 0x559351ba3950 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f669e7fc640 1 -- 172.21.15.66:0/3314832042 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f66a8014970 con 0x559351ba3950 2024-03-30T13:12:42.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 10 client.6402 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x559351ba1960 con 0x559351ba3950 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f669e7fc640 1 -- 172.21.15.66:0/3314832042 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b80aa070 con 0x559351ba3950 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f669e7fc640 1 -- 172.21.15.66:0/3314832042 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f66a8014b10 con 0x559351ba3950 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 10 client.6402 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 10 client.6402 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 20 client.6402 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bc3680 0x559351c3d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 4 client.6402 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628401}] 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628401}) -- 0x559351ae7e20 con 0x559351bc3680 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.525+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bc3680 0x559351c3d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f669e7fc640 10 client.6402 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66bf7fe640 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bc3680 0x559351c3d5d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b8d610aafc1b53 server_cookie=ee394dfe2f7d4bea in_seq=0 out_seq=0 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f669e7fc640 1 -- 172.21.15.66:0/3314832042 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3975 (crc 0 0 0) 0x559351ae7e20 con 0x559351bc3680 2024-03-30T13:12:42.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f669e7fc640 10 client.6402 handle_command_reply: tid=0 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 client.6402 shutdown 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 msgr2=0x559351c3a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 0x559351c3a380 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005880 tx=0x7f66a8016ad0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 shutdown_connections 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x559351bc3680 0x559351c3d5d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559351ba7d20 0x559351c3a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559351ba3950 0x559351c3a380 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 --2- 172.21.15.66:0/3314832042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559351ba3570 0x559351bc4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 >> 172.21.15.66:0/3314832042 conn(0x559351b76a50 msgr2=0x559351bd3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f669e7fc640 0 client.6402 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.526+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 shutdown_connections 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c8e23740 1 -- 172.21.15.66:0/3314832042 wait complete. 2024-03-30T13:12:42.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c8e23740 20 client.6402 trim_cache size 0 max 16384 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/ext2/xattr.h", 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628401, 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.067546+0000", 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.533819+0000", 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:42.536 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:42.537 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 3340, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.533819+0000", 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:42.538 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3340, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.067546+0000" 2024-03-30T13:12:42.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3340, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.067546+0000" 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 130, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 32, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:42.540 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.112549+0000", 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 86, 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:42.541 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:42.542 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:42.543 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.544 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:42.545 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:42.546 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:42.547 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:42.548 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:42.549 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f66a006c530 msgr2=0x7f66a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f66a006c530 0x7f66a006e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66b00011e0 tx=0x7f66b0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 msgr2=0x7f66c01f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 0x7f66c01f4290 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009df0 tx=0x7f66b8004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 shutdown_connections 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f66a006c530 0x7f66a006e9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0167120 0x7f66c01f4290 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66c00ba090 0x7f66c0156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 --2- 172.21.15.66:0/3754474516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c00b5dc0 0x7f66c0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 >> 172.21.15.66:0/3754474516 conn(0x7f66c00678f0 msgr2=0x7f66c00d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 shutdown_connections 2024-03-30T13:12:42.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.527+0000 7f66c6159640 1 -- 172.21.15.66:0/3754474516 wait complete. 2024-03-30T13:12:42.551 DEBUG:tasks.cephfs.test_uninlining:getting inode info #14 of 23:1099511628406 2024-03-30T13:12:42.551 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628406 2024-03-30T13:12:42.552 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628406 2024-03-30T13:12:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.721+0000 7f11184df640 1 Processor -- start 2024-03-30T13:12:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.721+0000 7f11184df640 1 -- start start 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f11184df640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1110145d70 0x7f1110160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f11184df640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f111015ceb0 0x7f111015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f11184df640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11100cc970 con 0x7f11100ce490 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f11184df640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f11100ce8f0 con 0x7f111015ceb0 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f11184df640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f1110146150 con 0x7f1110145d70 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f111015ceb0 0x7f111015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f111015ceb0 0x7f111015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36060/0 (socket says 172.21.15.66:36060) 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 -- 172.21.15.66:0/3021091452 learned_addr learned my addr 172.21.15.66:0/3021091452 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 -- 172.21.15.66:0/3021091452 >> v2:172.21.15.117:3300/0 conn(0x7f1110145d70 msgr2=0x7f1110160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f111627b640 1 --2- 172.21.15.66:0/3021091452 >> v2:172.21.15.117:3300/0 conn(0x7f1110145d70 0x7f1110160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 --2- 172.21.15.66:0/3021091452 >> v2:172.21.15.117:3300/0 conn(0x7f1110145d70 0x7f1110160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 -- 172.21.15.66:0/3021091452 >> v1:172.21.15.117:6789/0 conn(0x7f11100ce490 legacy=0x7f11100cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.723+0000 7f1116a7c640 1 -- 172.21.15.66:0/3021091452 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11100ccc50 con 0x7f111015ceb0 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.724+0000 7f1116a7c640 1 --2- 172.21.15.66:0/3021091452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111015ceb0 0x7f111015d320 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f11100ceb60 tx=0x7f110802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f95626692b0c51 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.724+0000 7f1115279640 1 -- 172.21.15.66:0/3021091452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110802c9f0 con 0x7f111015ceb0 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.724+0000 7f1115279640 1 -- 172.21.15.66:0/3021091452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110802cb90 con 0x7f111015ceb0 2024-03-30T13:12:42.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.724+0000 7f1115279640 1 -- 172.21.15.66:0/3021091452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080343f0 con 0x7f111015ceb0 2024-03-30T13:12:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 -- 172.21.15.66:0/3021091452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111015ceb0 msgr2=0x7f111015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 --2- 172.21.15.66:0/3021091452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111015ceb0 0x7f111015d320 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f11100ceb60 tx=0x7f110802be50 comp rx=0 tx=0).stop 2024-03-30T13:12:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 -- 172.21.15.66:0/3021091452 shutdown_connections 2024-03-30T13:12:42.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 --2- 172.21.15.66:0/3021091452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111015ceb0 0x7f111015d320 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 --2- 172.21.15.66:0/3021091452 >> v2:172.21.15.117:3300/0 conn(0x7f1110145d70 0x7f1110160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 -- 172.21.15.66:0/3021091452 >> 172.21.15.66:0/3021091452 conn(0x7f11100b3e70 msgr2=0x7f11100b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 -- 172.21.15.66:0/3021091452 shutdown_connections 2024-03-30T13:12:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.725+0000 7f11184df640 1 -- 172.21.15.66:0/3021091452 wait complete. 2024-03-30T13:12:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.727+0000 7f11184df640 1 Processor -- start 2024-03-30T13:12:42.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.727+0000 7f11184df640 1 -- start start 2024-03-30T13:12:42.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f11184df640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 0x7f1110142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f11184df640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1110145d70 0x7f11101424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f11184df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f111015ceb0 0x7f111015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f11184df640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11100cc500 con 0x7f11100ce490 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f11184df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f111015ca20 con 0x7f111015ceb0 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f11184df640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f11100ccf30 con 0x7f1110145d70 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f111627b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 0x7f1110142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f111627b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 0x7f1110142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36066/0 (socket says 172.21.15.66:36066) 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.728+0000 7f111627b640 1 -- 172.21.15.66:0/3131787483 learned_addr learned my addr 172.21.15.66:0/3131787483 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f1116a7c640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f111015ceb0 0x7f111015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f1115a7a640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1110145d70 0x7f11101424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f111627b640 1 -- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1110145d70 msgr2=0x7f11101424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f111627b640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1110145d70 0x7f11101424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f111627b640 1 -- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f111015ceb0 msgr2=0x7f111015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f111627b640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f111015ceb0 0x7f111015e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f111627b640 1 -- 172.21.15.66:0/3131787483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1110158300 con 0x7f11100ce490 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f1115a7a640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1110145d70 0x7f11101424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f111627b640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 0x7f1110142d60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f110000c7f0 tx=0x7f110000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f1106ffd640 1 -- 172.21.15.66:0/3131787483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100015070 con 0x7f11100ce490 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f1106ffd640 1 -- 172.21.15.66:0/3131787483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100004240 con 0x7f11100ce490 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f1106ffd640 1 -- 172.21.15.66:0/3131787483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100003610 con 0x7f11100ce490 2024-03-30T13:12:42.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11101585e0 con 0x7f11100ce490 2024-03-30T13:12:42.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.729+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1110158ba0 con 0x7f11100ce490 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.731+0000 7f1106ffd640 1 -- 172.21.15.66:0/3131787483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f11000039c0 con 0x7f11100ce490 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.731+0000 7f1106ffd640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f10f406c530 0x7f10f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.731+0000 7f1115a7a640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f10f406c530 0x7f10f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.732+0000 7f1106ffd640 1 -- 172.21.15.66:0/3131787483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f110008b830 con 0x7f11100ce490 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.732+0000 7f1115a7a640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f10f406c530 0x7f10f406e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.732+0000 7f111b19d740 1 Processor -- start 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.732+0000 7f111b19d740 1 -- start start 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe76205510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111b19d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe76235300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76205a50 0x55fe76235840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111b19d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fe7616f590 con 0x55fe76232950 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe7610dd30 con 0x55fe76232570 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fe7616b540 con 0x55fe76205a50 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f1115a7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe76235300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f1115a7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe76235300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36078/0 (socket says 172.21.15.66:36078) 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f1115a7a640 1 -- 172.21.15.66:0/537647812 learned_addr learned my addr 172.21.15.66:0/537647812 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f111627b640 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe76205510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.733+0000 7f1116a7c640 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76205a50 0x55fe76235840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f1115a7a640 1 -- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76205a50 msgr2=0x55fe76235840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f1115a7a640 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76205a50 0x55fe76235840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f1115a7a640 1 -- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 msgr2=0x55fe76205510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f1115a7a640 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe76205510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f1115a7a640 1 -- 172.21.15.66:0/537647812 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe75f4f920 con 0x55fe76232950 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f111627b640 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe76205510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f1115a7a640 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe76235300 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d910 tx=0x7f110c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712397e66b06986d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f10f3fff640 1 -- 172.21.15.66:0/537647812 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c024070 con 0x55fe76232950 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f10f3fff640 1 -- 172.21.15.66:0/537647812 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c0149b0 con 0x55fe76232950 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.734+0000 7f10f3fff640 1 -- 172.21.15.66:0/537647812 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c014d60 con 0x55fe76232950 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 -- 172.21.15.66:0/537647812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 msgr2=0x55fe76235300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe76235300 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d910 tx=0x7f110c010000 comp rx=0 tx=0).stop 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 -- 172.21.15.66:0/537647812 shutdown_connections 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76205a50 0x55fe76235840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe76235300 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 --2- 172.21.15.66:0/537647812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe76205510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 -- 172.21.15.66:0/537647812 >> 172.21.15.66:0/537647812 conn(0x55fe76207780 msgr2=0x55fe76250300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 -- 172.21.15.66:0/537647812 shutdown_connections 2024-03-30T13:12:42.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 -- 172.21.15.66:0/537647812 wait complete. 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.735+0000 7f111b19d740 1 -- start start 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 Processor -- start 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe7624d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe7624e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fe76209e20 con 0x55fe76232950 2024-03-30T13:12:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe7610b9f0 con 0x55fe76232570 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.736+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fe7620c660 con 0x55fe7624deb0 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f111b19d740 10 client.0 _ll_register_callbacks cb 0x55fe7610fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762346f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36092/0 (socket says 172.21.15.66:36092) 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 -- 172.21.15.66:0/1364167821 learned_addr learned my addr 172.21.15.66:0/1364167821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f111627b640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe7624d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1116a7c640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe7624e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 -- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 msgr2=0x55fe7624e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe7624e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 -- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 msgr2=0x55fe7624d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe7624d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 -- 172.21.15.66:0/1364167821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe75e43210 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1116a7c640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe7624e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f1115a7a640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762346f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f110c022040 tx=0x7f110c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5766f04cf0ba3d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f10f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f10f17fa640 1 -- 172.21.15.66:0/1364167821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c02c070 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f10f17fa640 1 -- 172.21.15.66:0/1364167821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f110c004220 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.737+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c0149b0 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f10f17fa640 1 -- 172.21.15.66:0/1364167821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f110c036050 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 10 client.6414 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55fe75fb00b0 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f10f17fa640 1 -- 172.21.15.66:0/1364167821 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c0149b0 con 0x55fe76232950 2024-03-30T13:12:42.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f10f17fa640 1 -- 172.21.15.66:0/1364167821 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f110c039020 con 0x55fe76232950 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 10 client.6414 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 10 client.6414 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 20 client.6414 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe76260dc0 0x55fe762611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 4 client.6414 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe7617cad0 con 0x55fe76260dc0 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111627b640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe76260dc0 0x55fe762611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f10f17fa640 10 client.6414 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.738+0000 7f111627b640 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe76260dc0 0x55fe762611a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac639bc8cdb8679 server_cookie=777ecd166d9b8ed2 in_seq=0 out_seq=0 2024-03-30T13:12:42.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f10f17fa640 1 -- 172.21.15.66:0/1364167821 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55fe7617cad0 con 0x55fe76260dc0 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f10f17fa640 10 client.6414 handle_command_reply: tid=0 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 client.6414 shutdown 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 msgr2=0x55fe762346f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762346f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f110c022040 tx=0x7f110c004640 comp rx=0 tx=0).stop 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 shutdown_connections 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe76260dc0 0x55fe762611a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe7624e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762346f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f10f17fa640 0 client.6414 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 --2- 172.21.15.66:0/1364167821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232570 0x55fe7624d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 >> 172.21.15.66:0/1364167821 conn(0x55fe76205a50 msgr2=0x55fe76253810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 shutdown_connections 2024-03-30T13:12:42.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.739+0000 7f111b19d740 1 -- 172.21.15.66:0/1364167821 wait complete. 2024-03-30T13:12:42.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.740+0000 7f111b19d740 20 client.6414 trim_cache size 0 max 16384 2024-03-30T13:12:42.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.758+0000 7f111b19d740 1 Processor -- start 2024-03-30T13:12:42.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.758+0000 7f111b19d740 1 -- start start 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111b19d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 0x55fe762524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232950 0x55fe76253810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe76253d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111b19d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11101ff080 con 0x55fe76232570 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11101fe140 con 0x55fe76232950 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f11100b5b40 con 0x55fe7624deb0 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 0x55fe762524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 0x55fe762524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36096/0 (socket says 172.21.15.66:36096) 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 -- 172.21.15.66:0/352499010 learned_addr learned my addr 172.21.15.66:0/352499010 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f1116a7c640 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe76253d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f1115a7a640 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232950 0x55fe76253810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 -- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 msgr2=0x55fe76253d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe76253d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 -- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232950 msgr2=0x55fe76253810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232950 0x55fe76253810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 -- 172.21.15.66:0/352499010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe75e1c2d0 con 0x55fe76232570 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f111627b640 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 0x55fe762524b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c00 tx=0x7f110001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fb1eaed94ad746 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f10f37fe640 1 -- 172.21.15.66:0/352499010 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110001e040 con 0x55fe76232570 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.759+0000 7f10f37fe640 1 -- 172.21.15.66:0/352499010 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100055d10 con 0x55fe76232570 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f10f37fe640 1 -- 172.21.15.66:0/352499010 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100071de0 con 0x55fe76232570 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 -- 172.21.15.66:0/352499010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 msgr2=0x55fe762524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 0x55fe762524b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c00 tx=0x7f110001b040 comp rx=0 tx=0).stop 2024-03-30T13:12:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 -- 172.21.15.66:0/352499010 shutdown_connections 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe7624deb0 0x55fe76253d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe76232950 0x55fe76253810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 --2- 172.21.15.66:0/352499010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232570 0x55fe762524b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 -- 172.21.15.66:0/352499010 >> 172.21.15.66:0/352499010 conn(0x55fe76205a50 msgr2=0x55fe76274590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 -- 172.21.15.66:0/352499010 shutdown_connections 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.760+0000 7f111b19d740 1 -- 172.21.15.66:0/352499010 wait complete. 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.761+0000 7f111b19d740 1 -- start start 2024-03-30T13:12:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.761+0000 7f111b19d740 1 Processor -- start 2024-03-30T13:12:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76232570 0x55fe762692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe7624deb0 0x55fe762048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fe762012f0 con 0x55fe76232950 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11101fdc40 con 0x55fe7624deb0 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55fe76235ea0 con 0x55fe76232570 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 10 client.0 _ll_register_callbacks cb 0x7f110c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36106/0 (socket says 172.21.15.66:36106) 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111627b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76232570 0x55fe762692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 -- 172.21.15.66:0/1054088721 learned_addr learned my addr 172.21.15.66:0/1054088721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1116a7c640 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe7624deb0 0x55fe762048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 -- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76232570 msgr2=0x55fe762692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76232570 0x55fe762692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 -- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe7624deb0 msgr2=0x55fe762048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe7624deb0 0x55fe762048e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 -- 172.21.15.66:0/1054088721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe76251430 con 0x55fe76232950 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f1115a7a640 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762043a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f110c0069b0 tx=0x7f110c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1c6cc481e2bb02 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f10f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f10f17fa640 1 -- 172.21.15.66:0/1054088721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c03f060 con 0x55fe76232950 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f10f17fa640 1 -- 172.21.15.66:0/1054088721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f110c0288e0 con 0x55fe76232950 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.762+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1100055d10 con 0x55fe76232950 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f10f17fa640 1 -- 172.21.15.66:0/1054088721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f110c045050 con 0x55fe76232950 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 10 client.6420 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55fe762cc5e0 con 0x55fe76232950 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f10f17fa640 1 -- 172.21.15.66:0/1054088721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100055d10 con 0x55fe76232950 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f10f17fa640 1 -- 172.21.15.66:0/1054088721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f110c048020 con 0x55fe76232950 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 10 client.6420 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 10 client.6420 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 20 client.6420 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe762c27b0 0x55fe762cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 4 client.6420 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628406}] 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628406}) -- 0x55fe76176e20 con 0x55fe762c27b0 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.763+0000 7f111627b640 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe762c27b0 0x55fe762cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f10f17fa640 10 client.6420 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111627b640 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe762c27b0 0x55fe762cf0b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f678c9ef0e6619a server_cookie=137238aa1203b4bd in_seq=0 out_seq=0 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f10f17fa640 1 -- 172.21.15.66:0/1054088721 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3975 (crc 0 0 0) 0x55fe76176e20 con 0x55fe762c27b0 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f10f17fa640 10 client.6420 handle_command_reply: tid=0 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 client.6420 shutdown 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 msgr2=0x55fe762043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762043a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f110c0069b0 tx=0x7f110c01d000 comp rx=0 tx=0).stop 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 shutdown_connections 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55fe762c27b0 0x55fe762cf0b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe7624deb0 0x55fe762048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe76232950 0x55fe762043a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f10f17fa640 0 client.6420 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 --2- 172.21.15.66:0/1054088721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fe76232570 0x55fe762692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 >> 172.21.15.66:0/1054088721 conn(0x55fe76205a50 msgr2=0x55fe7625ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 shutdown_connections 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 1 -- 172.21.15.66:0/1054088721 wait complete. 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.764+0000 7f111b19d740 20 client.6420 trim_cache size 0 max 16384 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/Kconfig", 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628406, 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.072562+0000", 2024-03-30T13:12:42.776 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.636818+0000", 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.777 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 228, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:42.778 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.636818+0000", 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:42.779 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 228, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.072562+0000" 2024-03-30T13:12:42.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 228, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.072562+0000" 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 224, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:42.781 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 2, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.119819+0000", 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 58, 2024-03-30T13:12:42.782 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:42.783 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.784 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:42.785 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.786 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:42.787 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.788 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:42.789 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:42.790 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:42.791 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f10f406c530 msgr2=0x7f10f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f10f406c530 0x7f10f406e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 msgr2=0x7f1110142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 0x7f1110142d60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f110000c7f0 tx=0x7f110000ccb0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 shutdown_connections 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f10f406c530 0x7f10f406e9f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f111015ceb0 0x7f111015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1110145d70 0x7f11101424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 --2- 172.21.15.66:0/3131787483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11100ce490 0x7f1110142d60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 >> 172.21.15.66:0/3131787483 conn(0x7f11100b3e70 msgr2=0x7f11100ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 shutdown_connections 2024-03-30T13:12:42.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.765+0000 7f11184df640 1 -- 172.21.15.66:0/3131787483 wait complete. 2024-03-30T13:12:42.794 DEBUG:tasks.cephfs.test_uninlining:getting inode info #15 of 23:1099511628419 2024-03-30T13:12:42.794 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628419 2024-03-30T13:12:42.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628419 2024-03-30T13:12:42.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.902+0000 7f592ec22640 1 Processor -- start 2024-03-30T13:12:42.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.902+0000 7f592ec22640 1 -- start start 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592ec22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59280b2f20 0x7f592815f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592ec22640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59280b35e0 con 0x7f5928144cb0 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592ec22640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59280b38c0 con 0x7f5928072d50 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592ec22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f59280b3e80 con 0x7f59280b2f20 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592c9be640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5928144cb0 0x7f5928072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37758/0 (socket says 172.21.15.66:37758) 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592c9be640 1 -- 172.21.15.66:0/3560871343 learned_addr learned my addr 172.21.15.66:0/3560871343 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592d1bf640 1 --2- 172.21.15.66:0/3560871343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59280b2f20 0x7f592815f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467642671 0 0) 0x7f59280b35e0 con 0x7f5928144cb0 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5918003520 con 0x7f5928144cb0 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905948126 0 0) 0x7f59280b38c0 con 0x7f5928072d50 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59280b35e0 con 0x7f5928072d50 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 551877998 0 0) 0x7f5918003520 con 0x7f5928144cb0 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592d1bf640 1 -- 172.21.15.66:0/3560871343 >> v1:172.21.15.117:6789/0 conn(0x7f5928072d50 legacy=0x7f59280ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592d1bf640 1 -- 172.21.15.66:0/3560871343 >> v1:172.21.15.66:6789/0 conn(0x7f5928144cb0 legacy=0x7f5928072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.903+0000 7f592d1bf640 1 -- 172.21.15.66:0/3560871343 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59280b3ba0 con 0x7f59280b2f20 2024-03-30T13:12:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.904+0000 7f592d1bf640 1 --2- 172.21.15.66:0/3560871343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59280b2f20 0x7f592815f800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59280b4000 tx=0x7f592002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee39cc7c70807ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.904+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920039070 con 0x7f59280b2f20 2024-03-30T13:12:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592002c910 con 0x7f59280b2f20 2024-03-30T13:12:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f59277fe640 1 -- 172.21.15.66:0/3560871343 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c001670 con 0x7f59280b2f20 2024-03-30T13:12:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 -- 172.21.15.66:0/3560871343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59280b2f20 msgr2=0x7f592815f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 --2- 172.21.15.66:0/3560871343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59280b2f20 0x7f592815f800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59280b4000 tx=0x7f592002bde0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 -- 172.21.15.66:0/3560871343 shutdown_connections 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 --2- 172.21.15.66:0/3560871343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59280b2f20 0x7f592815f800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 -- 172.21.15.66:0/3560871343 >> 172.21.15.66:0/3560871343 conn(0x7f59280b5b00 msgr2=0x7f592806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 -- 172.21.15.66:0/3560871343 shutdown_connections 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.905+0000 7f592ec22640 1 -- 172.21.15.66:0/3560871343 wait complete. 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.906+0000 7f592ec22640 1 Processor -- start 2024-03-30T13:12:42.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.906+0000 7f592ec22640 1 -- start start 2024-03-30T13:12:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592ec22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5928072d50 0x7f592813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592ec22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59280b2f20 0x7f5928141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592ec22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 0x7f5928141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592ec22640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59280ce8e0 con 0x7f5928144cb0 2024-03-30T13:12:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592ec22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59280ce760 con 0x7f59280b2f20 2024-03-30T13:12:42.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592ec22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f59280cea60 con 0x7f5928072d50 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 0x7f5928141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 0x7f5928141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36120/0 (socket says 172.21.15.66:36120) 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 -- 172.21.15.66:0/2337527037 learned_addr learned my addr 172.21.15.66:0/2337527037 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592c9be640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5928072d50 0x7f592813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f5927fff640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59280b2f20 0x7f5928141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 -- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5928072d50 msgr2=0x7f592813da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5928072d50 0x7f592813da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 -- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59280b2f20 msgr2=0x7f5928141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59280b2f20 0x7f5928141300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f592d1bf640 1 -- 172.21.15.66:0/2337527037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592813dfc0 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.908+0000 7f5927fff640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59280b2f20 0x7f5928141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.909+0000 7f592d1bf640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 0x7f5928141840 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d90 tx=0x7f5920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.909+0000 7f59257fa640 1 -- 172.21.15.66:0/2337527037 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920039050 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.909+0000 7f59257fa640 1 -- 172.21.15.66:0/2337527037 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592003a040 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.909+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5928158eb0 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.909+0000 7f59257fa640 1 -- 172.21.15.66:0/2337527037 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920009450 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.909+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5928159470 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.910+0000 7f59257fa640 1 -- 172.21.15.66:0/2337527037 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5920047020 con 0x7f5928144cb0 2024-03-30T13:12:42.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.911+0000 7f59257fa640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f590c06c530 0x7f590c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.911+0000 7f59257fa640 1 -- 172.21.15.66:0/2337527037 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f59200ad710 con 0x7f5928144cb0 2024-03-30T13:12:42.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.911+0000 7f592c9be640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f590c06c530 0x7f590c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.911+0000 7f592c9be640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f590c06c530 0x7f590c06e9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f591c0011e0 tx=0x7f591c001710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 Processor -- start 2024-03-30T13:12:42.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 -- start start 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475dff510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 0x55d475e2f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d475d69590 con 0x55d475dffa50 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d475d07d30 con 0x55d475e2c570 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.912+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d475d65540 con 0x55d475e2c950 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 0x55d475e2f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 0x55d475e2f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36136/0 (socket says 172.21.15.66:36136) 2024-03-30T13:12:42.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 -- 172.21.15.66:0/3300145616 learned_addr learned my addr 172.21.15.66:0/3300145616 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592c9be640 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475dff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f5927fff640 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 -- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 msgr2=0x55d475e2f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 -- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 msgr2=0x55d475dff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475dff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 -- 172.21.15.66:0/3300145616 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d475b49920 con 0x55d475dffa50 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f592d1bf640 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 0x55d475e2f840 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f59200ab9c0 tx=0x7f592007ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2964bf9a852e6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f59127fc640 1 -- 172.21.15.66:0/3300145616 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592007f7f0 con 0x55d475dffa50 2024-03-30T13:12:42.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f59127fc640 1 -- 172.21.15.66:0/3300145616 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920084020 con 0x55d475dffa50 2024-03-30T13:12:42.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.913+0000 7f59127fc640 1 -- 172.21.15.66:0/3300145616 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920077ab0 con 0x55d475dffa50 2024-03-30T13:12:42.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 -- 172.21.15.66:0/3300145616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 msgr2=0x55d475e2f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 0x55d475e2f840 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f59200ab9c0 tx=0x7f592007ada0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 -- 172.21.15.66:0/3300145616 shutdown_connections 2024-03-30T13:12:42.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475dffa50 0x55d475e2f840 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 --2- 172.21.15.66:0/3300145616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475dff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 -- 172.21.15.66:0/3300145616 >> 172.21.15.66:0/3300145616 conn(0x55d475e01780 msgr2=0x55d475e4a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 -- 172.21.15.66:0/3300145616 shutdown_connections 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.914+0000 7f593180f740 1 -- 172.21.15.66:0/3300145616 wait complete. 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.915+0000 7f593180f740 1 -- start start 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.915+0000 7f593180f740 1 Processor -- start 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.915+0000 7f593180f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 0x55d475e49070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.915+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.915+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475e2e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d475e03e20 con 0x55d475e2c570 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d475d059f0 con 0x55d475e4aa90 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d475e06660 con 0x55d475e2c950 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 10 client.0 _ll_register_callbacks cb 0x55d475d09c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 0x55d475e49070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 0x55d475e49070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36140/0 (socket says 172.21.15.66:36140) 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 -- 172.21.15.66:0/384333316 learned_addr learned my addr 172.21.15.66:0/384333316 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f5927fff640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592d1bf640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475e2e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 -- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 msgr2=0x55d475e2e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 -- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 msgr2=0x55d475e2e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475e2e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 -- 172.21.15.66:0/384333316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d475a3d210 con 0x55d475e2c570 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f592c9be640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 0x55d475e49070 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f591c00fe70 tx=0x7f591c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2537b0a441bbb3d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f59077fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f59077fe640 1 -- 172.21.15.66:0/384333316 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c013600 con 0x55d475e2c570 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f59077fe640 1 -- 172.21.15.66:0/384333316 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f591c010040 con 0x55d475e2c570 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5920084020 con 0x55d475e2c570 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f59077fe640 1 -- 172.21.15.66:0/384333316 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f591c0137a0 con 0x55d475e2c570 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 10 client.6432 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.916+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d475baa0b0 con 0x55d475e2c570 2024-03-30T13:12:42.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f59077fe640 1 -- 172.21.15.66:0/384333316 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920084020 con 0x55d475e2c570 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f59077fe640 1 -- 172.21.15.66:0/384333316 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f591c013940 con 0x55d475e2c570 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f593180f740 10 client.6432 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f593180f740 10 client.6432 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f593180f740 20 client.6432 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f593180f740 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475e5e2c0 0x55d475e5e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f593180f740 4 client.6432 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d475d76ad0 con 0x55d475e5e2c0 2024-03-30T13:12:42.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f5927fff640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475e5e2c0 0x55d475e5e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f59077fe640 10 client.6432 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.917+0000 7f5927fff640 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475e5e2c0 0x55d475e5e6a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6a2e36965a5062 server_cookie=338163c919776b3b in_seq=0 out_seq=0 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f59077fe640 1 -- 172.21.15.66:0/384333316 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d475d76ad0 con 0x55d475e5e2c0 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f59077fe640 10 client.6432 handle_command_reply: tid=0 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 client.6432 shutdown 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 msgr2=0x55d475e49070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 0x55d475e49070 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f591c00fe70 tx=0x7f591c0017a0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 shutdown_connections 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475e5e2c0 0x55d475e5e6a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475e2e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c950 0x55d475e2e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f59077fe640 0 client.6432 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 --2- 172.21.15.66:0/384333316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c570 0x55d475e49070 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 >> 172.21.15.66:0/384333316 conn(0x55d475dffa50 msgr2=0x55d475e4d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 shutdown_connections 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 1 -- 172.21.15.66:0/384333316 wait complete. 2024-03-30T13:12:42.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.918+0000 7f593180f740 20 client.6432 trim_cache size 0 max 16384 2024-03-30T13:12:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.940+0000 7f593180f740 1 Processor -- start 2024-03-30T13:12:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.940+0000 7f593180f740 1 -- start start 2024-03-30T13:12:42.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.941+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475e48360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.941+0000 7f593180f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475ebd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.941+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e4aa90 0x55d475e32aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.941+0000 7f593180f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59281ff020 con 0x55d475e2c950 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.941+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59281fe080 con 0x55d475e2c570 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.941+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f592812fe40 con 0x55d475e4aa90 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475ebd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475ebd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36142/0 (socket says 172.21.15.66:36142) 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 -- 172.21.15.66:0/2815050843 learned_addr learned my addr 172.21.15.66:0/2815050843 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f592c9be640 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475e48360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f592d1bf640 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e4aa90 0x55d475e32aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 -- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e4aa90 msgr2=0x55d475e32aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e4aa90 0x55d475e32aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 -- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 msgr2=0x55d475e48360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475e48360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 -- 172.21.15.66:0/2815050843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d475e5ea10 con 0x55d475e2c950 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5927fff640 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475ebd5d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f59280b39b0 tx=0x7f5914004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192dbaef2de0f336 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5911ffb640 1 -- 172.21.15.66:0/2815050843 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591401a1c0 con 0x55d475e2c950 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5911ffb640 1 -- 172.21.15.66:0/2815050843 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f591401e030 con 0x55d475e2c950 2024-03-30T13:12:42.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.942+0000 7f5911ffb640 1 -- 172.21.15.66:0/2815050843 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591401a580 con 0x55d475e2c950 2024-03-30T13:12:42.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 -- 172.21.15.66:0/2815050843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 msgr2=0x55d475ebd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.949 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475ebd5d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f59280b39b0 tx=0x7f5914004ab0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 -- 172.21.15.66:0/2815050843 shutdown_connections 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e4aa90 0x55d475e32aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475ebd5d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 --2- 172.21.15.66:0/2815050843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e2c570 0x55d475e48360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 -- 172.21.15.66:0/2815050843 >> 172.21.15.66:0/2815050843 conn(0x55d475dffa50 msgr2=0x55d475e5ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 -- 172.21.15.66:0/2815050843 shutdown_connections 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.943+0000 7f593180f740 1 -- 172.21.15.66:0/2815050843 wait complete. 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.944+0000 7f593180f740 1 -- start start 2024-03-30T13:12:42.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.944+0000 7f593180f740 1 Processor -- start 2024-03-30T13:12:42.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c570 0x55d475ebcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475e4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475eb63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d475e2cfa0 con 0x55d475e2c950 2024-03-30T13:12:42.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59281fda60 con 0x55d475e4aa90 2024-03-30T13:12:42.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d475eba810 con 0x55d475e2c570 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f593180f740 10 client.0 _ll_register_callbacks cb 0x7f591c0058d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f5927fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475e4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f5927fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475e4dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36144/0 (socket says 172.21.15.66:36144) 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f5927fff640 1 -- 172.21.15.66:0/4147172415 learned_addr learned my addr 172.21.15.66:0/4147172415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f592c9be640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c570 0x55d475ebcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f592d1bf640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475eb63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f5927fff640 1 -- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c570 msgr2=0x55d475ebcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.945+0000 7f5927fff640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c570 0x55d475ebcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f5927fff640 1 -- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 msgr2=0x55d475eb63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:42.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f5927fff640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475eb63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f5927fff640 1 -- 172.21.15.66:0/4147172415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d475a162d0 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f5927fff640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475e4dc10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f59140054c0 tx=0x7f5914000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440dc59038151807 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f59077fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f59077fe640 1 -- 172.21.15.66:0/4147172415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591401a1c0 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f59077fe640 1 -- 172.21.15.66:0/4147172415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f591402ecc0 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591401e030 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f59077fe640 1 -- 172.21.15.66:0/4147172415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f591401a360 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 10 client.6438 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d475eca300 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f59077fe640 1 -- 172.21.15.66:0/4147172415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f591401e030 con 0x55d475e2c950 2024-03-30T13:12:42.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f59077fe640 1 -- 172.21.15.66:0/4147172415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f5914035050 con 0x55d475e2c950 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 10 client.6438 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 10 client.6438 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 20 client.6438 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475eb6900 0x55d475dfe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 4 client.6438 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628419}] 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.946+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628419}) -- 0x55d475d70e20 con 0x55d475eb6900 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.947+0000 7f592c9be640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475eb6900 0x55d475dfe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.947+0000 7f59077fe640 10 client.6438 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.947+0000 7f592c9be640 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475eb6900 0x55d475dfe3a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f14dd334967e61 server_cookie=940a41970a899e0b in_seq=0 out_seq=0 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.947+0000 7f59077fe640 1 -- 172.21.15.66:0/4147172415 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3988 (crc 0 0 0) 0x55d475d70e20 con 0x55d475eb6900 2024-03-30T13:12:42.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.947+0000 7f59077fe640 10 client.6438 handle_command_reply: tid=0 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.947+0000 7f593180f740 1 client.6438 shutdown 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 msgr2=0x55d475e4dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475e4dc10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f59140054c0 tx=0x7f5914000bc0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 shutdown_connections 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d475eb6900 0x55d475dfe3a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d475e4aa90 0x55d475eb63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d475e2c950 0x55d475e4dc10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f59077fe640 0 client.6438 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 --2- 172.21.15.66:0/4147172415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d475e2c570 0x55d475ebcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 >> 172.21.15.66:0/4147172415 conn(0x55d475dffa50 msgr2=0x55d475e49590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 shutdown_connections 2024-03-30T13:12:42.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 1 -- 172.21.15.66:0/4147172415 wait complete. 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f593180f740 20 client.6438 trim_cache size 0 max 16384 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debug.h", 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628419, 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.077926+0000", 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.647818+0000", 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:42.960 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:42.961 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:42.962 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 2579, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.647818+0000", 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 3, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:42.963 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.964 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2579, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.077926+0000" 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2579, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:42.965 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.077926+0000" 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 176, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 28, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:42.966 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.120702+0000", 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 60, 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.967 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:42.968 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:42.969 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:42.970 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:42.971 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.972 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:42.973 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:42.974 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:42.975 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f590c06c530 msgr2=0x7f590c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f590c06c530 0x7f590c06e9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f591c0011e0 tx=0x7f591c001710 comp rx=0 tx=0).stop 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 msgr2=0x7f5928141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 0x7f5928141840 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d90 tx=0x7f5920004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 shutdown_connections 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f590c06c530 0x7f590c06e9f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5928144cb0 0x7f5928141840 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59280b2f20 0x7f5928141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 --2- 172.21.15.66:0/2337527037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5928072d50 0x7f592813da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.948+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 >> 172.21.15.66:0/2337527037 conn(0x7f59280b5b00 msgr2=0x7f59280ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:42.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.949+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 shutdown_connections 2024-03-30T13:12:42.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:42.949+0000 7f592ec22640 1 -- 172.21.15.66:0/2337527037 wait complete. 2024-03-30T13:12:42.978 DEBUG:tasks.cephfs.test_uninlining:getting inode info #16 of 23:1099511628426 2024-03-30T13:12:42.978 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628426 2024-03-30T13:12:42.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628426 2024-03-30T13:12:43.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.087+0000 7f4a2fb90640 1 Processor -- start 2024-03-30T13:12:43.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.087+0000 7f4a2fb90640 1 -- start start 2024-03-30T13:12:43.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2fb90640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4a280ba090 0x7f4a280d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2fb90640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a280d2ff0 con 0x7f4a280b5dc0 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2fb90640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a280d32d0 con 0x7f4a280b73d0 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2fb90640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4a280d35b0 con 0x7f4a280ba090 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2d12b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4a280ba090 0x7f4a280d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2d92c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f4a280b5dc0 0x7f4a280b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37774/0 (socket says 172.21.15.66:37774) 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1399092355 learned_addr learned my addr 172.21.15.66:0/1399092355 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2d12b640 1 -- 172.21.15.66:0/1399092355 >> v1:172.21.15.117:6789/0 conn(0x7f4a280b73d0 legacy=0x7f4a280b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2d12b640 1 -- 172.21.15.66:0/1399092355 >> v1:172.21.15.66:6789/0 conn(0x7f4a280b5dc0 legacy=0x7f4a280b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2c92a640 1 -- 172.21.15.66:0/1399092355 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657702336 0 0) 0x7f4a280d2ff0 con 0x7f4a280b5dc0 2024-03-30T13:12:43.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.088+0000 7f4a2d12b640 1 -- 172.21.15.66:0/1399092355 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280d3890 con 0x7f4a280ba090 2024-03-30T13:12:43.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.089+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/1399092355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280ba090 0x7f4a280d2680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4a180097d0 tx=0x7f4a1802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c95e635f8228c95 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.089+0000 7f4a2c92a640 1 -- 172.21.15.66:0/1399092355 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18034070 con 0x7f4a280ba090 2024-03-30T13:12:43.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.089+0000 7f4a2c92a640 1 -- 172.21.15.66:0/1399092355 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a18030610 con 0x7f4a280ba090 2024-03-30T13:12:43.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.089+0000 7f4a2c92a640 1 -- 172.21.15.66:0/1399092355 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a180309e0 con 0x7f4a280ba090 2024-03-30T13:12:43.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1399092355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280ba090 msgr2=0x7f4a280d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1399092355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280ba090 0x7f4a280d2680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4a180097d0 tx=0x7f4a1802b960 comp rx=0 tx=0).stop 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1399092355 shutdown_connections 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1399092355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280ba090 0x7f4a280d2680 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1399092355 >> 172.21.15.66:0/1399092355 conn(0x7f4a280678f0 msgr2=0x7f4a280d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1399092355 shutdown_connections 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.090+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1399092355 wait complete. 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.091+0000 7f4a2fb90640 1 Processor -- start 2024-03-30T13:12:43.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.091+0000 7f4a2fb90640 1 -- start start 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2fb90640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 0x7f4a28155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2fb90640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a280b73d0 0x7f4a28160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2fb90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a280ba090 0x7f4a28161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2fb90640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2815e880 con 0x7f4a280b5dc0 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2fb90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a280d4090 con 0x7f4a280ba090 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2fb90640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2815ea00 con 0x7f4a280b73d0 2024-03-30T13:12:43.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 0x7f4a28155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 0x7f4a28155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36170/0 (socket says 172.21.15.66:36170) 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1889645835 learned_addr learned my addr 172.21.15.66:0/1889645835 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2e12d640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a280ba090 0x7f4a28161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a280b73d0 msgr2=0x7f4a28160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a280b73d0 0x7f4a28160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a280ba090 msgr2=0x7f4a28161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a280ba090 0x7f4a28161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1889645835 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280cc190 con 0x7f4a280b5dc0 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.092+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 0x7f4a28155a80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400c4d0 tx=0x7f4a2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.093+0000 7f4a1e7fc640 1 -- 172.21.15.66:0/1889645835 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24018070 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.093+0000 7f4a1e7fc640 1 -- 172.21.15.66:0/1889645835 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a240074e0 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.093+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a280cc470 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.093+0000 7f4a1e7fc640 1 -- 172.21.15.66:0/1889645835 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24004a20 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.093+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a280cca30 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.094+0000 7f4a1e7fc640 1 -- 172.21.15.66:0/1889645835 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4a2401b070 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.095+0000 7f4a1e7fc640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a0c06c530 0x7f4a0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.095+0000 7f4a1e7fc640 1 -- 172.21.15.66:0/1889645835 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f4a2408ac40 con 0x7f4a280b5dc0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.095+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a0c06c530 0x7f4a0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.095+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a0c06c530 0x7f4a0c06e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4a18036000 tx=0x7f4a18030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 Processor -- start 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 -- start start 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda5723570 0x55eda56f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda5726300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 0x55eda5726840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eda5660590 con 0x55eda56f6a50 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eda55fed30 con 0x55eda5723950 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.096+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55eda565c540 con 0x55eda5723570 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 0x55eda5726840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 0x55eda5726840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36176/0 (socket says 172.21.15.66:36176) 2024-03-30T13:12:43.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 -- 172.21.15.66:0/3620214569 learned_addr learned my addr 172.21.15.66:0/3620214569 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda5723570 0x55eda56f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda5726300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 -- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda5723570 msgr2=0x55eda56f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda5723570 0x55eda56f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 -- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 msgr2=0x55eda5726300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda5726300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 -- 172.21.15.66:0/3620214569 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda5440920 con 0x55eda56f6a50 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a2e12d640 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 0x55eda5726840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4a20001dc0 tx=0x7f4a2000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50df95d3ca8588f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a0b7fe640 1 -- 172.21.15.66:0/3620214569 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20016020 con 0x55eda56f6a50 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a0b7fe640 1 -- 172.21.15.66:0/3620214569 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2000ddd0 con 0x55eda56f6a50 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.097+0000 7f4a0b7fe640 1 -- 172.21.15.66:0/3620214569 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20012610 con 0x55eda56f6a50 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 -- 172.21.15.66:0/3620214569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 msgr2=0x55eda5726840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 0x55eda5726840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4a20001dc0 tx=0x7f4a2000d520 comp rx=0 tx=0).stop 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 -- 172.21.15.66:0/3620214569 shutdown_connections 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda56f6a50 0x55eda5726840 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda5726300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 --2- 172.21.15.66:0/3620214569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda5723570 0x55eda56f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 -- 172.21.15.66:0/3620214569 >> 172.21.15.66:0/3620214569 conn(0x55eda56f8780 msgr2=0x55eda5741300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 -- 172.21.15.66:0/3620214569 shutdown_connections 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 -- 172.21.15.66:0/3620214569 wait complete. 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.098+0000 7f4a3277f740 1 -- start start 2024-03-30T13:12:43.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.099+0000 7f4a3277f740 1 Processor -- start 2024-03-30T13:12:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda5722c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda572e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda572ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eda56fae20 con 0x55eda5723950 2024-03-30T13:12:43.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eda55fc9f0 con 0x55eda5723570 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55eda56fd660 con 0x55eda572bd90 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda572e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 10 client.0 _ll_register_callbacks cb 0x55eda5600c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda572e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36192/0 (socket says 172.21.15.66:36192) 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 -- 172.21.15.66:0/672565486 learned_addr learned my addr 172.21.15.66:0/672565486 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda5722c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2e12d640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda572ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 -- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 msgr2=0x55eda572ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda572ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 -- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 msgr2=0x55eda5722c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda5722c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 -- 172.21.15.66:0/672565486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda5334210 con 0x55eda5723950 2024-03-30T13:12:43.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda572e500 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098f0 tx=0x7f4a18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fab1a78560c882 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a08ff9640 1 -- 172.21.15.66:0/672565486 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18034040 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a08ff9640 1 -- 172.21.15.66:0/672565486 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f4a1803f030 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.100+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2000ddd0 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a08ff9640 1 -- 172.21.15.66:0/672565486 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4a180469b0 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 10 client.6453 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55eda54a10b0 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a08ff9640 1 -- 172.21.15.66:0/672565486 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2000ddd0 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a08ff9640 1 -- 172.21.15.66:0/672565486 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f4a1802f050 con 0x55eda5723950 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 10 client.6453 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 10 client.6453 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 20 client.6453 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda5729e00 0x55eda572c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 4 client.6453 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eda566dad0 con 0x55eda5729e00 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda5729e00 0x55eda572c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a08ff9640 10 client.6453 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.101+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda5729e00 0x55eda572c6b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60a64ee70ab06f4 server_cookie=261309bcbf7cf057 in_seq=0 out_seq=0 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a08ff9640 1 -- 172.21.15.66:0/672565486 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55eda566dad0 con 0x55eda5729e00 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a08ff9640 10 client.6453 handle_command_reply: tid=0 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 client.6453 shutdown 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 msgr2=0x55eda572e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda572e500 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098f0 tx=0x7f4a18004730 comp rx=0 tx=0).stop 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 shutdown_connections 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda5729e00 0x55eda572c6b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda572ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda572e500 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a08ff9640 0 client.6453 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 --2- 172.21.15.66:0/672565486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda5722c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 >> 172.21.15.66:0/672565486 conn(0x55eda56f6a50 msgr2=0x55eda56fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 shutdown_connections 2024-03-30T13:12:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.102+0000 7f4a3277f740 1 -- 172.21.15.66:0/672565486 wait complete. 2024-03-30T13:12:43.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.103+0000 7f4a3277f740 20 client.6453 trim_cache size 0 max 16384 2024-03-30T13:12:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.117+0000 7f4a3277f740 1 Processor -- start 2024-03-30T13:12:43.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.117+0000 7f4a3277f740 1 -- start start 2024-03-30T13:12:43.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda572d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda5754700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda5754c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a3277f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28201320 con 0x55eda5723950 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28200360 con 0x55eda5723570 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4a281ff7b0 con 0x55eda572bd90 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda5754700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda5754700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36198/0 (socket says 172.21.15.66:36198) 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 -- 172.21.15.66:0/4130512828 learned_addr learned my addr 172.21.15.66:0/4130512828 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.124 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 -- 172.21.15.66:0/4130512828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 msgr2=0x55eda5754c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda5754c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 -- 172.21.15.66:0/4130512828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 msgr2=0x55eda572d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda572d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 -- 172.21.15.66:0/4130512828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda530d2d0 con 0x55eda5723950 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda5754700 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098c0 tx=0x7f4a1804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f821edb716772c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a0affd640 1 -- 172.21.15.66:0/4130512828 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18046c10 con 0x55eda5723950 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a0affd640 1 -- 172.21.15.66:0/4130512828 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a18004030 con 0x55eda5723950 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.118+0000 7f4a0affd640 1 -- 172.21.15.66:0/4130512828 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a180043e0 con 0x55eda5723950 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 -- 172.21.15.66:0/4130512828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 msgr2=0x55eda5754700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda5754700 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098c0 tx=0x7f4a1804b000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.125 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 -- 172.21.15.66:0/4130512828 shutdown_connections 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda5754c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723950 0x55eda5754700 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 --2- 172.21.15.66:0/4130512828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723570 0x55eda572d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 -- 172.21.15.66:0/4130512828 >> 172.21.15.66:0/4130512828 conn(0x55eda56f6a50 msgr2=0x55eda56fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 -- 172.21.15.66:0/4130512828 shutdown_connections 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 -- 172.21.15.66:0/4130512828 wait complete. 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 -- start start 2024-03-30T13:12:43.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.119+0000 7f4a3277f740 1 Processor -- start 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 0x55eda57a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda57bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda5750180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eda56f6e30 con 0x55eda5723570 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a281ffd40 con 0x55eda5723950 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55eda5765800 con 0x55eda572bd90 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a3277f740 10 client.0 _ll_register_callbacks cb 0x7f4a18009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a2d92c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 0x55eda57a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a2d92c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 0x55eda57a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36214/0 (socket says 172.21.15.66:36214) 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.120+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1547148634 learned_addr learned my addr 172.21.15.66:0/1547148634 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 msgr2=0x55eda5750180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda5750180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 msgr2=0x55eda57bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda57bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda57bcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d92c640 1 -- 172.21.15.66:0/1547148634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda57bd500 con 0x55eda5723570 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a2d92c640 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 0x55eda57a6250 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4a24054dd0 tx=0x7f4a2405cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f18df1b575c440f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a08ff9640 1 -- 172.21.15.66:0/1547148634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24058bd0 con 0x55eda5723570 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a08ff9640 1 -- 172.21.15.66:0/1547148634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f4a24058d70 con 0x55eda5723570 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a18004030 con 0x55eda5723570 2024-03-30T13:12:43.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a08ff9640 1 -- 172.21.15.66:0/1547148634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4a240610c0 con 0x55eda5723570 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 10 client.6459 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55eda57bd800 con 0x55eda5723570 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a08ff9640 1 -- 172.21.15.66:0/1547148634 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2407b070 con 0x55eda5723570 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a08ff9640 1 -- 172.21.15.66:0/1547148634 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f4a24004bc0 con 0x55eda5723570 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 10 client.6459 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 10 client.6459 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 20 client.6459 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda57506c0 0x55eda5754700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 4 client.6459 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628426}] 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.121+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628426}) -- 0x55eda572a550 con 0x55eda57506c0 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda57506c0 0x55eda5754700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a2d12b640 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda57506c0 0x55eda5754700 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdba2ca39caca9ce server_cookie=f3e9fd8a3e64f5de in_seq=0 out_seq=0 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a08ff9640 10 client.6459 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.133 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a08ff9640 1 -- 172.21.15.66:0/1547148634 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3982 (crc 0 0 0) 0x55eda572a550 con 0x55eda57506c0 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a08ff9640 10 client.6459 handle_command_reply: tid=0 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 client.6459 shutdown 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 msgr2=0x55eda57a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 0x55eda57a6250 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4a24054dd0 tx=0x7f4a2405cde0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 shutdown_connections 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55eda57506c0 0x55eda5754700 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eda572bd90 0x55eda5750180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eda5723950 0x55eda57bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 --2- 172.21.15.66:0/1547148634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eda5723570 0x55eda57a6250 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a08ff9640 0 client.6459 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.122+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 >> 172.21.15.66:0/1547148634 conn(0x55eda56f6a50 msgr2=0x55eda5750cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 shutdown_connections 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a3277f740 1 -- 172.21.15.66:0/1547148634 wait complete. 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a3277f740 20 client.6459 trim_cache size 0 max 16384 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-sysfs.h", 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628426, 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.080990+0000", 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.655818+0000", 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 77, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.655818+0000", 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 77, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.080990+0000" 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 77, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.080990+0000" 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 192, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 42, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.122418+0000", 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 62, 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:43.148 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:43.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a0c06c530 msgr2=0x7f4a0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a0c06c530 0x7f4a0c06e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4a18036000 tx=0x7f4a18030000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 msgr2=0x7f4a28155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 0x7f4a28155a80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400c4d0 tx=0x7f4a2400c990 comp rx=0 tx=0).stop 2024-03-30T13:12:43.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 shutdown_connections 2024-03-30T13:12:43.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4a0c06c530 0x7f4a0c06e9f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a280ba090 0x7f4a28161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a280b73d0 0x7f4a28160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 --2- 172.21.15.66:0/1889645835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a280b5dc0 0x7f4a28155a80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 >> 172.21.15.66:0/1889645835 conn(0x7f4a280678f0 msgr2=0x7f4a280b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 shutdown_connections 2024-03-30T13:12:43.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.123+0000 7f4a2fb90640 1 -- 172.21.15.66:0/1889645835 wait complete. 2024-03-30T13:12:43.150 DEBUG:tasks.cephfs.test_uninlining:getting inode info #17 of 23:1099511628409 2024-03-30T13:12:43.150 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628409 2024-03-30T13:12:43.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628409 2024-03-30T13:12:43.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.257+0000 7fb3f0ae4640 1 Processor -- start 2024-03-30T13:12:43.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.257+0000 7fb3f0ae4640 1 -- start start 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.258+0000 7fb3f0ae4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb3ec160880 0x7fb3ec0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.258+0000 7fb3f0ae4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec0cc970 con 0x7fb3ec15ceb0 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.258+0000 7fb3f0ae4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec0ccc50 con 0x7fb3ec145d70 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.258+0000 7fb3f0ae4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb3ec146150 con 0x7fb3ec160880 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3ead9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb3ec15ceb0 0x7fb3ec0ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37780/0 (socket says 172.21.15.66:37780) 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3ead9d640 1 -- 172.21.15.66:0/711394965 learned_addr learned my addr 172.21.15.66:0/711394965 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/711394965 >> v2:172.21.15.66:3300/0 conn(0x7fb3ec160880 0x7fb3ec0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/711394965 >> v1:172.21.15.117:6789/0 conn(0x7fb3ec145d70 legacy=0x7fb3ec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/711394965 >> v1:172.21.15.66:6789/0 conn(0x7fb3ec15ceb0 legacy=0x7fb3ec0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3e959a640 1 -- 172.21.15.66:0/711394965 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2672226740 0 0) 0x7fb3ec0cc970 con 0x7fb3ec15ceb0 2024-03-30T13:12:43.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.259+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/711394965 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec0d0460 con 0x7fb3ec160880 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.260+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/711394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0ce490 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0097a0 tx=0x7fb3dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b4a96842ec3a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.260+0000 7fb3e959a640 1 -- 172.21.15.66:0/711394965 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0315a0 con 0x7fb3ec160880 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.260+0000 7fb3e959a640 1 -- 172.21.15.66:0/711394965 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc037070 con 0x7fb3ec160880 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.260+0000 7fb3e959a640 1 -- 172.21.15.66:0/711394965 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc031960 con 0x7fb3ec160880 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/711394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 msgr2=0x7fb3ec0ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/711394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0ce490 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0097a0 tx=0x7fb3dc02b840 comp rx=0 tx=0).stop 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/711394965 shutdown_connections 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/711394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0ce490 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/711394965 >> 172.21.15.66:0/711394965 conn(0x7fb3ec0b3e70 msgr2=0x7fb3ec0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/711394965 shutdown_connections 2024-03-30T13:12:43.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.261+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/711394965 wait complete. 2024-03-30T13:12:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.263+0000 7fb3f0ae4640 1 Processor -- start 2024-03-30T13:12:43.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.263+0000 7fb3f0ae4640 1 -- start start 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3f0ae4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ec145d70 0x7fb3ec06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3f0ae4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec15ceb0 0x7fb3ec15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3f0ae4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3f0ae4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec0ccf30 con 0x7fb3ec160880 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3f0ae4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec15ca20 con 0x7fb3ec15ceb0 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3f0ae4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ec0cf2a0 con 0x7fb3ec145d70 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3ead9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3ead9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36232/0 (socket says 172.21.15.66:36232) 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.264+0000 7fb3ead9d640 1 -- 172.21.15.66:0/2487376356 learned_addr learned my addr 172.21.15.66:0/2487376356 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec15ceb0 0x7fb3ec15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ec145d70 0x7fb3ec06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ead9d640 1 -- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ec145d70 msgr2=0x7fb3ec06d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ec145d70 0x7fb3ec06d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ead9d640 1 -- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec15ceb0 msgr2=0x7fb3ec15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec15ceb0 0x7fb3ec15e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ead9d640 1 -- 172.21.15.66:0/2487376356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec06de80 con 0x7fb3ec160880 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ec145d70 0x7fb3ec06d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0cb4f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c7a0 tx=0x7fb3e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3daffd640 1 -- 172.21.15.66:0/2487376356 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4018070 con 0x7fb3ec160880 2024-03-30T13:12:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3daffd640 1 -- 172.21.15.66:0/2487376356 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e400a640 con 0x7fb3ec160880 2024-03-30T13:12:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ec1f4e00 con 0x7fb3ec160880 2024-03-30T13:12:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3daffd640 1 -- 172.21.15.66:0/2487376356 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400aab0 con 0x7fb3ec160880 2024-03-30T13:12:43.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.265+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ec15d600 con 0x7fb3ec160880 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.267+0000 7fb3daffd640 1 -- 172.21.15.66:0/2487376356 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb3e401b070 con 0x7fb3ec160880 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.267+0000 7fb3daffd640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb3cc06c530 0x7fb3cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.267+0000 7fb3daffd640 1 -- 172.21.15.66:0/2487376356 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fb3e408ae60 con 0x7fb3ec160880 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.267+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb3cc06c530 0x7fb3cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.268+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb3cc06c530 0x7fb3cc06e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0003eb0 tx=0x7fb3e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.268+0000 7fb3f37ac740 1 Processor -- start 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.268+0000 7fb3f37ac740 1 -- start start 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0ba8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 0x55b9b0bd8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0ba8a50 0x55b9b0bd8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9b0b12590 con 0x55b9b0bd5950 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b9b0ab0d30 con 0x55b9b0ba8a50 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b9b0b0e540 con 0x55b9b0bd5570 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 0x55b9b0bd8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 0x55b9b0bd8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36248/0 (socket says 172.21.15.66:36248) 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/1724092919 learned_addr learned my addr 172.21.15.66:0/1724092919 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0ba8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0ba8a50 0x55b9b0bd8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 msgr2=0x55b9b0ba8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0ba8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0ba8a50 msgr2=0x55b9b0bd8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0ba8a50 0x55b9b0bd8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.269+0000 7fb3e9d9b640 1 -- 172.21.15.66:0/1724092919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b08f2920 con 0x55b9b0bd5950 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.270+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 0x55b9b0bd8300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0097a0 tx=0x7fb3dc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7d43255f9468e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.270+0000 7fb3cbfff640 1 -- 172.21.15.66:0/1724092919 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc04b070 con 0x55b9b0bd5950 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.270+0000 7fb3cbfff640 1 -- 172.21.15.66:0/1724092919 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc037040 con 0x55b9b0bd5950 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.270+0000 7fb3cbfff640 1 -- 172.21.15.66:0/1724092919 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc003850 con 0x55b9b0bd5950 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1724092919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 msgr2=0x55b9b0bd8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 0x55b9b0bd8300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0097a0 tx=0x7fb3dc030000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1724092919 shutdown_connections 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0ba8a50 0x55b9b0bd8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5950 0x55b9b0bd8300 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1724092919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0ba8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1724092919 >> 172.21.15.66:0/1724092919 conn(0x55b9b0baa780 msgr2=0x55b9b0bf3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1724092919 shutdown_connections 2024-03-30T13:12:43.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.271+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1724092919 wait complete. 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.272+0000 7fb3f37ac740 1 -- start start 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.272+0000 7fb3f37ac740 1 Processor -- start 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.273+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0c066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.273+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.273+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 0x55b9b0bdf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.273+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9b0bace20 con 0x55b9b0bdecd0 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.273+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b9b0aae9f0 con 0x55b9b0bd5950 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b9b0baf660 con 0x55b9b0bd5570 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3f37ac740 10 client.0 _ll_register_callbacks cb 0x55b9b0ab2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 0x55b9b0bdf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 0x55b9b0bdf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36256/0 (socket says 172.21.15.66:36256) 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 -- 172.21.15.66:0/4293071292 learned_addr learned my addr 172.21.15.66:0/4293071292 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 -- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 msgr2=0x55b9b0c066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0c066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0c066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 -- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 msgr2=0x55b9b0bdd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 -- 172.21.15.66:0/4293071292 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b07e6210 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 0x55b9b0bdf150 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4001980 tx=0x7fb3e4058dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3384d899df791dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3c97fa640 1 -- 172.21.15.66:0/4293071292 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400ae50 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3c97fa640 1 -- 172.21.15.66:0/4293071292 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fb3e4058090 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3dc037040 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3c97fa640 1 -- 172.21.15.66:0/4293071292 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb3e40582b0 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3f37ac740 10 client.6474 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.274+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55b9b09530b0 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3c97fa640 1 -- 172.21.15.66:0/4293071292 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e4072070 con 0x55b9b0bdecd0 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3c97fa640 1 -- 172.21.15.66:0/4293071292 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fb3e4082b00 con 0x55b9b0bdecd0 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3f37ac740 10 client.6474 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3f37ac740 10 client.6474 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3f37ac740 20 client.6474 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bda580 0x55b9b0bda9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3f37ac740 4 client.6474 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b0b1fad0 con 0x55b9b0bda580 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bda580 0x55b9b0bda9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3c97fa640 10 client.6474 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.275+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bda580 0x55b9b0bda9a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9401932816c464ec server_cookie=6b97bf15c0a6f352 in_seq=0 out_seq=0 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3c97fa640 1 -- 172.21.15.66:0/4293071292 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55b9b0b1fad0 con 0x55b9b0bda580 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3c97fa640 10 client.6474 handle_command_reply: tid=0 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 client.6474 shutdown 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 msgr2=0x55b9b0bdf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 0x55b9b0bdf150 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4001980 tx=0x7fb3e4058dc0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 shutdown_connections 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bda580 0x55b9b0bda9a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bdecd0 0x55b9b0bdf150 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3c97fa640 0 client.6474 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/4293071292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5570 0x55b9b0c066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 >> 172.21.15.66:0/4293071292 conn(0x55b9b0ba8a50 msgr2=0x55b9b0bac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 shutdown_connections 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 1 -- 172.21.15.66:0/4293071292 wait complete. 2024-03-30T13:12:43.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.276+0000 7fb3f37ac740 20 client.6474 trim_cache size 0 max 16384 2024-03-30T13:12:43.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 Processor -- start 2024-03-30T13:12:43.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 -- start start 2024-03-30T13:12:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0ba8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5950 0x55b9b0bdf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bdecd0 0x55b9b0c04d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec201160 con 0x55b9b0bd5570 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec2001c0 con 0x55b9b0bdecd0 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ec1f81e0 con 0x55b9b0bd5950 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.298+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0ba8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0ba8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36258/0 (socket says 172.21.15.66:36258) 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 -- 172.21.15.66:0/2113673216 learned_addr learned my addr 172.21.15.66:0/2113673216 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5950 0x55b9b0bdf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bdecd0 0x55b9b0c04d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 -- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5950 msgr2=0x55b9b0bdf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5950 0x55b9b0bdf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 -- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bdecd0 msgr2=0x55b9b0c04d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bdecd0 0x55b9b0c04d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 -- 172.21.15.66:0/2113673216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b07bf2d0 con 0x55b9b0bd5570 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0ba8e30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000e530 tx=0x7fb3e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb4a38219f72c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3cb7fe640 1 -- 172.21.15.66:0/2113673216 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e002d070 con 0x55b9b0bd5570 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3cb7fe640 1 -- 172.21.15.66:0/2113673216 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e0005030 con 0x55b9b0bd5570 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.299+0000 7fb3cb7fe640 1 -- 172.21.15.66:0/2113673216 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0006950 con 0x55b9b0bd5570 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 -- 172.21.15.66:0/2113673216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 msgr2=0x55b9b0ba8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0ba8e30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000e530 tx=0x7fb3e0014000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 -- 172.21.15.66:0/2113673216 shutdown_connections 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bdecd0 0x55b9b0c04d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bd5950 0x55b9b0bdf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/2113673216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0ba8e30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 -- 172.21.15.66:0/2113673216 >> 172.21.15.66:0/2113673216 conn(0x55b9b0ba8a50 msgr2=0x55b9b0bac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 -- 172.21.15.66:0/2113673216 shutdown_connections 2024-03-30T13:12:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 -- 172.21.15.66:0/2113673216 wait complete. 2024-03-30T13:12:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.300+0000 7fb3f37ac740 1 -- start start 2024-03-30T13:12:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 Processor -- start 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0bdb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bdecd0 0x55b9b0bd3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b9b0bf3b30 con 0x55b9b0bd5570 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1ffba0 con 0x55b9b0bd5950 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.301+0000 7fb3f37ac740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55b9b0bda9f0 con 0x55b9b0bdecd0 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3f37ac740 10 client.0 _ll_register_callbacks cb 0x7fb3e4058b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0bdb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0bdb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36266/0 (socket says 172.21.15.66:36266) 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 -- 172.21.15.66:0/1840522618 learned_addr learned my addr 172.21.15.66:0/1840522618 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ead9d640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bdecd0 0x55b9b0bd3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 -- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bdecd0 msgr2=0x55b9b0bd3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bdecd0 0x55b9b0bd3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 -- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 msgr2=0x55b9b0bdb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 -- 172.21.15.66:0/1840522618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b0ba7440 con 0x55b9b0bd5570 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3ea59c640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0bdb450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00072c0 tx=0x7fb3e00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6869d4425d40e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3c97fa640 1 -- 172.21.15.66:0/1840522618 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e002d040 con 0x55b9b0bd5570 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3c97fa640 1 -- 172.21.15.66:0/1840522618 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fb3e0033cd0 con 0x55b9b0bd5570 2024-03-30T13:12:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.302+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e0005030 con 0x55b9b0bd5570 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3c97fa640 1 -- 172.21.15.66:0/1840522618 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb3e003a050 con 0x55b9b0bd5570 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 10 client.6480 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55b9b0c04e40 con 0x55b9b0bd5570 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3c97fa640 1 -- 172.21.15.66:0/1840522618 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e0005030 con 0x55b9b0bd5570 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3c97fa640 1 -- 172.21.15.66:0/1840522618 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fb3e003d020 con 0x55b9b0bd5570 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 10 client.6480 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 10 client.6480 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 20 client.6480 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bd7f20 0x55b9b0bd3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 4 client.6480 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628409}] 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628409}) -- 0x55b9b0b19e20 con 0x55b9b0bd7f20 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bd7f20 0x55b9b0bd3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3e9d9b640 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bd7f20 0x55b9b0bd3f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c939dd3bc6f2c498 server_cookie=9939a0c4c7ef6f1 in_seq=0 out_seq=0 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.303+0000 7fb3c97fa640 10 client.6480 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3c97fa640 1 -- 172.21.15.66:0/1840522618 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3979 (crc 0 0 0) 0x55b9b0b19e20 con 0x55b9b0bd7f20 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3c97fa640 10 client.6480 handle_command_reply: tid=0 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 client.6480 shutdown 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 msgr2=0x55b9b0bdb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0bdb450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00072c0 tx=0x7fb3e00036f0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 shutdown_connections 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55b9b0bd7f20 0x55b9b0bd3f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9b0bdecd0 0x55b9b0bd3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9b0bd5950 0x55b9b0bdb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3c97fa640 0 client.6480 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 --2- 172.21.15.66:0/1840522618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b9b0bd5570 0x55b9b0bdb450 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 >> 172.21.15.66:0/1840522618 conn(0x55b9b0ba8a50 msgr2=0x55b9b0bf3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 shutdown_connections 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 1 -- 172.21.15.66:0/1840522618 wait complete. 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.304+0000 7fb3f37ac740 20 client.6480 trim_cache size 0 max 16384 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/dcache.c", 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628409, 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.083120+0000", 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.638818+0000", 2024-03-30T13:12:43.316 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:43.317 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 3670, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.638818+0000", 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 3, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:43.318 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3670, 2024-03-30T13:12:43.319 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.083120+0000" 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 3670, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.083120+0000" 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.320 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 240, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 8, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.126742+0000", 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 64, 2024-03-30T13:12:43.321 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:43.322 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:43.323 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:43.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:43.325 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.327 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.328 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.329 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:43.329 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:43.329 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:43.329 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:43.329 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:43.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb3cc06c530 msgr2=0x7fb3cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb3cc06c530 0x7fb3cc06e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0003eb0 tx=0x7fb3e0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:43.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 msgr2=0x7fb3ec0cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0cb4f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c7a0 tx=0x7fb3e400cc60 comp rx=0 tx=0).stop 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 shutdown_connections 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb3cc06c530 0x7fb3cc06e9f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ec160880 0x7fb3ec0cb4f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec15ceb0 0x7fb3ec15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 --2- 172.21.15.66:0/2487376356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3ec145d70 0x7fb3ec06d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 >> 172.21.15.66:0/2487376356 conn(0x7fb3ec0b3e70 msgr2=0x7fb3ec0b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 shutdown_connections 2024-03-30T13:12:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.305+0000 7fb3f0ae4640 1 -- 172.21.15.66:0/2487376356 wait complete. 2024-03-30T13:12:43.331 DEBUG:tasks.cephfs.test_uninlining:getting inode info #18 of 23:1099511628417 2024-03-30T13:12:43.332 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628417 2024-03-30T13:12:43.332 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628417 2024-03-30T13:12:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.438+0000 7f59abf15640 1 Processor -- start 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.438+0000 7f59abf15640 1 -- start start 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59abf15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40ba090 0x7f59a40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59abf15640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f59a40b73d0 0x7f59a40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59abf15640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a40d2ff0 con 0x7f59a40b5dc0 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59abf15640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f59a40d32d0 con 0x7f59a40b73d0 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59abf15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f59a40d35b0 con 0x7f59a40ba090 2024-03-30T13:12:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59aa4b2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f59a40b73d0 0x7f59a40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59a9cb1640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f59a40b5dc0 0x7f59a40b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:33088/0 (socket says 172.21.15.66:33088) 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59a9cb1640 1 -- 172.21.15.66:0/3779666787 learned_addr learned my addr 172.21.15.66:0/3779666787 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3779666787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40ba090 0x7f59a40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59aa4b2640 1 -- 172.21.15.66:0/3779666787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40ba090 msgr2=0x7f59a40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/3779666787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40ba090 0x7f59a40d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59a8caf640 1 -- 172.21.15.66:0/3779666787 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392708393 0 0) 0x7f59a40d2ff0 con 0x7f59a40b5dc0 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59aa4b2640 1 -- 172.21.15.66:0/3779666787 >> v1:172.21.15.117:6789/0 conn(0x7f59a40b5dc0 legacy=0x7f59a40b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.439+0000 7f59aa4b2640 1 -- 172.21.15.66:0/3779666787 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a40d3890 con 0x7f59a40b73d0 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.440+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/3779666787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40b7880 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f599c009d90 tx=0x7f599c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaafb01dbdffc48 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.440+0000 7f59a8caf640 1 -- 172.21.15.66:0/3779666787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c039070 con 0x7f59a40b73d0 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.440+0000 7f59a8caf640 1 -- 172.21.15.66:0/3779666787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c02cc50 con 0x7f59a40b73d0 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.440+0000 7f59a8caf640 1 -- 172.21.15.66:0/3779666787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c034440 con 0x7f59a40b73d0 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 -- 172.21.15.66:0/3779666787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 msgr2=0x7f59a40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 --2- 172.21.15.66:0/3779666787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40b7880 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f599c009d90 tx=0x7f599c02bea0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 -- 172.21.15.66:0/3779666787 shutdown_connections 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 --2- 172.21.15.66:0/3779666787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40b7880 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 --2- 172.21.15.66:0/3779666787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40ba090 0x7f59a40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 -- 172.21.15.66:0/3779666787 >> 172.21.15.66:0/3779666787 conn(0x7f59a40678f0 msgr2=0x7f59a40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 -- 172.21.15.66:0/3779666787 shutdown_connections 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.441+0000 7f59abf15640 1 -- 172.21.15.66:0/3779666787 wait complete. 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.442+0000 7f59abf15640 1 Processor -- start 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.442+0000 7f59abf15640 1 -- start start 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59abf15640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40b5dc0 0x7f59a415f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59abf15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59abf15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a40ba090 0x7f59a41f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59abf15640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59a415e880 con 0x7f59a40b73d0 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59abf15640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59a40d4090 con 0x7f59a40ba090 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59abf15640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f59a415ea00 con 0x7f59a40b5dc0 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36276/0 (socket says 172.21.15.66:36276) 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 -- 172.21.15.66:0/3503032822 learned_addr learned my addr 172.21.15.66:0/3503032822 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a40ba090 0x7f59a41f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40b5dc0 0x7f59a415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 -- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40b5dc0 msgr2=0x7f59a415f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40b5dc0 0x7f59a415f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 -- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a40ba090 msgr2=0x7f59a41f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a40ba090 0x7f59a41f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 -- 172.21.15.66:0/3503032822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a41f4c00 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40b5dc0 0x7f59a415f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.443+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40ccc70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f599400e870 tx=0x7f599400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.444+0000 7f599a7fc640 1 -- 172.21.15.66:0/3503032822 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994017070 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.444+0000 7f599a7fc640 1 -- 172.21.15.66:0/3503032822 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599400c940 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.444+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a41f4ee0 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.444+0000 7f599a7fc640 1 -- 172.21.15.66:0/3503032822 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599400cd10 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.444+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a415fa60 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.445+0000 7f599a7fc640 1 -- 172.21.15.66:0/3503032822 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f599401a070 con 0x7f59a40b73d0 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.445+0000 7f599a7fc640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f598806c530 0x7f598806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.446+0000 7f599a7fc640 1 -- 172.21.15.66:0/3503032822 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f599408baf0 con 0x7f59a40b73d0 2024-03-30T13:12:43.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.446+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f598806c530 0x7f598806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.446+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f598806c530 0x7f598806e9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000f60 tx=0x7f59a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 Processor -- start 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 -- start start 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368570 0x561d7b33b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b36b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b33ba50 0x561d7b36b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d7b2a5590 con 0x561d7b368950 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561d7b243d30 con 0x561d7b33ba50 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561d7b2a1540 con 0x561d7b368570 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.447+0000 7f59a94b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b36b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b36b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36284/0 (socket says 172.21.15.66:36284) 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 -- 172.21.15.66:0/838683521 learned_addr learned my addr 172.21.15.66:0/838683521 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b33ba50 0x561d7b36b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368570 0x561d7b33b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 -- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368570 msgr2=0x561d7b33b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368570 0x561d7b33b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 -- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b33ba50 msgr2=0x561d7b36b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b33ba50 0x561d7b36b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 -- 172.21.15.66:0/838683521 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7b085920 con 0x561d7b368950 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59a94b0640 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b36b300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5994061ed0 tx=0x7f5994059f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5900257e4d3bf3f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59877fe640 1 -- 172.21.15.66:0/838683521 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994021070 con 0x561d7b368950 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59877fe640 1 -- 172.21.15.66:0/838683521 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599405dbd0 con 0x561d7b368950 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.448+0000 7f59877fe640 1 -- 172.21.15.66:0/838683521 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994062400 con 0x561d7b368950 2024-03-30T13:12:43.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 -- 172.21.15.66:0/838683521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 msgr2=0x561d7b36b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b36b300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5994061ed0 tx=0x7f5994059f20 comp rx=0 tx=0).stop 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 -- 172.21.15.66:0/838683521 shutdown_connections 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b33ba50 0x561d7b36b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b36b300 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 --2- 172.21.15.66:0/838683521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368570 0x561d7b33b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 -- 172.21.15.66:0/838683521 >> 172.21.15.66:0/838683521 conn(0x561d7b33d780 msgr2=0x561d7b386300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 -- 172.21.15.66:0/838683521 shutdown_connections 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 -- 172.21.15.66:0/838683521 wait complete. 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.449+0000 7f59aebd9740 1 -- start start 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 Processor -- start 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b397370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368950 0x561d7b36c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 0x561d7b3724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d7b33fe20 con 0x561d7b372020 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561d7b2419f0 con 0x561d7b368570 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561d7b342660 con 0x561d7b368950 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.450+0000 7f59aebd9740 10 client.0 _ll_register_callbacks cb 0x561d7b245c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 0x561d7b3724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 0x561d7b3724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36300/0 (socket says 172.21.15.66:36300) 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 -- 172.21.15.66:0/1799707165 learned_addr learned my addr 172.21.15.66:0/1799707165 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b397370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59a94b0640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368950 0x561d7b36c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 -- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368950 msgr2=0x561d7b36c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368950 0x561d7b36c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 -- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 msgr2=0x561d7b397370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b397370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 -- 172.21.15.66:0/1799707165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7af79210 con 0x561d7b372020 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 0x561d7b3724a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f599c02bee0 tx=0x7f599c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2865d3d148d3da26 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f5984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f5984ff9640 1 -- 172.21.15.66:0/1799707165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c044070 con 0x561d7b372020 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f5984ff9640 1 -- 172.21.15.66:0/1799707165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f599c034650 con 0x561d7b372020 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f599405dbd0 con 0x561d7b372020 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f5984ff9640 1 -- 172.21.15.66:0/1799707165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f599c039040 con 0x561d7b372020 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aebd9740 10 client.6492 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.451+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x561d7b0e60b0 con 0x561d7b372020 2024-03-30T13:12:43.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f5984ff9640 1 -- 172.21.15.66:0/1799707165 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599405dbd0 con 0x561d7b372020 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f5984ff9640 1 -- 172.21.15.66:0/1799707165 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f599c034850 con 0x561d7b372020 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59aebd9740 10 client.6492 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59aebd9740 10 client.6492 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59aebd9740 20 client.6492 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59aebd9740 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b370100 0x561d7b370510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59aebd9740 4 client.6492 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d7b2b2ad0 con 0x561d7b370100 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b370100 0x561d7b370510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b370100 0x561d7b370510 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1560439b706c7507 server_cookie=1fc9ec15842de29f in_seq=0 out_seq=0 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.452+0000 7f5984ff9640 10 client.6492 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f5984ff9640 1 -- 172.21.15.66:0/1799707165 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x561d7b2b2ad0 con 0x561d7b370100 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f5984ff9640 10 client.6492 handle_command_reply: tid=0 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 client.6492 shutdown 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 msgr2=0x561d7b3724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 0x561d7b3724a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f599c02bee0 tx=0x7f599c004ab0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 shutdown_connections 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b370100 0x561d7b370510 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b372020 0x561d7b3724a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b368950 0x561d7b36c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 --2- 172.21.15.66:0/1799707165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b397370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 >> 172.21.15.66:0/1799707165 conn(0x561d7b33ba50 msgr2=0x561d7b33f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f5984ff9640 0 client.6492 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 shutdown_connections 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 1 -- 172.21.15.66:0/1799707165 wait complete. 2024-03-30T13:12:43.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.453+0000 7f59aebd9740 20 client.6492 trim_cache size 0 max 16384 2024-03-30T13:12:43.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.468+0000 7f59aebd9740 1 Processor -- start 2024-03-30T13:12:43.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.468+0000 7f59aebd9740 1 -- start start 2024-03-30T13:12:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b370de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b386530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b3a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aebd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59a42010b0 con 0x561d7b368950 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4200110 con 0x561d7b368570 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f59a4159900 con 0x561d7b372020 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b386530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b386530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36312/0 (socket says 172.21.15.66:36312) 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 -- 172.21.15.66:0/3625081930 learned_addr learned my addr 172.21.15.66:0/3625081930 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b3a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b370de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 -- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 msgr2=0x561d7b3a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b3a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 -- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 msgr2=0x561d7b370de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b370de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 -- 172.21.15.66:0/3625081930 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7af522d0 con 0x561d7b368950 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f59a94b0640 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b386530 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5994089e80 tx=0x7f5994021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20754c29680bb1d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f5986ffd640 1 -- 172.21.15.66:0/3625081930 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994062400 con 0x561d7b368950 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f5986ffd640 1 -- 172.21.15.66:0/3625081930 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5994057b20 con 0x561d7b368950 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.469+0000 7f5986ffd640 1 -- 172.21.15.66:0/3625081930 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599405ab50 con 0x561d7b368950 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 -- 172.21.15.66:0/3625081930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 msgr2=0x561d7b386530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b386530 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5994089e80 tx=0x7f5994021000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 -- 172.21.15.66:0/3625081930 shutdown_connections 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b3a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368950 0x561d7b386530 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 --2- 172.21.15.66:0/3625081930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368570 0x561d7b370de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 -- 172.21.15.66:0/3625081930 >> 172.21.15.66:0/3625081930 conn(0x561d7b33ba50 msgr2=0x561d7b336740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 -- 172.21.15.66:0/3625081930 shutdown_connections 2024-03-30T13:12:43.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 -- 172.21.15.66:0/3625081930 wait complete. 2024-03-30T13:12:43.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.470+0000 7f59aebd9740 1 -- start start 2024-03-30T13:12:43.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 Processor -- start 2024-03-30T13:12:43.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 0x561d7b340400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368950 0x561d7b36e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b36ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d7b342510 con 0x561d7b368570 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41ffac0 con 0x561d7b368950 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561d7b2b3630 con 0x561d7b372020 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59aebd9740 10 client.0 _ll_register_callbacks cb 0x7f599c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.471+0000 7f59a9cb1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 0x561d7b340400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 0x561d7b340400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36320/0 (socket says 172.21.15.66:36320) 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 -- 172.21.15.66:0/884345875 learned_addr learned my addr 172.21.15.66:0/884345875 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a94b0640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368950 0x561d7b36e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b36ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 -- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 msgr2=0x561d7b36ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b36ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 -- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368950 msgr2=0x561d7b36e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368950 0x561d7b36e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 -- 172.21.15.66:0/884345875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7b3a0d50 con 0x561d7b368570 2024-03-30T13:12:43.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59aa4b2640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b36ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59a9cb1640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 0x561d7b340400 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f59a00073b0 tx=0x7f59a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6eda8cebe1825ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f5984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f5984ff9640 1 -- 172.21.15.66:0/884345875 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a000dc50 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f5984ff9640 1 -- 172.21.15.66:0/884345875 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f59a000ddf0 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5994057b20 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f5984ff9640 1 -- 172.21.15.66:0/884345875 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f59a0033050 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59aebd9740 10 client.6498 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.472+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x561d7b3a0a90 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f5984ff9640 1 -- 172.21.15.66:0/884345875 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5994057b20 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f5984ff9640 1 -- 172.21.15.66:0/884345875 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f59a0036020 con 0x561d7b368570 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 10 client.6498 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 10 client.6498 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 20 client.6498 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b3f3e80 0x561d7b366980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 4 client.6498 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628417}] 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628417}) -- 0x561d7b396720 con 0x561d7b3f3e80 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59a94b0640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b3f3e80 0x561d7b366980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f5984ff9640 10 client.6498 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59a94b0640 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b3f3e80 0x561d7b366980 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0c8be61338d782 server_cookie=378c1736be60972d in_seq=0 out_seq=0 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f5984ff9640 1 -- 172.21.15.66:0/884345875 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3986 (crc 0 0 0) 0x561d7b396720 con 0x561d7b3f3e80 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f5984ff9640 10 client.6498 handle_command_reply: tid=0 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.473+0000 7f59aebd9740 1 client.6498 shutdown 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 msgr2=0x561d7b340400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 0x561d7b340400 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f59a00073b0 tx=0x7f59a0000ed0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 shutdown_connections 2024-03-30T13:12:43.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561d7b3f3e80 0x561d7b366980 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561d7b372020 0x561d7b36ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d7b368950 0x561d7b36e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 --2- 172.21.15.66:0/884345875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d7b368570 0x561d7b340400 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f5984ff9640 0 client.6498 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 >> 172.21.15.66:0/884345875 conn(0x561d7b33ba50 msgr2=0x561d7b370de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 shutdown_connections 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 1 -- 172.21.15.66:0/884345875 wait complete. 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59aebd9740 20 client.6498 trim_cache size 0 max 16384 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:43.484 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-bufmap.h", 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628417, 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.090873+0000", 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.645818+0000", 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:43.485 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 925, 2024-03-30T13:12:43.486 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.645818+0000", 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:43.487 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:43.489 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 925, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.090873+0000" 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 925, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.090873+0000" 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 228, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 24, 2024-03-30T13:12:43.491 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.129489+0000", 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 66, 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:43.492 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:43.493 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:43.494 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.495 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.496 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.497 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:43.498 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:43.499 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f598806c530 msgr2=0x7f598806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f598806c530 0x7f598806e9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000f60 tx=0x7f59a0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 msgr2=0x7f59a40ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40ccc70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f599400e870 tx=0x7f599400ed30 comp rx=0 tx=0).stop 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 shutdown_connections 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f598806c530 0x7f598806e9f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a40ba090 0x7f59a41f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a40b73d0 0x7f59a40ccc70 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 --2- 172.21.15.66:0/3503032822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59a40b5dc0 0x7f59a415f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 >> 172.21.15.66:0/3503032822 conn(0x7f59a40678f0 msgr2=0x7f59a40ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.474+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 shutdown_connections 2024-03-30T13:12:43.502 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.475+0000 7f59abf15640 1 -- 172.21.15.66:0/3503032822 wait complete. 2024-03-30T13:12:43.502 DEBUG:tasks.cephfs.test_uninlining:getting inode info #19 of 23:1099511628421 2024-03-30T13:12:43.502 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628421 2024-03-30T13:12:43.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628421 2024-03-30T13:12:43.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.608+0000 7f56d6eb1640 1 Processor -- start 2024-03-30T13:12:43.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.608+0000 7f56d6eb1640 1 -- start start 2024-03-30T13:12:43.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d6eb1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00b5dc0 0x7f56d00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d6eb1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56d00ba090 0x7f56d00d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d6eb1640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d00b66e0 con 0x7f56d006c620 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d6eb1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f56d00d2ff0 con 0x7f56d00ba090 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d6eb1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f56d00d32d0 con 0x7f56d00b5dc0 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00b5dc0 0x7f56d00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56cffff640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56d00ba090 0x7f56d00d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d544e640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f56d006c620 0x7f56d006cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:33102/0 (socket says 172.21.15.66:33102) 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.609+0000 7f56d544e640 1 -- 172.21.15.66:0/2735892387 learned_addr learned my addr 172.21.15.66:0/2735892387 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.610+0000 7f56d4c4d640 1 -- 172.21.15.66:0/2735892387 >> v2:172.21.15.117:3300/0 conn(0x7f56d00ba090 msgr2=0x7f56d00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.610+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/2735892387 >> v2:172.21.15.117:3300/0 conn(0x7f56d00ba090 0x7f56d00d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.610+0000 7f56d4c4d640 1 -- 172.21.15.66:0/2735892387 >> v1:172.21.15.117:6789/0 conn(0x7f56d006c620 legacy=0x7f56d006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.610+0000 7f56cf7fe640 1 -- 172.21.15.66:0/2735892387 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545766034 0 0) 0x7f56d00b66e0 con 0x7f56d006c620 2024-03-30T13:12:43.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.610+0000 7f56d4c4d640 1 -- 172.21.15.66:0/2735892387 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d00d35b0 con 0x7f56d00b5dc0 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.611+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/2735892387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00b5dc0 0x7f56d00b61a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f56c40097a0 tx=0x7f56c402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2559ae49da86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.611+0000 7f56cf7fe640 1 -- 172.21.15.66:0/2735892387 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4035020 con 0x7f56d00b5dc0 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.611+0000 7f56cf7fe640 1 -- 172.21.15.66:0/2735892387 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c4031830 con 0x7f56d00b5dc0 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.611+0000 7f56cf7fe640 1 -- 172.21.15.66:0/2735892387 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4031c00 con 0x7f56d00b5dc0 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 -- 172.21.15.66:0/2735892387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00b5dc0 msgr2=0x7f56d00b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/2735892387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00b5dc0 0x7f56d00b61a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f56c40097a0 tx=0x7f56c402b8d0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 -- 172.21.15.66:0/2735892387 shutdown_connections 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/2735892387 >> v2:172.21.15.117:3300/0 conn(0x7f56d00ba090 0x7f56d00d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/2735892387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00b5dc0 0x7f56d00b61a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 -- 172.21.15.66:0/2735892387 >> 172.21.15.66:0/2735892387 conn(0x7f56d00678f0 msgr2=0x7f56d00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 -- 172.21.15.66:0/2735892387 shutdown_connections 2024-03-30T13:12:43.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.612+0000 7f56d6eb1640 1 -- 172.21.15.66:0/2735892387 wait complete. 2024-03-30T13:12:43.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.614+0000 7f56d6eb1640 1 Processor -- start 2024-03-30T13:12:43.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.614+0000 7f56d6eb1640 1 -- start start 2024-03-30T13:12:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d6eb1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 0x7f56d00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d6eb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56d00b5dc0 0x7f56d00b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d6eb1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00ba090 0x7f56d01f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d6eb1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f56d00d4090 con 0x7f56d006c620 2024-03-30T13:12:43.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d6eb1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56d00d3b70 con 0x7f56d00b5dc0 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d6eb1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f56d00b83e0 con 0x7f56d00ba090 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 0x7f56d00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 0x7f56d00cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36322/0 (socket says 172.21.15.66:36322) 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d4c4d640 1 -- 172.21.15.66:0/321273626 learned_addr learned my addr 172.21.15.66:0/321273626 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56cffff640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56d00b5dc0 0x7f56d00b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.615+0000 7f56d544e640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00ba090 0x7f56d01f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d4c4d640 1 -- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00ba090 msgr2=0x7f56d01f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00ba090 0x7f56d01f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d4c4d640 1 -- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56d00b5dc0 msgr2=0x7f56d00b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56d00b5dc0 0x7f56d00b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d4c4d640 1 -- 172.21.15.66:0/321273626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d00b2cd0 con 0x7f56d006c620 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 0x7f56d00cd560 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000c00 tx=0x7f56c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56cd7fa640 1 -- 172.21.15.66:0/321273626 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004230 con 0x7f56d006c620 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56cd7fa640 1 -- 172.21.15.66:0/321273626 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c40043d0 con 0x7f56d006c620 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d015e880 con 0x7f56d006c620 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56cd7fa640 1 -- 172.21.15.66:0/321273626 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c403a8a0 con 0x7f56d006c620 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.616+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01f4a10 con 0x7f56d006c620 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.618+0000 7f56cd7fa640 1 -- 172.21.15.66:0/321273626 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f56c4038020 con 0x7f56d006c620 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.618+0000 7f56cd7fa640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f56b406c530 0x7f56b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.618+0000 7f56cd7fa640 1 -- 172.21.15.66:0/321273626 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f56c40ad8c0 con 0x7f56d006c620 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.618+0000 7f56cffff640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f56b406c530 0x7f56b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.619+0000 7f56cffff640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f56b406c530 0x7f56b406e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f56d015fc30 tx=0x7f56bc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.619+0000 7f56d9b75740 1 Processor -- start 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.619+0000 7f56d9b75740 1 -- start start 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae8027510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 0x55aae8057300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8027a50 0x55aae8057840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d9b75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55aae7f91590 con 0x55aae8054950 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aae7f2fd30 con 0x55aae8054570 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55aae7f8d540 con 0x55aae8027a50 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 0x55aae8057300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 0x55aae8057300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36338/0 (socket says 172.21.15.66:36338) 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 -- 172.21.15.66:0/2701968077 learned_addr learned my addr 172.21.15.66:0/2701968077 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d544e640 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8027a50 0x55aae8057840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 -- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8027a50 msgr2=0x55aae8057840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae8027510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8027a50 0x55aae8057840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 -- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 msgr2=0x55aae8027510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae8027510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 -- 172.21.15.66:0/2701968077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae7d71920 con 0x55aae8054950 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56cffff640 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 0x55aae8057300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f56bc004420 tx=0x7f56bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b3f9a712c851e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.620+0000 7f56ba7fc640 1 -- 172.21.15.66:0/2701968077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc01b070 con 0x55aae8054950 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56ba7fc640 1 -- 172.21.15.66:0/2701968077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc027070 con 0x55aae8054950 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56ba7fc640 1 -- 172.21.15.66:0/2701968077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc003a20 con 0x55aae8054950 2024-03-30T13:12:43.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 -- 172.21.15.66:0/2701968077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 msgr2=0x55aae8057300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 0x55aae8057300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f56bc004420 tx=0x7f56bc014000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 -- 172.21.15.66:0/2701968077 shutdown_connections 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8027a50 0x55aae8057840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054950 0x55aae8057300 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 --2- 172.21.15.66:0/2701968077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae8027510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 -- 172.21.15.66:0/2701968077 >> 172.21.15.66:0/2701968077 conn(0x55aae8029780 msgr2=0x55aae8072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 -- 172.21.15.66:0/2701968077 shutdown_connections 2024-03-30T13:12:43.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.621+0000 7f56d9b75740 1 -- 172.21.15.66:0/2701968077 wait complete. 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.622+0000 7f56d9b75740 1 -- start start 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.622+0000 7f56d9b75740 1 Processor -- start 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 0x55aae804ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae8072800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae80502a0 0x55aae8056b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55aae802be20 con 0x55aae8054570 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aae7f2d9f0 con 0x55aae80502a0 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55aae802e660 con 0x55aae8054950 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 10 client.0 _ll_register_callbacks cb 0x55aae7f31c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 0x55aae804ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 0x55aae804ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36340/0 (socket says 172.21.15.66:36340) 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 -- 172.21.15.66:0/139853264 learned_addr learned my addr 172.21.15.66:0/139853264 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d544e640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae80502a0 0x55aae8056b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56cffff640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae8072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 -- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 msgr2=0x55aae8072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae8072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 -- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae80502a0 msgr2=0x55aae8056b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae80502a0 0x55aae8056b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 -- 172.21.15.66:0/139853264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae7c65210 con 0x55aae8054570 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 0x55aae804ca80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f56c407b6d0 tx=0x7f56c407b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f4f690f2ffcab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56af7fe640 1 -- 172.21.15.66:0/139853264 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4098060 con 0x55aae8054570 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56af7fe640 1 -- 172.21.15.66:0/139853264 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f56c4080bd0 con 0x55aae8054570 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.623+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc027070 con 0x55aae8054570 2024-03-30T13:12:43.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56af7fe640 1 -- 172.21.15.66:0/139853264 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f56c4093070 con 0x55aae8054570 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 10 client.6507 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55aae7dd20b0 con 0x55aae8054570 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56af7fe640 1 -- 172.21.15.66:0/139853264 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc027070 con 0x55aae8054570 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56af7fe640 1 -- 172.21.15.66:0/139853264 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f56c4043070 con 0x55aae8054570 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 10 client.6507 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 10 client.6507 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 20 client.6507 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae805a8c0 0x55aae805acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 4 client.6507 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae7f9ead0 con 0x55aae805a8c0 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56cffff640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae805a8c0 0x55aae805acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56cffff640 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae805a8c0 0x55aae805acd0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d2cd4475fa1180 server_cookie=890cd2052d2f9c91 in_seq=0 out_seq=0 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.624+0000 7f56af7fe640 10 client.6507 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56af7fe640 1 -- 172.21.15.66:0/139853264 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55aae7f9ead0 con 0x55aae805a8c0 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56af7fe640 10 client.6507 handle_command_reply: tid=0 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 client.6507 shutdown 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 msgr2=0x55aae804ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 0x55aae804ca80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f56c407b6d0 tx=0x7f56c407b340 comp rx=0 tx=0).stop 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 shutdown_connections 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae805a8c0 0x55aae805acd0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae80502a0 0x55aae8056b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56af7fe640 0 client.6507 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae8072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 --2- 172.21.15.66:0/139853264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae8054570 0x55aae804ca80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 >> 172.21.15.66:0/139853264 conn(0x55aae8027a50 msgr2=0x55aae802b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 shutdown_connections 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 1 -- 172.21.15.66:0/139853264 wait complete. 2024-03-30T13:12:43.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.625+0000 7f56d9b75740 20 client.6507 trim_cache size 0 max 16384 2024-03-30T13:12:43.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.640+0000 7f56d9b75740 1 Processor -- start 2024-03-30T13:12:43.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.640+0000 7f56d9b75740 1 -- start start 2024-03-30T13:12:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.640+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.640+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054570 0x55aae8022740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.640+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054950 0x55aae80595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d9b75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f56d02011f0 con 0x55aae80502a0 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0158e40 con 0x55aae8054950 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f56d0158140 con 0x55aae8054570 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80e66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36342/0 (socket says 172.21.15.66:36342) 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3922852258 learned_addr learned my addr 172.21.15.66:0/3922852258 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054570 msgr2=0x55aae8022740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054570 0x55aae8022740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054950 msgr2=0x55aae80595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d544e640 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054950 0x55aae80595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054950 0x55aae80595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3922852258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae7c3e2d0 con 0x55aae80502a0 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d544e640 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054950 0x55aae80595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80e66e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f56c407b440 tx=0x7f56c407f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50c56aba5ee9d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56b9ffb640 1 -- 172.21.15.66:0/3922852258 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c40842c0 con 0x55aae80502a0 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56b9ffb640 1 -- 172.21.15.66:0/3922852258 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c4080bd0 con 0x55aae80502a0 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.641+0000 7f56b9ffb640 1 -- 172.21.15.66:0/3922852258 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4099260 con 0x55aae80502a0 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 -- 172.21.15.66:0/3922852258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 msgr2=0x55aae80e66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80e66e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f56c407b440 tx=0x7f56c407f000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 -- 172.21.15.66:0/3922852258 shutdown_connections 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054950 0x55aae80595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054570 0x55aae8022740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3922852258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80e66e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 -- 172.21.15.66:0/3922852258 >> 172.21.15.66:0/3922852258 conn(0x55aae8027a50 msgr2=0x55aae8081f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 -- 172.21.15.66:0/3922852258 shutdown_connections 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 -- 172.21.15.66:0/3922852258 wait complete. 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.642+0000 7f56d9b75740 1 -- start start 2024-03-30T13:12:43.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 Processor -- start 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae802b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae802b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0158790 con 0x55aae80502a0 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aae8080ce0 con 0x55aae8054570 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55aae8084e90 con 0x55aae8054950 2024-03-30T13:12:43.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.643+0000 7f56d9b75740 10 client.0 _ll_register_callbacks cb 0x7f56c40069c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80847b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36352/0 (socket says 172.21.15.66:36352) 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3856819821 learned_addr learned my addr 172.21.15.66:0/3856819821 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d544e640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae802b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56cffff640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae802b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 msgr2=0x55aae802b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae802b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 msgr2=0x55aae802b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae802b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 -- 172.21.15.66:0/3856819821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae80850e0 con 0x55aae80502a0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d4c4d640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80847b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f56c407b6d0 tx=0x7f56c40abad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4446a6c5e5fc13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56af7fe640 1 -- 172.21.15.66:0/3856819821 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4098020 con 0x55aae80502a0 2024-03-30T13:12:43.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56af7fe640 1 -- 172.21.15.66:0/3856819821 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f56c4083de0 con 0x55aae80502a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c4080bd0 con 0x55aae80502a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56af7fe640 1 -- 172.21.15.66:0/3856819821 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f56c4084270 con 0x55aae80502a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d9b75740 10 client.6513 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55aae80e68a0 con 0x55aae80502a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.644+0000 7f56af7fe640 1 -- 172.21.15.66:0/3856819821 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c4080bd0 con 0x55aae80502a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56af7fe640 1 -- 172.21.15.66:0/3856819821 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f56c4043040 con 0x55aae80502a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 10 client.6513 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 10 client.6513 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 20 client.6513 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae80815a0 0x55aae8048c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 4 client.6513 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628421}] 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628421}) -- 0x55aae7f98e20 con 0x55aae80815a0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56cffff640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae80815a0 0x55aae8048c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56af7fe640 10 client.6513 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56cffff640 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae80815a0 0x55aae8048c90 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e34fb75d80fa14 server_cookie=849d861cf12beef5 in_seq=0 out_seq=0 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56af7fe640 1 -- 172.21.15.66:0/3856819821 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3987 (crc 0 0 0) 0x55aae7f98e20 con 0x55aae80815a0 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56af7fe640 10 client.6513 handle_command_reply: tid=0 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.645+0000 7f56d9b75740 1 client.6513 shutdown 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 msgr2=0x55aae80847b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80847b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f56c407b6d0 tx=0x7f56c40abad0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 shutdown_connections 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55aae80815a0 0x55aae8048c90 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55aae8054950 0x55aae802b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56af7fe640 0 client.6513 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aae8054570 0x55aae802b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 --2- 172.21.15.66:0/3856819821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aae80502a0 0x55aae80847b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 >> 172.21.15.66:0/3856819821 conn(0x55aae8027a50 msgr2=0x55aae80e5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 shutdown_connections 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 1 -- 172.21.15.66:0/3856819821 wait complete. 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d9b75740 20 client.6513 trim_cache size 0 max 16384 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debugfs.h", 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628421, 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.119417+0000", 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.649818+0000", 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 311, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.649818+0000", 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 311, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.119417+0000" 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 311, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.119417+0000" 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 184, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 32, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.146281+0000", 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 84, 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:43.668 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:43.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f56b406c530 msgr2=0x7f56b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f56b406c530 0x7f56b406e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f56d015fc30 tx=0x7f56bc009190 comp rx=0 tx=0).stop 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 msgr2=0x7f56d00cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 0x7f56d00cd560 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000c00 tx=0x7f56c40046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 shutdown_connections 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f56b406c530 0x7f56b406e9f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56d00ba090 0x7f56d01f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56d00b5dc0 0x7f56d00b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 --2- 172.21.15.66:0/321273626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56d006c620 0x7f56d00cd560 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 >> 172.21.15.66:0/321273626 conn(0x7f56d00678f0 msgr2=0x7f56d00cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 shutdown_connections 2024-03-30T13:12:43.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.646+0000 7f56d6eb1640 1 -- 172.21.15.66:0/321273626 wait complete. 2024-03-30T13:12:43.671 DEBUG:tasks.cephfs.test_uninlining:getting inode info #20 of 23:1099511628422 2024-03-30T13:12:43.671 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628422 2024-03-30T13:12:43.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628422 2024-03-30T13:12:43.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.784+0000 7f83367d4640 1 Processor -- start 2024-03-30T13:12:43.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.784+0000 7f83367d4640 1 -- start start 2024-03-30T13:12:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f83367d4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8330144cb0 0x7f83300b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f83367d4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8330072d50 0x7f83300ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f83367d4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83300b35e0 con 0x7f833015f800 2024-03-30T13:12:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f83367d4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f83300b3e80 con 0x7f8330072d50 2024-03-30T13:12:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f83367d4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f83300b38c0 con 0x7f8330144cb0 2024-03-30T13:12:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8330072d50 0x7f83300ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8330072d50 0x7f83300ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36368/0 (socket says 172.21.15.66:36368) 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 -- 172.21.15.66:0/250662268 learned_addr learned my addr 172.21.15.66:0/250662268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f832f7fe640 1 --1- 172.21.15.66:0/250662268 >> v1:172.21.15.66:6789/0 conn(0x7f833015f800 0x7f8330072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37792/0 (socket says 172.21.15.66:37792) 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f832ffff640 1 --2- 172.21.15.66:0/250662268 >> v2:172.21.15.117:3300/0 conn(0x7f8330144cb0 0x7f83300b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 -- 172.21.15.66:0/250662268 >> v2:172.21.15.117:3300/0 conn(0x7f8330144cb0 msgr2=0x7f83300b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 --2- 172.21.15.66:0/250662268 >> v2:172.21.15.117:3300/0 conn(0x7f8330144cb0 0x7f83300b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 -- 172.21.15.66:0/250662268 >> v1:172.21.15.66:6789/0 conn(0x7f833015f800 legacy=0x7f8330072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f832effd640 1 -- 172.21.15.66:0/250662268 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444503559 0 0) 0x7f83300b35e0 con 0x7f833015f800 2024-03-30T13:12:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.785+0000 7f8334d71640 1 -- 172.21.15.66:0/250662268 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83300b3ba0 con 0x7f8330072d50 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.786+0000 7f8334d71640 1 --2- 172.21.15.66:0/250662268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330072d50 0x7f83300ce050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f833006e3e0 tx=0x7f832802e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c258e01c286273ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.786+0000 7f832effd640 1 -- 172.21.15.66:0/250662268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832800b0d0 con 0x7f8330072d50 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.786+0000 7f832effd640 1 -- 172.21.15.66:0/250662268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832800b270 con 0x7f8330072d50 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.786+0000 7f832effd640 1 -- 172.21.15.66:0/250662268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328009600 con 0x7f8330072d50 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 -- 172.21.15.66:0/250662268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330072d50 msgr2=0x7f83300ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 --2- 172.21.15.66:0/250662268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330072d50 0x7f83300ce050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f833006e3e0 tx=0x7f832802e180 comp rx=0 tx=0).stop 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 -- 172.21.15.66:0/250662268 shutdown_connections 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 --2- 172.21.15.66:0/250662268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330072d50 0x7f83300ce050 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 --2- 172.21.15.66:0/250662268 >> v2:172.21.15.117:3300/0 conn(0x7f8330144cb0 0x7f83300b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 -- 172.21.15.66:0/250662268 >> 172.21.15.66:0/250662268 conn(0x7f83300b5b00 msgr2=0x7f833006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 -- 172.21.15.66:0/250662268 shutdown_connections 2024-03-30T13:12:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.787+0000 7f83367d4640 1 -- 172.21.15.66:0/250662268 wait complete. 2024-03-30T13:12:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.788+0000 7f83367d4640 1 Processor -- start 2024-03-30T13:12:43.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.788+0000 7f83367d4640 1 -- start start 2024-03-30T13:12:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f83367d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8330072d50 0x7f8330156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f83367d4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8330144cb0 0x7f83300ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f83367d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 0x7f833015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f83367d4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83300ce590 con 0x7f833015f800 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f83367d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f833015fc40 con 0x7f8330072d50 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f83367d4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f83300ce710 con 0x7f8330144cb0 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 0x7f833015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 0x7f833015b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36374/0 (socket says 172.21.15.66:36374) 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 -- 172.21.15.66:0/2664266646 learned_addr learned my addr 172.21.15.66:0/2664266646 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f832ffff640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8330072d50 0x7f8330156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f832f7fe640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8330144cb0 0x7f83300ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 -- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8330144cb0 msgr2=0x7f83300ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8330144cb0 0x7f83300ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 -- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8330072d50 msgr2=0x7f8330156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8330072d50 0x7f8330156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 -- 172.21.15.66:0/2664266646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83300ad940 con 0x7f833015f800 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.789+0000 7f8334d71640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 0x7f833015b960 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8328038040 tx=0x7f8328004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.790+0000 7f832cff9640 1 -- 172.21.15.66:0/2664266646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832800b0d0 con 0x7f833015f800 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.790+0000 7f832cff9640 1 -- 172.21.15.66:0/2664266646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8328009370 con 0x7f833015f800 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.790+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83300adbe0 con 0x7f833015f800 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.790+0000 7f832cff9640 1 -- 172.21.15.66:0/2664266646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832800b720 con 0x7f833015f800 2024-03-30T13:12:43.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.790+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83300ae1a0 con 0x7f833015f800 2024-03-30T13:12:43.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.791+0000 7f832cff9640 1 -- 172.21.15.66:0/2664266646 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8328047020 con 0x7f833015f800 2024-03-30T13:12:43.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.792+0000 7f832cff9640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f831c06c560 0x7f831c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.792+0000 7f832ffff640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f831c06c560 0x7f831c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.792+0000 7f832cff9640 1 -- 172.21.15.66:0/2664266646 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f83280adfa0 con 0x7f833015f800 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.792+0000 7f832ffff640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f831c06c560 0x7f831c06ea20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8320009ad0 tx=0x7f8320008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.792+0000 7f83393bf740 1 Processor -- start 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 -- start start 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f2b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f5b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f2ba50 0x55d4e4f5b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d4e4e95590 con 0x55d4e4f58950 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d4e4e33d30 con 0x55d4e4f58570 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d4e4e91540 con 0x55d4e4f2ba50 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f832f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f5b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f832f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f5b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36390/0 (socket says 172.21.15.66:36390) 2024-03-30T13:12:43.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.793+0000 7f832f7fe640 1 -- 172.21.15.66:0/9404622 learned_addr learned my addr 172.21.15.66:0/9404622 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f8334d71640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f2ba50 0x55d4e4f5b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832ffff640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f2b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832f7fe640 1 -- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f2ba50 msgr2=0x55d4e4f5b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832f7fe640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f2ba50 0x55d4e4f5b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832f7fe640 1 -- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 msgr2=0x55d4e4f2b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832f7fe640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f2b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832f7fe640 1 -- 172.21.15.66:0/9404622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e4c75920 con 0x55d4e4f58950 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832ffff640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f2b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f8334d71640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f2ba50 0x55d4e4f5b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f832f7fe640 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f5b300 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f83240074f0 tx=0x7f8324005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5c0c2138d20978 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f8315ffb640 1 -- 172.21.15.66:0/9404622 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324019070 con 0x55d4e4f58950 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f8315ffb640 1 -- 172.21.15.66:0/9404622 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83240016d0 con 0x55d4e4f58950 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.794+0000 7f8315ffb640 1 -- 172.21.15.66:0/9404622 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324001aa0 con 0x55d4e4f58950 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 -- 172.21.15.66:0/9404622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 msgr2=0x55d4e4f5b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f5b300 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f83240074f0 tx=0x7f8324005bc0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 -- 172.21.15.66:0/9404622 shutdown_connections 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f2ba50 0x55d4e4f5b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f5b300 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 --2- 172.21.15.66:0/9404622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f2b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 -- 172.21.15.66:0/9404622 >> 172.21.15.66:0/9404622 conn(0x55d4e4f2d780 msgr2=0x55d4e4f76300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 -- 172.21.15.66:0/9404622 shutdown_connections 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 -- 172.21.15.66:0/9404622 wait complete. 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.795+0000 7f83393bf740 1 -- start start 2024-03-30T13:12:43.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 Processor -- start 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58950 0x55d4e4f5dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f86520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d4e4f2fe20 con 0x55d4e4f4b500 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d4e4e319f0 con 0x55d4e4f58570 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d4e4f32660 con 0x55d4e4f58950 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f83393bf740 10 client.0 _ll_register_callbacks cb 0x55d4e4e35c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f8334d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f86520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f8334d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f86520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36402/0 (socket says 172.21.15.66:36402) 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.796+0000 7f8334d71640 1 -- 172.21.15.66:0/3989934410 learned_addr learned my addr 172.21.15.66:0/3989934410 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f832ffff640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f832f7fe640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58950 0x55d4e4f5dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f8334d71640 1 -- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58950 msgr2=0x55d4e4f5dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f8334d71640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58950 0x55d4e4f5dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f8334d71640 1 -- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 msgr2=0x55d4e4f5c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f8334d71640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f5c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f8334d71640 1 -- 172.21.15.66:0/3989934410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e4b69210 con 0x55d4e4f4b500 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f8334d71640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f86520 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f83280780a0 tx=0x7f8328009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0aab5d50c8291 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f830effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f830effd640 1 -- 172.21.15.66:0/3989934410 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280aa060 con 0x55d4e4f4b500 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f830effd640 1 -- 172.21.15.66:0/3989934410 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f8328084420 con 0x55d4e4f4b500 2024-03-30T13:12:43.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83240016d0 con 0x55d4e4f4b500 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f830effd640 1 -- 172.21.15.66:0/3989934410 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8328094070 con 0x55d4e4f4b500 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f83393bf740 10 client.6528 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d4e4cd60b0 con 0x55d4e4f4b500 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.797+0000 7f830effd640 1 -- 172.21.15.66:0/3989934410 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83240016d0 con 0x55d4e4f4b500 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f830effd640 1 -- 172.21.15.66:0/3989934410 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f8328008050 con 0x55d4e4f4b500 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f83393bf740 10 client.6528 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f83393bf740 10 client.6528 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f83393bf740 20 client.6528 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f83393bf740 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f5ef50 0x55d4e4f73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f83393bf740 4 client.6528 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e4ea2ad0 con 0x55d4e4f5ef50 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f832ffff640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f5ef50 0x55d4e4f73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f830effd640 10 client.6528 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.798+0000 7f832ffff640 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f5ef50 0x55d4e4f73ed0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0204d4ad0fd7e8 server_cookie=3514786e518c1530 in_seq=0 out_seq=0 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f830effd640 1 -- 172.21.15.66:0/3989934410 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d4e4ea2ad0 con 0x55d4e4f5ef50 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f830effd640 10 client.6528 handle_command_reply: tid=0 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 client.6528 shutdown 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 msgr2=0x55d4e4f86520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f86520 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f83280780a0 tx=0x7f8328009ef0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 shutdown_connections 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f5ef50 0x55d4e4f73ed0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f86520 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58950 0x55d4e4f5dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f830effd640 0 client.6528 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 --2- 172.21.15.66:0/3989934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58570 0x55d4e4f5c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 >> 172.21.15.66:0/3989934410 conn(0x55d4e4f2ba50 msgr2=0x55d4e4f2f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 shutdown_connections 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 1 -- 172.21.15.66:0/3989934410 wait complete. 2024-03-30T13:12:43.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.799+0000 7f83393bf740 20 client.6528 trim_cache size 0 max 16384 2024-03-30T13:12:43.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 Processor -- start 2024-03-30T13:12:43.817 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 -- start start 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f2ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8330200ed0 con 0x55d4e4f4b500 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83301ffee0 con 0x55d4e4f58950 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f833015b2c0 con 0x55d4e4f58570 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f832ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f2ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f832ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f2ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36408/0 (socket says 172.21.15.66:36408) 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.814+0000 7f832ffff640 1 -- 172.21.15.66:0/677347802 learned_addr learned my addr 172.21.15.66:0/677347802 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f832ffff640 1 -- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 msgr2=0x55d4e4f86060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f832ffff640 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f8334d71640 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f832ffff640 1 -- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58950 msgr2=0x55d4e4f865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f832ffff640 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f832ffff640 1 -- 172.21.15.66:0/677347802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e4b422d0 con 0x55d4e4f4b500 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f832ffff640 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f2ab80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55d4e4f5f420 tx=0x7f8320009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2fc50675a22c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f83157fa640 1 -- 172.21.15.66:0/677347802 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320015e10 con 0x55d4e4f4b500 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f83157fa640 1 -- 172.21.15.66:0/677347802 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832002e070 con 0x55d4e4f4b500 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.815+0000 7f83157fa640 1 -- 172.21.15.66:0/677347802 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320029d40 con 0x55d4e4f4b500 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 -- 172.21.15.66:0/677347802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 msgr2=0x55d4e4f2ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f2ab80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55d4e4f5f420 tx=0x7f8320009f90 comp rx=0 tx=0).stop 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 -- 172.21.15.66:0/677347802 shutdown_connections 2024-03-30T13:12:43.821 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 --2- 172.21.15.66:0/677347802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f4b500 0x55d4e4f2ab80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 -- 172.21.15.66:0/677347802 >> 172.21.15.66:0/677347802 conn(0x55d4e4f2ba50 msgr2=0x55d4e4f75c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 -- 172.21.15.66:0/677347802 shutdown_connections 2024-03-30T13:12:43.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.816+0000 7f83393bf740 1 -- 172.21.15.66:0/677347802 wait complete. 2024-03-30T13:12:43.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.817+0000 7f83393bf740 1 -- start start 2024-03-30T13:12:43.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.817+0000 7f83393bf740 1 Processor -- start 2024-03-30T13:12:43.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f4b500 0x55d4e4f78850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83301ff950 con 0x55d4e4f58950 2024-03-30T13:12:43.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d4e4f78d90 con 0x55d4e4f4b500 2024-03-30T13:12:43.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x55d4e4f73900 con 0x55d4e4f58570 2024-03-30T13:12:43.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f83393bf740 10 client.0 _ll_register_callbacks cb 0x7f83280071f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f8334d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f8334d71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36412/0 (socket says 172.21.15.66:36412) 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f8334d71640 1 -- 172.21.15.66:0/3663559333 learned_addr learned my addr 172.21.15.66:0/3663559333 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.818+0000 7f832ffff640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f4b500 0x55d4e4f78850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f832f7fe640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f8334d71640 1 -- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 msgr2=0x55d4e4f86060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f8334d71640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f8334d71640 1 -- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f4b500 msgr2=0x55d4e4f78850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f8334d71640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f4b500 0x55d4e4f78850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f8334d71640 1 -- 172.21.15.66:0/3663559333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e4f31160 con 0x55d4e4f58950 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f832f7fe640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f8334d71640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f83280072a0 tx=0x7f83280ac1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dae9c95eed104a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f830effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f830effd640 1 -- 172.21.15.66:0/3663559333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280aa070 con 0x55d4e4f58950 2024-03-30T13:12:43.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f830effd640 1 -- 172.21.15.66:0/3663559333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f832809cca0 con 0x55d4e4f58950 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832002e070 con 0x55d4e4f58950 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f830effd640 1 -- 172.21.15.66:0/3663559333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f83280a0070 con 0x55d4e4f58950 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f83393bf740 10 client.6534 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d4e4ff1840 con 0x55d4e4f58950 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f830effd640 1 -- 172.21.15.66:0/3663559333 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832002e070 con 0x55d4e4f58950 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f830effd640 1 -- 172.21.15.66:0/3663559333 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f83280a3070 con 0x55d4e4f58950 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f83393bf740 10 client.6534 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f83393bf740 10 client.6534 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.819+0000 7f83393bf740 20 client.6534 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f83393bf740 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f774c0 0x55d4e4f73eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f83393bf740 4 client.6534 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628422}] 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628422}) -- 0x55d4e4e9ce20 con 0x55d4e4f774c0 2024-03-30T13:12:43.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f832ffff640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f774c0 0x55d4e4f73eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f832ffff640 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f774c0 0x55d4e4f73eb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a901edc9573f51a6 server_cookie=f1505c89e099694 in_seq=0 out_seq=0 2024-03-30T13:12:43.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f830effd640 10 client.6534 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f830effd640 1 -- 172.21.15.66:0/3663559333 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3993 (crc 0 0 0) 0x55d4e4e9ce20 con 0x55d4e4f774c0 2024-03-30T13:12:43.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f830effd640 10 client.6534 handle_command_reply: tid=0 2024-03-30T13:12:43.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f83393bf740 1 client.6534 shutdown 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 msgr2=0x55d4e4f865a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.820+0000 7f83393bf740 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f83280072a0 tx=0x7f83280ac1b0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 shutdown_connections 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x55d4e4f774c0 0x55d4e4f73eb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d4e4f58950 0x55d4e4f865a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d4e4f58570 0x55d4e4f86060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 --2- 172.21.15.66:0/3663559333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4e4f4b500 0x55d4e4f78850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 >> 172.21.15.66:0/3663559333 conn(0x55d4e4f2ba50 msgr2=0x55d4e4f60440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f830effd640 0 client.6534 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 shutdown_connections 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 1 -- 172.21.15.66:0/3663559333 wait complete. 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83393bf740 20 client.6534 trim_cache size 0 max 16384 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-dev-proto.h", 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628422, 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.129393+0000", 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.651818+0000", 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:43.832 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:43.833 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 2111, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.651818+0000", 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 3, 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:43.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2111, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.129393+0000" 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2111, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.129393+0000" 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 272, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 34, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.226564+0000", 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 100, 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.842 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:43.846 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:43.846 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:43.846 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:43.846 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f831c06c560 msgr2=0x7f831c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f831c06c560 0x7f831c06ea20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8320009ad0 tx=0x7f8320008040 comp rx=0 tx=0).stop 2024-03-30T13:12:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 msgr2=0x7f833015b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 0x7f833015b960 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8328038040 tx=0x7f8328004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 shutdown_connections 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f831c06c560 0x7f831c06ea20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f833015f800 0x7f833015b960 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8330144cb0 0x7f83300ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 --2- 172.21.15.66:0/2664266646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8330072d50 0x7f8330156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.821+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 >> 172.21.15.66:0/2664266646 conn(0x7f83300b5b00 msgr2=0x7f8330072250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.822+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 shutdown_connections 2024-03-30T13:12:43.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.822+0000 7f83367d4640 1 -- 172.21.15.66:0/2664266646 wait complete. 2024-03-30T13:12:43.849 DEBUG:tasks.cephfs.test_uninlining:getting inode info #21 of 23:1099511628407 2024-03-30T13:12:43.849 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628407 2024-03-30T13:12:43.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628407 2024-03-30T13:12:43.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.955+0000 7f604ff1c640 1 Processor -- start 2024-03-30T13:12:43.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.955+0000 7f604ff1c640 1 -- start start 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604ff1c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60480b5dc0 0x7f60480b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604ff1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f60480d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604ff1c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f604806c620 0x7f604806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604ff1c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f60480b66e0 con 0x7f604806c620 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604ff1c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f60480d2ff0 con 0x7f60480b5dc0 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604ff1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f60480d32d0 con 0x7f60480ba090 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604e4b9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f604806c620 0x7f604806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604e4b9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f604806c620 0x7f604806cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36418/0 (socket says 172.21.15.66:36418) 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604e4b9640 1 -- 172.21.15.66:0/2760949770 learned_addr learned my addr 172.21.15.66:0/2760949770 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2760949770 >> v2:172.21.15.117:3300/0 conn(0x7f60480b5dc0 0x7f60480b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.956+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2760949770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f60480d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.957+0000 7f604e4b9640 1 -- 172.21.15.66:0/2760949770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 msgr2=0x7f60480d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.957+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2760949770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f60480d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.957+0000 7f604e4b9640 1 -- 172.21.15.66:0/2760949770 >> v2:172.21.15.117:3300/0 conn(0x7f60480b5dc0 msgr2=0x7f60480b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.957+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2760949770 >> v2:172.21.15.117:3300/0 conn(0x7f60480b5dc0 0x7f60480b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.957+0000 7f604e4b9640 1 -- 172.21.15.66:0/2760949770 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60480d35b0 con 0x7f604806c620 2024-03-30T13:12:43.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.958+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2760949770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604806c620 0x7f604806cad0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f604806d770 tx=0x7f604002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766722e380ac41f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.958+0000 7f604ccb6640 1 -- 172.21.15.66:0/2760949770 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604002ca70 con 0x7f604806c620 2024-03-30T13:12:43.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.958+0000 7f604ccb6640 1 -- 172.21.15.66:0/2760949770 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604002cc10 con 0x7f604806c620 2024-03-30T13:12:43.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.958+0000 7f604ccb6640 1 -- 172.21.15.66:0/2760949770 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400343f0 con 0x7f604806c620 2024-03-30T13:12:43.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 -- 172.21.15.66:0/2760949770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604806c620 msgr2=0x7f604806cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2760949770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604806c620 0x7f604806cad0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f604806d770 tx=0x7f604002bed0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 -- 172.21.15.66:0/2760949770 shutdown_connections 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2760949770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604806c620 0x7f604806cad0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2760949770 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f60480d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2760949770 >> v2:172.21.15.117:3300/0 conn(0x7f60480b5dc0 0x7f60480b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 -- 172.21.15.66:0/2760949770 >> 172.21.15.66:0/2760949770 conn(0x7f60480678f0 msgr2=0x7f60480d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 -- 172.21.15.66:0/2760949770 shutdown_connections 2024-03-30T13:12:43.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.959+0000 7f604ff1c640 1 -- 172.21.15.66:0/2760949770 wait complete. 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.960+0000 7f604ff1c640 1 Processor -- start 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.960+0000 7f604ff1c640 1 -- start start 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604806c620 0x7f60480cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 0x7f60480b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f604814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60480d4090 con 0x7f60480b5dc0 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60480d3b70 con 0x7f604806c620 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f604815e880 con 0x7f60480ba090 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 0x7f60480b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 0x7f60480b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36430/0 (socket says 172.21.15.66:36430) 2024-03-30T13:12:43.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 -- 172.21.15.66:0/2129275034 learned_addr learned my addr 172.21.15.66:0/2129275034 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f604814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604806c620 0x7f60480cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 -- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 msgr2=0x7f604814d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f604814d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 -- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604806c620 msgr2=0x7f60480cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604806c620 0x7f60480cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 -- 172.21.15.66:0/2129275034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60481e6c20 con 0x7f60480b5dc0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604806c620 0x7f60480cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 0x7f60480b8740 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f604400e840 tx=0x7f604400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f603a7fc640 1 -- 172.21.15.66:0/2129275034 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044017070 con 0x7f60480b5dc0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f603a7fc640 1 -- 172.21.15.66:0/2129275034 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604400c940 con 0x7f60480b5dc0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60481e6f20 con 0x7f60480b5dc0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.961+0000 7f603a7fc640 1 -- 172.21.15.66:0/2129275034 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604400cd10 con 0x7f60480b5dc0 2024-03-30T13:12:43.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.962+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60481e7760 con 0x7f60480b5dc0 2024-03-30T13:12:43.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.963+0000 7f603a7fc640 1 -- 172.21.15.66:0/2129275034 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f604401a070 con 0x7f60480b5dc0 2024-03-30T13:12:43.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.963+0000 7f603a7fc640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f602c06c530 0x7f602c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.964+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f602c06c530 0x7f602c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.964+0000 7f603a7fc640 1 -- 172.21.15.66:0/2129275034 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f604408b650 con 0x7f60480b5dc0 2024-03-30T13:12:43.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.964+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f602c06c530 0x7f602c06e9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f603c009b10 tx=0x7f603c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.965+0000 7f6052be1740 1 Processor -- start 2024-03-30T13:12:43.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.965+0000 7f6052be1740 1 -- start start 2024-03-30T13:12:43.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.966+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88b88510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.966+0000 7f6052be1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 0x557c88bb8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.966+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88b88a50 0x557c88bb8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.966+0000 7f6052be1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c88af2590 con 0x557c88bb5950 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.966+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557c88a90d30 con 0x557c88bb5570 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.966+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x557c88aee540 con 0x557c88b88a50 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 0x557c88bb8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 0x557c88bb8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36444/0 (socket says 172.21.15.66:36444) 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 -- 172.21.15.66:0/2455183518 learned_addr learned my addr 172.21.15.66:0/2455183518 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88b88510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88b88a50 0x557c88bb8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 -- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88b88a50 msgr2=0x557c88bb8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88b88a50 0x557c88bb8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 -- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 msgr2=0x557c88b88510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88b88510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 -- 172.21.15.66:0/2455183518 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c888d2920 con 0x557c88bb5950 2024-03-30T13:12:43.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 0x557c88bb8300 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6044061ac0 tx=0x7f60440594c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ebe86dbc24f100 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f602b7fe640 1 -- 172.21.15.66:0/2455183518 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044021040 con 0x557c88bb5950 2024-03-30T13:12:43.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f602b7fe640 1 -- 172.21.15.66:0/2455183518 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6044061ff0 con 0x557c88bb5950 2024-03-30T13:12:43.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.967+0000 7f602b7fe640 1 -- 172.21.15.66:0/2455183518 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044055b30 con 0x557c88bb5950 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 -- 172.21.15.66:0/2455183518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 msgr2=0x557c88bb8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 0x557c88bb8300 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6044061ac0 tx=0x7f60440594c0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 -- 172.21.15.66:0/2455183518 shutdown_connections 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88b88a50 0x557c88bb8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5950 0x557c88bb8300 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 --2- 172.21.15.66:0/2455183518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88b88510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 -- 172.21.15.66:0/2455183518 >> 172.21.15.66:0/2455183518 conn(0x557c88b8a780 msgr2=0x557c88bd3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 -- 172.21.15.66:0/2455183518 shutdown_connections 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.968+0000 7f6052be1740 1 -- 172.21.15.66:0/2455183518 wait complete. 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.969+0000 7f6052be1740 1 -- start start 2024-03-30T13:12:43.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.969+0000 7f6052be1740 1 Processor -- start 2024-03-30T13:12:43.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 0x557c88bd0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88be7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb9b90 0x557c88bba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c88b8ce20 con 0x557c88bb5570 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557c88a8e9f0 con 0x557c88bb9b90 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x557c88b8f660 con 0x557c88bb5950 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f6052be1740 10 client.0 _ll_register_callbacks cb 0x557c88a92c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 0x557c88bd0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 0x557c88bd0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36456/0 (socket says 172.21.15.66:36456) 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 -- 172.21.15.66:0/1262399664 learned_addr learned my addr 172.21.15.66:0/1262399664 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604d4b7640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88be7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604e4b9640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb9b90 0x557c88bba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 -- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 msgr2=0x557c88be7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88be7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 -- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb9b90 msgr2=0x557c88bba010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb9b90 0x557c88bba010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 -- 172.21.15.66:0/1262399664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c887c6210 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604d4b7640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88be7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604e4b9640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb9b90 0x557c88bba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.971+0000 7f604dcb8640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 0x557c88bd0850 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f603c0157c0 tx=0x7f603c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d0e8fdbd9e6417 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6028ff9640 1 -- 172.21.15.66:0/1262399664 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c026070 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6028ff9640 1 -- 172.21.15.66:0/1262399664 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f603c014960 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6044061ff0 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6028ff9640 1 -- 172.21.15.66:0/1262399664 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f603c031050 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 10 client.6546 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x557c889330b0 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6028ff9640 1 -- 172.21.15.66:0/1262399664 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6044061ff0 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6028ff9640 1 -- 172.21.15.66:0/1262399664 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f603c034020 con 0x557c88bb5570 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 10 client.6546 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 10 client.6546 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 20 client.6546 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88be3ff0 0x557c88be43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 4 client.6546 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c88affad0 con 0x557c88be3ff0 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.972+0000 7f604d4b7640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88be3ff0 0x557c88be43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f604d4b7640 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88be3ff0 0x557c88be43f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4dfcd016eb2b0d server_cookie=b6c95c1e605b0500 in_seq=0 out_seq=0 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6028ff9640 10 client.6546 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6028ff9640 1 -- 172.21.15.66:0/1262399664 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x557c88affad0 con 0x557c88be3ff0 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6028ff9640 10 client.6546 handle_command_reply: tid=0 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 client.6546 shutdown 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 msgr2=0x557c88bd0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 0x557c88bd0850 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f603c0157c0 tx=0x7f603c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 shutdown_connections 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88be3ff0 0x557c88be43f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb9b90 0x557c88bba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88be7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6028ff9640 0 client.6546 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 --2- 172.21.15.66:0/1262399664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb5570 0x557c88bd0850 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.973+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 >> 172.21.15.66:0/1262399664 conn(0x557c88b88a50 msgr2=0x557c88b8c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.974+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 shutdown_connections 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.974+0000 7f6052be1740 1 -- 172.21.15.66:0/1262399664 wait complete. 2024-03-30T13:12:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.974+0000 7f6052be1740 20 client.6546 trim_cache size 0 max 16384 2024-03-30T13:12:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.988+0000 7f6052be1740 1 Processor -- start 2024-03-30T13:12:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.988+0000 7f6052be1740 1 -- start start 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88bd4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88b8e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f6052be1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88bd13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f6052be1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60482012e0 con 0x557c88bb9b90 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6048200320 con 0x557c88bb5570 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f60481f4f60 con 0x557c88bb5950 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88bd13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88bd13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36464/0 (socket says 172.21.15.66:36464) 2024-03-30T13:12:43.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 -- 172.21.15.66:0/691467735 learned_addr learned my addr 172.21.15.66:0/691467735 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604dcb8640 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88bd4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604d4b7640 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88b8e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 -- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 msgr2=0x557c88b8e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88b8e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 -- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 msgr2=0x557c88bd4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88bd4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.989+0000 7f604e4b9640 1 -- 172.21.15.66:0/691467735 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8879f2d0 con 0x557c88bb9b90 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f604e4b9640 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88bd13a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f604002c4e0 tx=0x7f604002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1269b22568ef6e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f602affd640 1 -- 172.21.15.66:0/691467735 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604004c070 con 0x557c88bb9b90 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f602affd640 1 -- 172.21.15.66:0/691467735 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604002c870 con 0x557c88bb9b90 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f602affd640 1 -- 172.21.15.66:0/691467735 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604002ca20 con 0x557c88bb9b90 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 -- 172.21.15.66:0/691467735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 msgr2=0x557c88bd13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88bd13a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f604002c4e0 tx=0x7f604002e000 comp rx=0 tx=0).stop 2024-03-30T13:12:43.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 -- 172.21.15.66:0/691467735 shutdown_connections 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88bd13a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88b8e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 --2- 172.21.15.66:0/691467735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88bd4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 -- 172.21.15.66:0/691467735 >> 172.21.15.66:0/691467735 conn(0x557c88b88a50 msgr2=0x557c88bd0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 -- 172.21.15.66:0/691467735 shutdown_connections 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.990+0000 7f6052be1740 1 -- 172.21.15.66:0/691467735 wait complete. 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.991+0000 7f6052be1740 1 -- start start 2024-03-30T13:12:43.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.991+0000 7f6052be1740 1 Processor -- start 2024-03-30T13:12:44.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88c51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88bb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88c52b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c88a778d0 con 0x557c88bb9b90 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ffcd0 con 0x557c88bb5570 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x557c88bd18e0 con 0x557c88bb5950 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f6052be1740 10 client.0 _ll_register_callbacks cb 0x7f603c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88c52b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88c52b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36468/0 (socket says 172.21.15.66:36468) 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604dcb8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88c51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 -- 172.21.15.66:0/2011203344 learned_addr learned my addr 172.21.15.66:0/2011203344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604d4b7640 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88bb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 -- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 msgr2=0x557c88bb78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88bb78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 -- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 msgr2=0x557c88c51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88c51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.992+0000 7f604e4b9640 1 -- 172.21.15.66:0/2011203344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c88bd1540 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f604e4b9640 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88c52b90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6040009f40 tx=0x7f6040004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f0a590b04b86b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6028ff9640 1 -- 172.21.15.66:0/2011203344 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604004c070 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6028ff9640 1 -- 172.21.15.66:0/2011203344 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f6040051ca0 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604002c870 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6028ff9640 1 -- 172.21.15.66:0/2011203344 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6040047840 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 10 client.6552 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x557c88bd3b00 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6028ff9640 1 -- 172.21.15.66:0/2011203344 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604002c870 con 0x557c88bb9b90 2024-03-30T13:12:44.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6028ff9640 1 -- 172.21.15.66:0/2011203344 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f6040008040 con 0x557c88bb9b90 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 10 client.6552 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 10 client.6552 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 20 client.6552 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88b8a230 0x557c88bb3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 4 client.6552 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628407}] 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628407}) -- 0x557c88af9e20 con 0x557c88b8a230 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.993+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88b8a230 0x557c88bb3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6028ff9640 10 client.6552 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f604dcb8640 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88b8a230 0x557c88bb3b30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c64183c6d899f8 server_cookie=8f46c0429ed7f93b in_seq=0 out_seq=0 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6028ff9640 1 -- 172.21.15.66:0/2011203344 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3977 (crc 0 0 0) 0x557c88af9e20 con 0x557c88b8a230 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6028ff9640 10 client.6552 handle_command_reply: tid=0 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 client.6552 shutdown 2024-03-30T13:12:44.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 msgr2=0x557c88c52b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88c52b90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6040009f40 tx=0x7f6040004ab0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 shutdown_connections 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x557c88b8a230 0x557c88bb3b30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88bb9b90 0x557c88c52b90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557c88bb5950 0x557c88bb78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6028ff9640 0 client.6552 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 --2- 172.21.15.66:0/2011203344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557c88bb5570 0x557c88c51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 >> 172.21.15.66:0/2011203344 conn(0x557c88b88a50 msgr2=0x557c88bd6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 shutdown_connections 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 1 -- 172.21.15.66:0/2011203344 wait complete. 2024-03-30T13:12:44.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.994+0000 7f6052be1740 20 client.6552 trim_cache size 0 max 16384 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/Makefile", 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628407, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.131683+0000", 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.637818+0000", 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:44.005 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 356, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:44.006 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.637818+0000", 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:44.007 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.131683+0000" 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 356, 2024-03-30T13:12:44.008 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.131683+0000" 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 220, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 4, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:44.009 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.226275+0000", 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 104, 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:44.010 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:44.011 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:44.012 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.013 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.014 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.015 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:44.016 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:44.017 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:44.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f602c06c530 msgr2=0x7f602c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f602c06c530 0x7f602c06e9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f603c009b10 tx=0x7f603c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 msgr2=0x7f60480b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 0x7f60480b8740 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f604400e840 tx=0x7f604400ed00 comp rx=0 tx=0).stop 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 shutdown_connections 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f602c06c530 0x7f602c06e9f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60480ba090 0x7f604814d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60480b5dc0 0x7f60480b8740 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 --2- 172.21.15.66:0/2129275034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604806c620 0x7f60480cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 >> 172.21.15.66:0/2129275034 conn(0x7f60480678f0 msgr2=0x7f6048160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 shutdown_connections 2024-03-30T13:12:44.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:43.995+0000 7f604ff1c640 1 -- 172.21.15.66:0/2129275034 wait complete. 2024-03-30T13:12:44.020 DEBUG:tasks.cephfs.test_uninlining:getting inode info #22 of 23:1099511628430 2024-03-30T13:12:44.020 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628430 2024-03-30T13:12:44.020 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628430 2024-03-30T13:12:44.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.126+0000 7fcb9367e640 1 Processor -- start 2024-03-30T13:12:44.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.126+0000 7fcb9367e640 1 -- start start 2024-03-30T13:12:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9367e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcb8c1472d0 0x7fcb8c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9367e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb8c06c690 0x7fcb8c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9367e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9367e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c148140 con 0x7fcb8c1472d0 2024-03-30T13:12:44.131 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9367e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c071040 con 0x7fcb8c06c690 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9367e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c1476b0 con 0x7fcb8c06d930 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcb8c1472d0 0x7fcb8c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcb8c1472d0 0x7fcb8c0cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36478/0 (socket says 172.21.15.66:36478) 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 -- 172.21.15.66:0/1023864393 learned_addr learned my addr 172.21.15.66:0/1023864393 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb90c19640 1 --2- 172.21.15.66:0/1023864393 >> v2:172.21.15.117:3300/0 conn(0x7fcb8c06c690 0x7fcb8c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 -- 172.21.15.66:0/1023864393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 msgr2=0x7fcb8c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 -- 172.21.15.66:0/1023864393 >> v2:172.21.15.117:3300/0 conn(0x7fcb8c06c690 msgr2=0x7fcb8c06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 --2- 172.21.15.66:0/1023864393 >> v2:172.21.15.117:3300/0 conn(0x7fcb8c06c690 0x7fcb8c06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb9141a640 1 -- 172.21.15.66:0/1023864393 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c06cb20 con 0x7fcb8c1472d0 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb90c19640 1 --2- 172.21.15.66:0/1023864393 >> v2:172.21.15.117:3300/0 conn(0x7fcb8c06c690 0x7fcb8c06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:44.132 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.127+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.128+0000 7fcb9141a640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c0cf410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c074280 tx=0x7fcb8802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6de75d4729d39d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.128+0000 7fcb83fff640 1 -- 172.21.15.66:0/1023864393 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8802e070 con 0x7fcb8c1472d0 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.128+0000 7fcb83fff640 1 -- 172.21.15.66:0/1023864393 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb88037070 con 0x7fcb8c1472d0 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.128+0000 7fcb83fff640 1 -- 172.21.15.66:0/1023864393 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88031a00 con 0x7fcb8c1472d0 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 -- 172.21.15.66:0/1023864393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1472d0 msgr2=0x7fcb8c0cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c0cf410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c074280 tx=0x7fcb8802b880 comp rx=0 tx=0).stop 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 -- 172.21.15.66:0/1023864393 shutdown_connections 2024-03-30T13:12:44.134 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 --2- 172.21.15.66:0/1023864393 >> v2:172.21.15.117:3300/0 conn(0x7fcb8c06c690 0x7fcb8c06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 --2- 172.21.15.66:0/1023864393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c0cf410 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 -- 172.21.15.66:0/1023864393 >> 172.21.15.66:0/1023864393 conn(0x7fcb8c0b3e70 msgr2=0x7fcb8c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 -- 172.21.15.66:0/1023864393 shutdown_connections 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.129+0000 7fcb9367e640 1 -- 172.21.15.66:0/1023864393 wait complete. 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.130+0000 7fcb9367e640 1 Processor -- start 2024-03-30T13:12:44.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.130+0000 7fcb9367e640 1 -- start start 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9367e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 0x7fcb8c0ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9367e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c1f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9367e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9367e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c071850 con 0x7fcb8c06c690 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9367e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c071570 con 0x7fcb8c1472d0 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9367e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c071b30 con 0x7fcb8c06d930 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 0x7fcb8c0ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 0x7fcb8c0ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36490/0 (socket says 172.21.15.66:36490) 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 -- 172.21.15.66:0/607432764 learned_addr learned my addr 172.21.15.66:0/607432764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb90c19640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c1f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 -- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 msgr2=0x7fcb8c1f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c1f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 -- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c1472d0 msgr2=0x7fcb8c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 -- 172.21.15.66:0/607432764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c0b1690 con 0x7fcb8c06c690 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb90c19640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c1f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.131+0000 7fcb9141a640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 0x7fcb8c0ac550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.132+0000 7fcb81ffb640 1 -- 172.21.15.66:0/607432764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88032880 con 0x7fcb8c06c690 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.132+0000 7fcb81ffb640 1 -- 172.21.15.66:0/607432764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb88032a20 con 0x7fcb8c06c690 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.132+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c0b1970 con 0x7fcb8c06c690 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.132+0000 7fcb81ffb640 1 -- 172.21.15.66:0/607432764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88032e10 con 0x7fcb8c06c690 2024-03-30T13:12:44.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.132+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c15e460 con 0x7fcb8c06c690 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.134+0000 7fcb81ffb640 1 -- 172.21.15.66:0/607432764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcb8804c060 con 0x7fcb8c06c690 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.134+0000 7fcb81ffb640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcb7006c530 0x7fcb7006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.134+0000 7fcb81ffb640 1 -- 172.21.15.66:0/607432764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fcb880ad430 con 0x7fcb8c06c690 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.134+0000 7fcb90c19640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcb7006c530 0x7fcb7006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.135+0000 7fcb90c19640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcb7006c530 0x7fcb7006e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c1f7460 tx=0x7fcb7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.135+0000 7fcb96260740 1 Processor -- start 2024-03-30T13:12:44.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.135+0000 7fcb96260740 1 -- start start 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x55591415b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb96260740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 0x55591418b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55591415ba50 0x55591418b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb96260740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5559140c5590 con 0x555914188950 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555914063d30 con 0x55591415ba50 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x5559140c1540 con 0x555914188570 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 0x55591418b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 0x55591418b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36500/0 (socket says 172.21.15.66:36500) 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 -- 172.21.15.66:0/847172395 learned_addr learned my addr 172.21.15.66:0/847172395 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb9141a640 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x55591415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55591415ba50 0x55591418b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 -- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 msgr2=0x55591415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x55591415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 -- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55591415ba50 msgr2=0x55591418b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55591415ba50 0x55591418b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 -- 172.21.15.66:0/847172395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555913ea5920 con 0x555914188950 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.136+0000 7fcb90c19640 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 0x55591418b300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00fa90 tx=0x7fcb7c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6b61e4fe26524d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb76ffd640 1 -- 172.21.15.66:0/847172395 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c01b050 con 0x555914188950 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb76ffd640 1 -- 172.21.15.66:0/847172395 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c025070 con 0x555914188950 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb76ffd640 1 -- 172.21.15.66:0/847172395 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c014de0 con 0x555914188950 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 -- 172.21.15.66:0/847172395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 msgr2=0x55591418b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 0x55591418b300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00fa90 tx=0x7fcb7c012000 comp rx=0 tx=0).stop 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 -- 172.21.15.66:0/847172395 shutdown_connections 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55591415ba50 0x55591418b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188950 0x55591418b300 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 --2- 172.21.15.66:0/847172395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x55591415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 -- 172.21.15.66:0/847172395 >> 172.21.15.66:0/847172395 conn(0x55591415d780 msgr2=0x5559141a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 -- 172.21.15.66:0/847172395 shutdown_connections 2024-03-30T13:12:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.137+0000 7fcb96260740 1 -- 172.21.15.66:0/847172395 wait complete. 2024-03-30T13:12:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.138+0000 7fcb96260740 1 -- start start 2024-03-30T13:12:44.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.138+0000 7fcb96260740 1 Processor -- start 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x5559141813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x555914181930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x55591418dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55591415fe20 con 0x555914180be0 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559140619f0 con 0x555914188950 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555914162660 con 0x555914188570 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb96260740 10 client.0 _ll_register_callbacks cb 0x555914065c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x55591418dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x55591418dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36514/0 (socket says 172.21.15.66:36514) 2024-03-30T13:12:44.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 -- 172.21.15.66:0/1040698068 learned_addr learned my addr 172.21.15.66:0/1040698068 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb90c19640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x555914181930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb9141a640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x5559141813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 -- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 msgr2=0x5559141813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x5559141813f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 -- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 msgr2=0x555914181930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x555914181930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 -- 172.21.15.66:0/1040698068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555913d99210 con 0x555914180be0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x55591418dbc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400df10 tx=0x7fcb8400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bf3147337d35c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.139+0000 7fcb6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb6bfff640 1 -- 172.21.15.66:0/1040698068 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8401c060 con 0x555914180be0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb6bfff640 1 -- 172.21.15.66:0/1040698068 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fcb8400a610 con 0x555914180be0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7c025070 con 0x555914180be0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb6bfff640 1 -- 172.21.15.66:0/1040698068 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcb84022050 con 0x555914180be0 2024-03-30T13:12:44.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 10 client.6564 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:44.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x555913f060b0 con 0x555914180be0 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb6bfff640 1 -- 172.21.15.66:0/1040698068 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c025070 con 0x555914180be0 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb6bfff640 1 -- 172.21.15.66:0/1040698068 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fcb84025020 con 0x555914180be0 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 10 client.6564 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 10 client.6564 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 20 client.6564 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141b9b10 0x5559141b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 4 client.6564 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559140d2ad0 con 0x5559141b9b10 2024-03-30T13:12:44.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.140+0000 7fcb9141a640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141b9b10 0x5559141b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb6bfff640 10 client.6564 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb9141a640 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141b9b10 0x5559141b9f20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6080d893d413c05e server_cookie=5a74ac2959d463e1 in_seq=0 out_seq=0 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb6bfff640 1 -- 172.21.15.66:0/1040698068 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5559140d2ad0 con 0x5559141b9b10 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb6bfff640 10 client.6564 handle_command_reply: tid=0 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 client.6564 shutdown 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 msgr2=0x55591418dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x55591418dbc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400df10 tx=0x7fcb8400c7f0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 shutdown_connections 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141b9b10 0x5559141b9f20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x55591418dbc0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x555914181930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb6bfff640 0 client.6564 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 --2- 172.21.15.66:0/1040698068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188570 0x5559141813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.141+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 >> 172.21.15.66:0/1040698068 conn(0x55591415ba50 msgr2=0x55591415f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.142+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 shutdown_connections 2024-03-30T13:12:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.142+0000 7fcb96260740 1 -- 172.21.15.66:0/1040698068 wait complete. 2024-03-30T13:12:44.153 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.142+0000 7fcb96260740 20 client.6564 trim_cache size 0 max 16384 2024-03-30T13:12:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.156+0000 7fcb96260740 1 Processor -- start 2024-03-30T13:12:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.156+0000 7fcb96260740 1 -- start start 2024-03-30T13:12:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb96260740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x5559141c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188570 0x55591418e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188950 0x5559141a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb96260740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c200fe0 con 0x555914180be0 2024-03-30T13:12:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c200030 con 0x555914188570 2024-03-30T13:12:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c0ab030 con 0x555914188950 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x5559141c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x5559141c00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36524/0 (socket says 172.21.15.66:36524) 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 -- 172.21.15.66:0/2626980846 learned_addr learned my addr 172.21.15.66:0/2626980846 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 -- 172.21.15.66:0/2626980846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188950 msgr2=0x5559141a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188950 0x5559141a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 -- 172.21.15.66:0/2626980846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188570 msgr2=0x55591418e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188570 0x55591418e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 -- 172.21.15.66:0/2626980846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555913d722d0 con 0x555914180be0 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.157+0000 7fcb9141a640 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x5559141c00d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcb8807b0f0 tx=0x7fcb880ad040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbe4b8819ab2cb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb767fc640 1 -- 172.21.15.66:0/2626980846 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88083d40 con 0x555914180be0 2024-03-30T13:12:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb767fc640 1 -- 172.21.15.66:0/2626980846 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb880a9070 con 0x555914180be0 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb767fc640 1 -- 172.21.15.66:0/2626980846 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88079460 con 0x555914180be0 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 -- 172.21.15.66:0/2626980846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 msgr2=0x5559141c00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x5559141c00d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcb8807b0f0 tx=0x7fcb880ad040 comp rx=0 tx=0).stop 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 -- 172.21.15.66:0/2626980846 shutdown_connections 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914188950 0x5559141a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188570 0x55591418e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 --2- 172.21.15.66:0/2626980846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914180be0 0x5559141c00d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 -- 172.21.15.66:0/2626980846 >> 172.21.15.66:0/2626980846 conn(0x55591415ba50 msgr2=0x55591415f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 -- 172.21.15.66:0/2626980846 shutdown_connections 2024-03-30T13:12:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.158+0000 7fcb96260740 1 -- 172.21.15.66:0/2626980846 wait complete. 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.159+0000 7fcb96260740 1 -- start start 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.159+0000 7fcb96260740 1 Processor -- start 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914180be0 0x5559141816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 0x5559141a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x5559141a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55591415f9f0 con 0x555914188570 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1ff9e0 con 0x555914188950 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x555914223600 con 0x555914180be0 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb96260740 10 client.0 _ll_register_callbacks cb 0x7fcb84007bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 0x5559141a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 0x5559141a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36536/0 (socket says 172.21.15.66:36536) 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 -- 172.21.15.66:0/4216662436 learned_addr learned my addr 172.21.15.66:0/4216662436 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb91c1b640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x5559141a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb9141a640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914180be0 0x5559141816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 -- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914180be0 msgr2=0x5559141816d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914180be0 0x5559141816d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 -- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 msgr2=0x5559141a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x5559141a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 -- 172.21.15.66:0/4216662436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559141891c0 con 0x555914188570 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb90c19640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 0x5559141a6350 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c022040 tx=0x7fcb7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801e64769633fcb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.161+0000 7fcb6bfff640 1 -- 172.21.15.66:0/4216662436 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c005da0 con 0x555914188570 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb6bfff640 1 -- 172.21.15.66:0/4216662436 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7fcb7c036ce0 con 0x555914188570 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb880a9070 con 0x555914188570 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb6bfff640 1 -- 172.21.15.66:0/4216662436 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcb7c01b020 con 0x555914188570 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 10 client.6570 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x555914181c10 con 0x555914188570 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb6bfff640 1 -- 172.21.15.66:0/4216662436 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb880a9070 con 0x555914188570 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb6bfff640 1 -- 172.21.15.66:0/4216662436 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7fcb7c033070 con 0x555914188570 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 10 client.6570 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 10 client.6570 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 20 client.6570 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141bfb30 0x55591417f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 4 client.6570 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628430}] 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628430}) -- 0x5559140cce20 con 0x5559141bfb30 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.162+0000 7fcb9141a640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141bfb30 0x55591417f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb6bfff640 10 client.6570 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb9141a640 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141bfb30 0x55591417f110 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56c22fd272d5199 server_cookie=3804f8f33cd54da6 in_seq=0 out_seq=0 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb6bfff640 1 -- 172.21.15.66:0/4216662436 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3979 (crc 0 0 0) 0x5559140cce20 con 0x5559141bfb30 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb6bfff640 10 client.6570 handle_command_reply: tid=0 2024-03-30T13:12:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 client.6570 shutdown 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 msgr2=0x5559141a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 0x5559141a6350 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c022040 tx=0x7fcb7c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 shutdown_connections 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x5559141bfb30 0x55591417f110 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555914188950 0x5559141a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555914188570 0x5559141a6350 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 --2- 172.21.15.66:0/4216662436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555914180be0 0x5559141816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb6bfff640 0 client.6570 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 >> 172.21.15.66:0/4216662436 conn(0x55591415ba50 msgr2=0x555914183670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 shutdown_connections 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 1 -- 172.21.15.66:0/4216662436 wait complete. 2024-03-30T13:12:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.163+0000 7fcb96260740 20 client.6570 trim_cache size 0 max 16384 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/symlink.c", 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628430, 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.132598+0000", 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.660817+0000", 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:44.173 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:44.174 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 497, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.660817+0000", 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:44.175 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 497, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.132598+0000" 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.176 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 497, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.132598+0000" 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 196, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 50, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:44.177 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.227150+0000", 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 106, 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:44.179 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:44.180 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.181 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.182 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.183 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:44.184 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:44.185 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcb7006c530 msgr2=0x7fcb7006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcb7006c530 0x7fcb7006e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c1f7460 tx=0x7fcb7c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 msgr2=0x7fcb8c0ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 0x7fcb8c0ac550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 shutdown_connections 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fcb7006c530 0x7fcb7006e9f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb8c1472d0 0x7fcb8c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8c06d930 0x7fcb8c1f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 --2- 172.21.15.66:0/607432764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb8c06c690 0x7fcb8c0ac550 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 >> 172.21.15.66:0/607432764 conn(0x7fcb8c0b3e70 msgr2=0x7fcb8c0749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 shutdown_connections 2024-03-30T13:12:44.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.164+0000 7fcb9367e640 1 -- 172.21.15.66:0/607432764 wait complete. 2024-03-30T13:12:44.188 DEBUG:tasks.cephfs.test_uninlining:getting inode info #23 of 23:1099511628412 2024-03-30T13:12:44.188 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628412 2024-03-30T13:12:44.188 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628412 2024-03-30T13:12:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.294+0000 7ff4815aa640 1 Processor -- start 2024-03-30T13:12:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.294+0000 7ff4815aa640 1 -- start start 2024-03-30T13:12:44.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff4815aa640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff47c145d70 0x7ff47c160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff4815aa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c15ceb0 0x7ff47c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff4815aa640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff47c0cc970 con 0x7ff47c0ce490 2024-03-30T13:12:44.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff4815aa640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff47c0ce8f0 con 0x7ff47c145d70 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff4815aa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c146150 con 0x7ff47c15ceb0 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47a7fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff47c0ce490 0x7ff47c0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37794/0 (socket says 172.21.15.66:37794) 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47affd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff47c145d70 0x7ff47c160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3022594043 learned_addr learned my addr 172.21.15.66:0/3022594043 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/3022594043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c15ceb0 0x7ff47c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47affd640 1 -- 172.21.15.66:0/3022594043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c15ceb0 msgr2=0x7ff47c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47affd640 1 --2- 172.21.15.66:0/3022594043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c15ceb0 0x7ff47c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47affd640 1 -- 172.21.15.66:0/3022594043 >> v1:172.21.15.66:6789/0 conn(0x7ff47c0ce490 legacy=0x7ff47c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff47affd640 1 -- 172.21.15.66:0/3022594043 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c0ccc50 con 0x7ff47c145d70 2024-03-30T13:12:44.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.296+0000 7ff479ffb640 1 -- 172.21.15.66:0/3022594043 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201544164 0 0) 0x7ff47c0cc970 con 0x7ff47c0ce490 2024-03-30T13:12:44.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.297+0000 7ff47affd640 1 --2- 172.21.15.66:0/3022594043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c160880 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff4640097a0 tx=0x7ff46402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd7dda082d2a3e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.297+0000 7ff479ffb640 1 -- 172.21.15.66:0/3022594043 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640315a0 con 0x7ff47c145d70 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.297+0000 7ff479ffb640 1 -- 172.21.15.66:0/3022594043 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff464037070 con 0x7ff47c145d70 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.297+0000 7ff479ffb640 1 -- 172.21.15.66:0/3022594043 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464031960 con 0x7ff47c145d70 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 -- 172.21.15.66:0/3022594043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 msgr2=0x7ff47c160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3022594043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c160880 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff4640097a0 tx=0x7ff46402b840 comp rx=0 tx=0).stop 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 -- 172.21.15.66:0/3022594043 shutdown_connections 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3022594043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c15ceb0 0x7ff47c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3022594043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c160880 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 -- 172.21.15.66:0/3022594043 >> 172.21.15.66:0/3022594043 conn(0x7ff47c0b3e70 msgr2=0x7ff47c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 -- 172.21.15.66:0/3022594043 shutdown_connections 2024-03-30T13:12:44.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.298+0000 7ff4815aa640 1 -- 172.21.15.66:0/3022594043 wait complete. 2024-03-30T13:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.299+0000 7ff4815aa640 1 Processor -- start 2024-03-30T13:12:44.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.299+0000 7ff4815aa640 1 -- start start 2024-03-30T13:12:44.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff4815aa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c0ce490 0x7ff47c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff4815aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c1407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff4815aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c15ceb0 0x7ff47c0b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff4815aa640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c0cc500 con 0x7ff47c145d70 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff4815aa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c15ca20 con 0x7ff47c15ceb0 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff4815aa640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c0ccf30 con 0x7ff47c0ce490 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c1407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c1407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36564/0 (socket says 172.21.15.66:36564) 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3133151057 learned_addr learned my addr 172.21.15.66:0/3133151057 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c15ceb0 0x7ff47c0b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47affd640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c0ce490 0x7ff47c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c0ce490 msgr2=0x7ff47c153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c0ce490 0x7ff47c153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c15ceb0 msgr2=0x7ff47c0b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.300+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c15ceb0 0x7ff47c0b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3133151057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c153d40 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c1407d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c430 tx=0x7ff46c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff4737fe640 1 -- 172.21.15.66:0/3133151057 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0074e0 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff4737fe640 1 -- 172.21.15.66:0/3133151057 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c0047e0 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c0b6680 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff4737fe640 1 -- 172.21.15.66:0/3133151057 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c004bd0 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.301+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c0b5ba0 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.302+0000 7ff4737fe640 1 -- 172.21.15.66:0/3133151057 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff46c011070 con 0x7ff47c145d70 2024-03-30T13:12:44.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.303+0000 7ff4737fe640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff45c06c530 0x7ff45c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.303+0000 7ff4737fe640 1 -- 172.21.15.66:0/3133151057 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7ff46c08b480 con 0x7ff47c145d70 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.303+0000 7ff47affd640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff45c06c530 0x7ff45c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.303+0000 7ff47affd640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff45c06c530 0x7ff45c06e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff464036000 tx=0x7ff464008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.304+0000 7ff48418a740 1 Processor -- start 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 -- start start 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c514510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 0x561b2c544300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c514a50 0x561b2c544840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561b2c47e590 con 0x561b2c541950 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b2c41cd30 con 0x561b2c541570 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561b2c47a540 con 0x561b2c514a50 2024-03-30T13:12:44.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 0x561b2c544300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff47a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 0x561b2c544300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36568/0 (socket says 172.21.15.66:36568) 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.305+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3579599946 learned_addr learned my addr 172.21.15.66:0/3579599946 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47affd640 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c514510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c514a50 0x561b2c544840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c514a50 msgr2=0x561b2c544840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c514a50 0x561b2c544840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 msgr2=0x561b2c514510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c514510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47a7fc640 1 -- 172.21.15.66:0/3579599946 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2c25e920 con 0x561b2c541950 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 0x561b2c544300 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff46c007fd0 tx=0x7ff46c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b7a00211f916cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff470ff9640 1 -- 172.21.15.66:0/3579599946 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c059240 con 0x561b2c541950 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff470ff9640 1 -- 172.21.15.66:0/3579599946 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c05d5f0 con 0x561b2c541950 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.306+0000 7ff470ff9640 1 -- 172.21.15.66:0/3579599946 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c061e20 con 0x561b2c541950 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 -- 172.21.15.66:0/3579599946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 msgr2=0x561b2c544300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 0x561b2c544300 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff46c007fd0 tx=0x7ff46c00f000 comp rx=0 tx=0).stop 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 -- 172.21.15.66:0/3579599946 shutdown_connections 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c514a50 0x561b2c544840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541950 0x561b2c544300 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 --2- 172.21.15.66:0/3579599946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c514510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 -- 172.21.15.66:0/3579599946 >> 172.21.15.66:0/3579599946 conn(0x561b2c516780 msgr2=0x561b2c55f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 -- 172.21.15.66:0/3579599946 shutdown_connections 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 -- 172.21.15.66:0/3579599946 wait complete. 2024-03-30T13:12:44.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.307+0000 7ff48418a740 1 -- start start 2024-03-30T13:12:44.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 Processor -- start 2024-03-30T13:12:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c5442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c570320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c539930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561b2c518e20 con 0x561b2c542c60 2024-03-30T13:12:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b2c41a9f0 con 0x561b2c541570 2024-03-30T13:12:44.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561b2c51b660 con 0x561b2c541950 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff48418a740 10 client.0 _ll_register_callbacks cb 0x561b2c41ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.308+0000 7ff47b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c539930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c539930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36576/0 (socket says 172.21.15.66:36576) 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 -- 172.21.15.66:0/413548895 learned_addr learned my addr 172.21.15.66:0/413548895 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47affd640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c5442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c570320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 -- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 msgr2=0x561b2c570320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c570320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 -- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 msgr2=0x561b2c5442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c5442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 -- 172.21.15.66:0/413548895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2c152210 con 0x561b2c542c60 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c570320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c539930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff474007da0 tx=0x7ff474013620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992085df67ea79b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff459ffb640 1 -- 172.21.15.66:0/413548895 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47401d020 con 0x561b2c542c60 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff459ffb640 1 -- 172.21.15.66:0/413548895 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7ff474019480 con 0x561b2c542c60 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff46c05d5f0 con 0x561b2c542c60 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff459ffb640 1 -- 172.21.15.66:0/413548895 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff474027050 con 0x561b2c542c60 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff48418a740 10 client.6585 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.309+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x561b2c2bf0b0 con 0x561b2c542c60 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff459ffb640 1 -- 172.21.15.66:0/413548895 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c05d5f0 con 0x561b2c542c60 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff459ffb640 1 -- 172.21.15.66:0/413548895 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7ff47402a020 con 0x561b2c542c60 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff48418a740 10 client.6585 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff48418a740 10 client.6585 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff48418a740 20 client.6585 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff48418a740 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c570cb0 0x561b2c5710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff48418a740 4 client.6585 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2c48bad0 con 0x561b2c570cb0 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff47affd640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c570cb0 0x561b2c5710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff459ffb640 10 client.6585 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.310+0000 7ff47affd640 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c570cb0 0x561b2c5710d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9f9ab07c4edecf server_cookie=2496eab8cdbceba9 in_seq=0 out_seq=0 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff459ffb640 1 -- 172.21.15.66:0/413548895 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x561b2c48bad0 con 0x561b2c570cb0 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff459ffb640 10 client.6585 handle_command_reply: tid=0 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 client.6585 shutdown 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 msgr2=0x561b2c539930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c539930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff474007da0 tx=0x7ff474013620 comp rx=0 tx=0).stop 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 shutdown_connections 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c570cb0 0x561b2c5710d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c539930 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c570320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 --2- 172.21.15.66:0/413548895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541570 0x561b2c5442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 >> 172.21.15.66:0/413548895 conn(0x561b2c514a50 msgr2=0x561b2c562810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff459ffb640 0 client.6585 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 shutdown_connections 2024-03-30T13:12:44.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 1 -- 172.21.15.66:0/413548895 wait complete. 2024-03-30T13:12:44.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.311+0000 7ff48418a740 20 client.6585 trim_cache size 0 max 16384 2024-03-30T13:12:44.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 Processor -- start 2024-03-30T13:12:44.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 -- start start 2024-03-30T13:12:44.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541570 0x561b2c562f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541950 0x561b2c540ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c570cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c201320 con 0x561b2c542c60 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c200320 con 0x561b2c541950 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c1562a0 con 0x561b2c541570 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff47b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c570cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff47b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c570cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36586/0 (socket says 172.21.15.66:36586) 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.326+0000 7ff47b7fe640 1 -- 172.21.15.66:0/583789898 learned_addr learned my addr 172.21.15.66:0/583789898 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541950 0x561b2c540ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47affd640 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541570 0x561b2c562f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47b7fe640 1 -- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541570 msgr2=0x561b2c562f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541570 0x561b2c562f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47b7fe640 1 -- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541950 msgr2=0x561b2c540ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541950 0x561b2c540ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47b7fe640 1 -- 172.21.15.66:0/583789898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2c12b2d0 con 0x561b2c542c60 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541950 0x561b2c540ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c570cb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff4740227b0 tx=0x7ff474019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6c4490aa627215 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff45bfff640 1 -- 172.21.15.66:0/583789898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474030070 con 0x561b2c542c60 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff45bfff640 1 -- 172.21.15.66:0/583789898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff474022990 con 0x561b2c542c60 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.327+0000 7ff45bfff640 1 -- 172.21.15.66:0/583789898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47401d040 con 0x561b2c542c60 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 -- 172.21.15.66:0/583789898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 msgr2=0x561b2c570cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c570cb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff4740227b0 tx=0x7ff474019000 comp rx=0 tx=0).stop 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 -- 172.21.15.66:0/583789898 shutdown_connections 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c542c60 0x561b2c570cb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c541950 0x561b2c540ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 --2- 172.21.15.66:0/583789898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541570 0x561b2c562f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 -- 172.21.15.66:0/583789898 >> 172.21.15.66:0/583789898 conn(0x561b2c514a50 msgr2=0x561b2c582490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 -- 172.21.15.66:0/583789898 shutdown_connections 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 -- 172.21.15.66:0/583789898 wait complete. 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 -- start start 2024-03-30T13:12:44.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.328+0000 7ff48418a740 1 Processor -- start 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 0x561b2c5dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c545300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c542c60 0x561b2c545840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1ffcd0 con 0x561b2c541570 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b2c5db770 con 0x561b2c542c60 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x561b2c518b20 con 0x561b2c541950 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff48418a740 10 client.0 _ll_register_callbacks cb 0x7ff474009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 0x561b2c5dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 0x561b2c5dac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36602/0 (socket says 172.21.15.66:36602) 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 -- 172.21.15.66:0/3138085523 learned_addr learned my addr 172.21.15.66:0/3138085523 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c542c60 0x561b2c545840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c545300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 -- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 msgr2=0x561b2c545300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c545300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 -- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c542c60 msgr2=0x561b2c545840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c542c60 0x561b2c545840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47affd640 1 -- 172.21.15.66:0/3138085523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2c578700 con 0x561b2c541570 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.329+0000 7ff47b7fe640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c542c60 0x561b2c545840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff47affd640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 0x561b2c5dac30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3c58fefefe9427 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff459ffb640 1 -- 172.21.15.66:0/3138085523 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464003ca0 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff459ffb640 1 -- 172.21.15.66:0/3138085523 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7ff464058e00 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff474022990 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff459ffb640 1 -- 172.21.15.66:0/3138085523 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff464003e40 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 10 client.6591 fetch_fsmap learned FSMap version 19 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x561b2c5db170 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff459ffb640 1 -- 172.21.15.66:0/3138085523 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff474022990 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff459ffb640 1 -- 172.21.15.66:0/3138085523 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7ff464031070 con 0x561b2c541570 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 10 client.6591 fetch_fsmap finished waiting for FSMap version 19 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 10 client.6591 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-30T13:12:44.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 20 client.6591 populate_metadata read hostname 'smithi066' 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c56d9d0 0x561b2c518fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 4 client.6591 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628412}] 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 --> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628412}) -- 0x561b2c485e20 con 0x561b2c56d9d0 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.330+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c56d9d0 0x561b2c518fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff459ffb640 10 client.6591 ms_handle_connect on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff47a7fc640 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c56d9d0 0x561b2c518fb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab55f656841e442 server_cookie=bd6daf9ae293e34d in_seq=0 out_seq=0 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff459ffb640 1 -- 172.21.15.66:0/3138085523 <== mds.0 v2:172.21.15.66:6835/1537031567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3983 (crc 0 0 0) 0x561b2c485e20 con 0x561b2c56d9d0 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff459ffb640 10 client.6591 handle_command_reply: tid=0 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 client.6591 shutdown 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 msgr2=0x561b2c5dac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 0x561b2c5dac30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464004730 comp rx=0 tx=0).stop 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 shutdown_connections 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:6835/1537031567,v1:172.21.15.66:6837/1537031567] conn(0x561b2c56d9d0 0x561b2c518fb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b2c542c60 0x561b2c545840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561b2c541950 0x561b2c545300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff459ffb640 0 client.6591 ms_handle_reset on v2:172.21.15.66:6835/1537031567 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 --2- 172.21.15.66:0/3138085523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c541570 0x561b2c5dac30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 >> 172.21.15.66:0/3138085523 conn(0x561b2c514a50 msgr2=0x561b2c570cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 shutdown_connections 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 1 -- 172.21.15.66:0/3138085523 wait complete. 2024-03-30T13:12:44.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.331+0000 7ff48418a740 20 client.6591 trim_cache size 0 max 16384 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/linux-5.4/fs/orangefs/downcall.h", 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628412, 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout: "rdev": 0, 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout: "ctime": "2024-03-30T13:12:08.137430+0000", 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout: "btime": "2024-03-30T13:12:06.640818+0000", 2024-03-30T13:12:44.344 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": 33188, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "uid": 1000, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 1301, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "nlink": 1, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_layout": { 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_hash": 0, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "unused1": 0, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "unused2": 0, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "unused3": 0 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "layout": { 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_unit": 4194304, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "stripe_count": 1, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "object_size": 4194304, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_id": 6, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ns": "" 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "old_pools": [], 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 2953, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_seq": 1, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_from": 0, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "truncate_pending": 0, 2024-03-30T13:12:44.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "atime": "2024-03-30T13:12:06.640818+0000", 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time_warp_seq": 1, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 2, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "export_pin": -1, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "quiesce_block": false, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "client_ranges": [], 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "dirstat": { 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "mtime": "0.000000", 2024-03-30T13:12:44.347 INFO:teuthology.orchestra.run.smithi066.stdout: "num_files": 0, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "num_subdirs": 0, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "change_attr": 0 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "rstat": { 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2953, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.137430+0000" 2024-03-30T13:12:44.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "accounted_rstat": { 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 0, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "rbytes": 2953, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "rfiles": 1, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "rsubdirs": 0, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "rsnaps": 0, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "rctime": "2024-03-30T13:12:08.137430+0000" 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "version": 188, 2024-03-30T13:12:44.349 INFO:teuthology.orchestra.run.smithi066.stdout: "file_data_version": 0, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "xattr_version": 1, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "backtrace_version": 14, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "inline_data_length": 0, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "stray_prior_path": "", 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "max_size_ever": 0, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "quota": { 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "max_bytes": 0, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "max_files": 0 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_stamp": "2024-03-30T13:12:08.228337+0000", 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "last_scrub_version": 118, 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "symlink": "", 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrs": [ 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "security.selinux", 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtree": { 2024-03-30T13:12:44.351 INFO:teuthology.orchestra.run.smithi066.stdout: "splits": [] 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "old_inodes": [], 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "oldest_snap": 6, 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_flags": 0, 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "is_auth": true, 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_state": { 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "replicas": {} 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_state": { 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: "authority": [ 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: 0, 2024-03-30T13:12:44.352 INFO:teuthology.orchestra.run.smithi066.stdout: -2 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "replica_nonce": 0 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_pins": 0, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "is_frozen": false, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "is_freezing": false, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "pins": { 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "ptrwaiter": 0, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "lock": 0, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 1, 2024-03-30T13:12:44.353 INFO:teuthology.orchestra.run.smithi066.stdout: "needsnapflush": 0, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent": 1, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty": 1, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "waiter": 0, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "authpin": 0, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubqueue": 0 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "nref": 3, 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "versionlock": { 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iversion", 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "authlock": {}, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "linklock": {}, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "dirfragtreelock": {}, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "filelock": {}, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "xattrlock": {}, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "snaplock": {}, 2024-03-30T13:12:44.355 INFO:teuthology.orchestra.run.smithi066.stdout: "nestlock": { 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "inest", 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "flocklock": {}, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "policylock": {}, 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "quiescelock": { 2024-03-30T13:12:44.356 INFO:teuthology.orchestra.run.smithi066.stdout: "gather_set": [], 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "lock", 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "iquiesce", 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "is_leased": false, 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rdlocks": 0, 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "num_wrlocks": 0, 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "num_xlocks": 0, 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "xlock_by": {} 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "states": [ 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "auth", 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "dirty", 2024-03-30T13:12:44.357 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtyparent", 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "dirtypool" 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "client_caps": [ 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "client_id": 5120, 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "pending": "pAsLsXsFsc", 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "issued": "pAsLsXsFsc", 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "wanted": "-", 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "last_sent": 6 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "loner": 5120, 2024-03-30T13:12:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: "want_loner": -1, 2024-03-30T13:12:44.359 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_caps_wanted": [], 2024-03-30T13:12:44.359 INFO:teuthology.orchestra.run.smithi066.stdout: "is_subvolume": false 2024-03-30T13:12:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:12:44.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff45c06c530 msgr2=0x7ff45c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff45c06c530 0x7ff45c06e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff464036000 tx=0x7ff464008030 comp rx=0 tx=0).stop 2024-03-30T13:12:44.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 msgr2=0x7ff47c1407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c1407d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c430 tx=0x7ff46c00c8f0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 shutdown_connections 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7ff45c06c530 0x7ff45c06e9f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff47c15ceb0 0x7ff47c0b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c145d70 0x7ff47c1407d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 --2- 172.21.15.66:0/3133151057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff47c0ce490 0x7ff47c153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 >> 172.21.15.66:0/3133151057 conn(0x7ff47c0b3e70 msgr2=0x7ff47c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 shutdown_connections 2024-03-30T13:12:44.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.332+0000 7ff4815aa640 1 -- 172.21.15.66:0/3133151057 wait complete. 2024-03-30T13:12:44.362 DEBUG:tasks.cephfs.test_uninlining:removing dir linux-5.4 2024-03-30T13:12:44.362 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && rm -rf linux-5.4/ 2024-03-30T13:12:44.535 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_length == 0 2024-03-30T13:12:44.535 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_version == CEPH_INLINE_NONE 2024-03-30T13:12:44.535 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: clear cache before scrub, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-30T13:12:44.535 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-30T13:12:44.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-30T13:12:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.642+0000 7f8397b24640 1 Processor -- start 2024-03-30T13:12:44.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.642+0000 7f8397b24640 1 -- start start 2024-03-30T13:12:44.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f8397b24640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f83900cecc0 0x7f83900b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f8397b24640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83900cf860 con 0x7f83900b3760 2024-03-30T13:12:44.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f8397b24640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83900cfb40 con 0x7f83900d13d0 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f8397b24640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f83900d42b0 con 0x7f83900cecc0 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f83960c1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f83900cecc0 0x7f83900b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f83950bf640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f83900b3760 0x7f83900d3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37806/0 (socket says 172.21.15.66:37806) 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.643+0000 7f83950bf640 1 -- 172.21.15.66:0/4183911443 learned_addr learned my addr 172.21.15.66:0/4183911443 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.644+0000 7f83960c1640 1 -- 172.21.15.66:0/4183911443 >> v1:172.21.15.117:6789/0 conn(0x7f83900d13d0 legacy=0x7f83900d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.644+0000 7f83960c1640 1 -- 172.21.15.66:0/4183911443 >> v1:172.21.15.66:6789/0 conn(0x7f83900b3760 legacy=0x7f83900d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.644+0000 7f83960c1640 1 -- 172.21.15.66:0/4183911443 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900d33a0 con 0x7f83900cecc0 2024-03-30T13:12:44.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.644+0000 7f83948be640 1 -- 172.21.15.66:0/4183911443 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2529730990 0 0) 0x7f83900cf860 con 0x7f83900b3760 2024-03-30T13:12:44.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.644+0000 7f83960c1640 1 --2- 172.21.15.66:0/4183911443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900cecc0 0x7f83900b7740 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8388009d90 tx=0x7f838802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f399e9162cda26c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f83948be640 1 -- 172.21.15.66:0/4183911443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388039070 con 0x7f83900cecc0 2024-03-30T13:12:44.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f83948be640 1 -- 172.21.15.66:0/4183911443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838802cc60 con 0x7f83900cecc0 2024-03-30T13:12:44.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f83948be640 1 -- 172.21.15.66:0/4183911443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388034660 con 0x7f83900cecc0 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 -- 172.21.15.66:0/4183911443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900cecc0 msgr2=0x7f83900b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 --2- 172.21.15.66:0/4183911443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900cecc0 0x7f83900b7740 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8388009d90 tx=0x7f838802beb0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 -- 172.21.15.66:0/4183911443 shutdown_connections 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 --2- 172.21.15.66:0/4183911443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900cecc0 0x7f83900b7740 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 -- 172.21.15.66:0/4183911443 >> 172.21.15.66:0/4183911443 conn(0x7f8390067710 msgr2=0x7f8390067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 -- 172.21.15.66:0/4183911443 shutdown_connections 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.645+0000 7f8397b24640 1 -- 172.21.15.66:0/4183911443 wait complete. 2024-03-30T13:12:44.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.646+0000 7f8397b24640 1 Processor -- start 2024-03-30T13:12:44.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.646+0000 7f8397b24640 1 -- start start 2024-03-30T13:12:44.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f8397b24640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 0x7f8390159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f8397b24640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83900cecc0 0x7f8390161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.652 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f8397b24640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83900d13d0 0x7f83901554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f8397b24640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f839015e930 con 0x7f83900b3760 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f8397b24640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83900d3090 con 0x7f83900d13d0 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f8397b24640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f839015eab0 con 0x7f83900cecc0 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f83958c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 0x7f8390159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f83958c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 0x7f8390159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36620/0 (socket says 172.21.15.66:36620) 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.647+0000 7f83958c0640 1 -- 172.21.15.66:0/3938090406 learned_addr learned my addr 172.21.15.66:0/3938090406 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83950bf640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83900cecc0 0x7f8390161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83960c1640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83900d13d0 0x7f83901554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83958c0640 1 -- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83900cecc0 msgr2=0x7f8390161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83958c0640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83900cecc0 0x7f8390161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83958c0640 1 -- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83900d13d0 msgr2=0x7f83901554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83958c0640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83900d13d0 0x7f83901554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83958c0640 1 -- 172.21.15.66:0/3938090406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839015fdc0 con 0x7f83900b3760 2024-03-30T13:12:44.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83958c0640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 0x7f8390159860 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f838c00b420 tx=0x7f838c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c011830 con 0x7f83900b3760 2024-03-30T13:12:44.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c016070 con 0x7f83900b3760 2024-03-30T13:12:44.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c011c10 con 0x7f83900b3760 2024-03-30T13:12:44.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8390160080 con 0x7f83900b3760 2024-03-30T13:12:44.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.648+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8390160360 con 0x7f83900b3760 2024-03-30T13:12:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.649+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390075890 con 0x7f83900b3760 2024-03-30T13:12:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.650+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f838c01b460 con 0x7f83900b3760 2024-03-30T13:12:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.650+0000 7f83867fc640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f837406c530 0x7f837406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.650+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6202+0+0 (secure 0 0 0) 0x7f838c08b170 con 0x7f83900b3760 2024-03-30T13:12:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.650+0000 7f83950bf640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f837406c530 0x7f837406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:44.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.651+0000 7f83950bf640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f837406c530 0x7f837406e9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83901f7bd0 tx=0x7f8380007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:44.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.654+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f838c020020 con 0x7f83900b3760 2024-03-30T13:12:44.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:44.879+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f8390161f20 con 0x7f83900b3760 2024-03-30T13:12:45.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.047+0000 7f83867fc640 1 -- 172.21.15.66:0/3938090406 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v20) ==== 105+0+0 (secure 0 0 0) 0x7f838c058fe0 con 0x7f83900b3760 2024-03-30T13:12:45.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f837406c530 msgr2=0x7f837406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:45.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f837406c530 0x7f837406e9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83901f7bd0 tx=0x7f8380007d50 comp rx=0 tx=0).stop 2024-03-30T13:12:45.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 msgr2=0x7f8390159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:45.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 0x7f8390159860 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f838c00b420 tx=0x7f838c00b8e0 comp rx=0 tx=0).stop 2024-03-30T13:12:45.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 shutdown_connections 2024-03-30T13:12:45.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f837406c530 0x7f837406e9f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83900d13d0 0x7f83901554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83900cecc0 0x7f8390161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 --2- 172.21.15.66:0/3938090406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83900b3760 0x7f8390159860 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 >> 172.21.15.66:0/3938090406 conn(0x7f8390067710 msgr2=0x7f83900d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 shutdown_connections 2024-03-30T13:12:45.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:45.051+0000 7f8397b24640 1 -- 172.21.15.66:0/3938090406 wait complete. 2024-03-30T13:12:45.070 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-30T13:12:45.070 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-30T13:12:45.070 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-30T13:12:45.070 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-30T13:12:49.403 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-30T13:12:49.404 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-30T13:12:49.435 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph 2024-03-30T13:12:49.435 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2 2024-03-30T13:12:49.435 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs 2024-03-30T13:12:49.435 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-30T13:12:49.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:49.435 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-30T13:12:49.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:49.491 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-30T13:12:49.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:49.547 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-30T13:12:49.602 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-30T13:12:49.602 DEBUG:teuthology.orchestra.run.smithi066:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-30T13:12:49.668 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:12:49.668 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/Makefile 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/io.h 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/Makefile 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/acl.h 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:12:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:12:49.670 INFO:teuthology.orchestra.run.smithi066.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:12:49.670 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-30T13:12:49.671 DEBUG:tasks.cephfs.test_uninlining:clearing cache 2024-03-30T13:12:49.671 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T13:12:49.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:12:49.759 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:12:49.811 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T13:12:49.811 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:12:49.811 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:12:49.811 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:49.827 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:12:49.827 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:49.882 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:50.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.024+0000 7fe7011e2640 1 Processor -- start 2024-03-30T13:12:50.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.024+0000 7fe7011e2640 1 -- start start 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe7011e2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6fc0d4a90 0x7fe6fc0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe7011e2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6fc0d2f00 0x7fe6fc0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe7011e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6fc0d36f0 con 0x7fe6fc15e950 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe7011e2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc0d3cb0 con 0x7fe6fc0d2f00 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe7011e2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc0d39d0 con 0x7fe6fc0d4a90 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe6fb59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6fc0d2f00 0x7fe6fc0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe6fb59e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe6fc0d2f00 0x7fe6fc0b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36634/0 (socket says 172.21.15.66:36634) 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.025+0000 7fe6fb59e640 1 -- 172.21.15.66:0/1051154283 learned_addr learned my addr 172.21.15.66:0/1051154283 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.026+0000 7fe6fa59c640 1 --2- 172.21.15.66:0/1051154283 >> v2:172.21.15.117:3300/0 conn(0x7fe6fc0d4a90 0x7fe6fc0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.026+0000 7fe6fb59e640 1 -- 172.21.15.66:0/1051154283 >> v2:172.21.15.117:3300/0 conn(0x7fe6fc0d4a90 msgr2=0x7fe6fc0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.026+0000 7fe6fb59e640 1 --2- 172.21.15.66:0/1051154283 >> v2:172.21.15.117:3300/0 conn(0x7fe6fc0d4a90 0x7fe6fc0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.026+0000 7fe6fb59e640 1 -- 172.21.15.66:0/1051154283 >> v1:172.21.15.117:6789/0 conn(0x7fe6fc15e950 legacy=0x7fe6fc0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:50.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.026+0000 7fe6fb59e640 1 -- 172.21.15.66:0/1051154283 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc0b7ac0 con 0x7fe6fc0d2f00 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe6fb59e640 1 --2- 172.21.15.66:0/1051154283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc0b82c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0d3e30 tx=0x7fe6f402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4131fa12eb6b0a51 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe6f9d9b640 1 -- 172.21.15.66:0/1051154283 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f402cb90 con 0x7fe6fc0d2f00 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe6f9d9b640 1 -- 172.21.15.66:0/1051154283 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f402cd30 con 0x7fe6fc0d2f00 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe6f9d9b640 1 -- 172.21.15.66:0/1051154283 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4034610 con 0x7fe6fc0d2f00 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 -- 172.21.15.66:0/1051154283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc0d2f00 msgr2=0x7fe6fc0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 --2- 172.21.15.66:0/1051154283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc0b82c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0d3e30 tx=0x7fe6f402bff0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 -- 172.21.15.66:0/1051154283 shutdown_connections 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 --2- 172.21.15.66:0/1051154283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc0b82c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 --2- 172.21.15.66:0/1051154283 >> v2:172.21.15.117:3300/0 conn(0x7fe6fc0d4a90 0x7fe6fc0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 -- 172.21.15.66:0/1051154283 >> 172.21.15.66:0/1051154283 conn(0x7fe6fc0676e0 msgr2=0x7fe6fc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 -- 172.21.15.66:0/1051154283 shutdown_connections 2024-03-30T13:12:50.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.027+0000 7fe7011e2640 1 -- 172.21.15.66:0/1051154283 wait complete. 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.029+0000 7fe7011e2640 1 Processor -- start 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.029+0000 7fe7011e2640 1 -- start start 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.029+0000 7fe7011e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc14d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.029+0000 7fe7011e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0d4a90 0x7fe6fc161990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe7011e2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 0x7fe6fc14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe7011e2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc15ed30 con 0x7fe6fc15e950 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe7011e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc0d33b0 con 0x7fe6fc0d2f00 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe7011e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc15eeb0 con 0x7fe6fc0d4a90 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 0x7fe6fc14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 0x7fe6fc14ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36644/0 (socket says 172.21.15.66:36644) 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 -- 172.21.15.66:0/2468043269 learned_addr learned my addr 172.21.15.66:0/2468043269 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fa59c640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0d4a90 0x7fe6fc161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fad9d640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc14d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 -- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0d4a90 msgr2=0x7fe6fc161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0d4a90 0x7fe6fc161990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 -- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0d2f00 msgr2=0x7fe6fc14d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc14d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 -- 172.21.15.66:0/2468043269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc14ddb0 con 0x7fe6fc15e950 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6fb59e640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 0x7fe6fc14ee50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4036040 tx=0x7fe6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4043020 con 0x7fe6fc15e950 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f403e450 con 0x7fe6fc15e950 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.030+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc14e090 con 0x7fe6fc15e950 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.031+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4009510 con 0x7fe6fc15e950 2024-03-30T13:12:50.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.031+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc14e650 con 0x7fe6fc15e950 2024-03-30T13:12:50.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.032+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc0cb050 con 0x7fe6fc15e950 2024-03-30T13:12:50.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.032+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe6f4046020 con 0x7fe6fc15e950 2024-03-30T13:12:50.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.033+0000 7fe6f37fe640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe6dc06c530 0x7fe6dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.033+0000 7fe6fad9d640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe6dc06c530 0x7fe6dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.033+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fe6f40addf0 con 0x7fe6fc15e950 2024-03-30T13:12:50.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.033+0000 7fe6fad9d640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe6dc06c530 0x7fe6dc06e9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40016e0 tx=0x7fe6e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.037+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe6f404d030 con 0x7fe6fc15e950 2024-03-30T13:12:50.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.223+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6fc075a10 con 0x7fe6fc15e950 2024-03-30T13:12:50.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.225+0000 7fe6f37fe640 1 -- 172.21.15.66:0/2468043269 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16990 (secure 0 0 0) 0x7fe6f407bc90 con 0x7fe6fc15e950 2024-03-30T13:12:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:50.229 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":45,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:12:48.198771+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":44,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe6dc06c530 msgr2=0x7fe6dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe6dc06c530 0x7fe6dc06e9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40016e0 tx=0x7fe6e4009040 comp rx=0 tx=0).stop 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 msgr2=0x7fe6fc14ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 0x7fe6fc14ee50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4036040 tx=0x7fe6f4004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 shutdown_connections 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe6dc06c530 0x7fe6dc06e9f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6fc15e950 0x7fe6fc14ee50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6fc0d4a90 0x7fe6fc161990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 --2- 172.21.15.66:0/2468043269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6fc0d2f00 0x7fe6fc14d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 >> 172.21.15.66:0/2468043269 conn(0x7fe6fc0676e0 msgr2=0x7fe6fc0cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:50.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 shutdown_connections 2024-03-30T13:12:50.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.227+0000 7fe7011e2640 1 -- 172.21.15.66:0/2468043269 wait complete. 2024-03-30T13:12:50.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:55.516141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:55.988604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:12:50.243 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.348+0000 7f2b0a23c640 1 Processor -- start 2024-03-30T13:12:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.348+0000 7f2b0a23c640 1 -- start start 2024-03-30T13:12:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b0a23c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2b040b5dc0 0x7f2b040b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b0a23c640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b040b66e0 con 0x7f2b040ba090 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b0a23c640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b040d2ff0 con 0x7f2b040b73d0 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b0a23c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2b040d2d90 con 0x7f2b040b5dc0 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b037fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2b040b5dc0 0x7f2b040b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b02ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2b040ba090 0x7f2b040d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37808/0 (socket says 172.21.15.66:37808) 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b02ffd640 1 -- 172.21.15.66:0/36812712 learned_addr learned my addr 172.21.15.66:0/36812712 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b037fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2b040b5dc0 0x7f2b040b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36658/0 (socket says 172.21.15.66:36658) 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b027fc640 1 -- 172.21.15.66:0/36812712 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421406716 0 0) 0x7f2b040b66e0 con 0x7f2b040ba090 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b027fc640 1 -- 172.21.15.66:0/36812712 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2af0003520 con 0x7f2b040ba090 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b037fe640 1 -- 172.21.15.66:0/36812712 >> v1:172.21.15.117:6789/0 conn(0x7f2b040b73d0 legacy=0x7f2b040b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b037fe640 1 -- 172.21.15.66:0/36812712 >> v1:172.21.15.66:6789/0 conn(0x7f2b040ba090 legacy=0x7f2b040d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.349+0000 7f2b037fe640 1 -- 172.21.15.66:0/36812712 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b040d32d0 con 0x7f2b040b5dc0 2024-03-30T13:12:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b037fe640 1 --2- 172.21.15.66:0/36812712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b61a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097a0 tx=0x7f2aec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99bd7a4c66944cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b027fc640 1 -- 172.21.15.66:0/36812712 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec034070 con 0x7f2b040b5dc0 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b027fc640 1 -- 172.21.15.66:0/36812712 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec030610 con 0x7f2b040b5dc0 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b027fc640 1 -- 172.21.15.66:0/36812712 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0309e0 con 0x7f2b040b5dc0 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 -- 172.21.15.66:0/36812712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b5dc0 msgr2=0x7f2b040b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/36812712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b61a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097a0 tx=0x7f2aec02b960 comp rx=0 tx=0).stop 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 -- 172.21.15.66:0/36812712 shutdown_connections 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/36812712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b61a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 -- 172.21.15.66:0/36812712 >> 172.21.15.66:0/36812712 conn(0x7f2b040678f0 msgr2=0x7f2b040d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 -- 172.21.15.66:0/36812712 shutdown_connections 2024-03-30T13:12:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.351+0000 7f2b0a23c640 1 -- 172.21.15.66:0/36812712 wait complete. 2024-03-30T13:12:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.352+0000 7f2b0a23c640 1 Processor -- start 2024-03-30T13:12:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 -- start start 2024-03-30T13:12:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 0x7f2b0415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b040ba090 0x7f2b041f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2b040d3b70 con 0x7f2b040b73d0 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b040d3890 con 0x7f2b040b5dc0 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b0a23c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2b040d4090 con 0x7f2b040ba090 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.353+0000 7f2b02ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 0x7f2b0415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 0x7f2b0415e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36672/0 (socket says 172.21.15.66:36672) 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 -- 172.21.15.66:0/716414203 learned_addr learned my addr 172.21.15.66:0/716414203 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 -- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b040ba090 msgr2=0x7f2b041f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b040ba090 0x7f2b041f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b037fe640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 -- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b040b5dc0 msgr2=0x7f2b040b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 -- 172.21.15.66:0/716414203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0415f800 con 0x7f2b040b73d0 2024-03-30T13:12:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b02ffd640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 0x7f2b0415e880 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2af400c410 tx=0x7f2af400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4017070 con 0x7f2b040b73d0 2024-03-30T13:12:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af4005ce0 con 0x7f2b040b73d0 2024-03-30T13:12:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0415f240 con 0x7f2b040b73d0 2024-03-30T13:12:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af40072f0 con 0x7f2b040b73d0 2024-03-30T13:12:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.354+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b04160000 con 0x7f2b040b73d0 2024-03-30T13:12:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.355+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0415f630 con 0x7f2b040b73d0 2024-03-30T13:12:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.356+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2af401a070 con 0x7f2b040b73d0 2024-03-30T13:12:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.356+0000 7f2afbfff640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ae406c560 0x7f2ae406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.356+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f2af408ac60 con 0x7f2b040b73d0 2024-03-30T13:12:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.356+0000 7f2b037fe640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ae406c560 0x7f2ae406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.357+0000 7f2b037fe640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ae406c560 0x7f2ae406ea20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2aec036000 tx=0x7f2aec03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.361+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2af4059210 con 0x7f2b040b73d0 2024-03-30T13:12:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.545+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b04075890 con 0x7f2b040b73d0 2024-03-30T13:12:50.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.545+0000 7f2afbfff640 1 -- 172.21.15.66:0/716414203 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f2af4058b10 con 0x7f2b040b73d0 2024-03-30T13:12:50.549 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:12:50.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ae406c560 msgr2=0x7f2ae406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ae406c560 0x7f2ae406ea20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2aec036000 tx=0x7f2aec03b040 comp rx=0 tx=0).stop 2024-03-30T13:12:50.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 msgr2=0x7f2b0415e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 0x7f2b0415e880 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2af400c410 tx=0x7f2af400c8d0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 shutdown_connections 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2ae406c560 0x7f2ae406ea20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b040ba090 0x7f2b041f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b040b73d0 0x7f2b0415e880 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 --2- 172.21.15.66:0/716414203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b040b5dc0 0x7f2b040b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 >> 172.21.15.66:0/716414203 conn(0x7f2b040678f0 msgr2=0x7f2b040d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 shutdown_connections 2024-03-30T13:12:50.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.547+0000 7f2b0a23c640 1 -- 172.21.15.66:0/716414203 wait complete. 2024-03-30T13:12:50.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:50.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.673+0000 7f35f2159640 1 Processor -- start 2024-03-30T13:12:50.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.673+0000 7f35f2159640 1 -- start start 2024-03-30T13:12:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.673+0000 7f35f2159640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 0x7f35ec0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.673+0000 7f35f2159640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ec0d2ff0 con 0x7f35ec0b5dc0 2024-03-30T13:12:50.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.673+0000 7f35f2159640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ec0d32d0 con 0x7f35ec0b73d0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.673+0000 7f35f2159640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f35ec0d35b0 con 0x7f35ec0ba090 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35eb7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f35ec0b5dc0 0x7f35ec0b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37812/0 (socket says 172.21.15.66:37812) 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35eb7fe640 1 -- 172.21.15.66:0/4014593704 learned_addr learned my addr 172.21.15.66:0/4014593704 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35eaffd640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 0x7f35ec0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69469135 0 0) 0x7f35ec0d2ff0 con 0x7f35ec0b5dc0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35d4003520 con 0x7f35ec0b5dc0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668892863 0 0) 0x7f35ec0d32d0 con 0x7f35ec0b73d0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35ec0d2ff0 con 0x7f35ec0b73d0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 488916917 0 0) 0x7f35d4003520 con 0x7f35ec0b5dc0 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 msgr2=0x7f35ec0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:50.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 0x7f35ec0d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 >> v1:172.21.15.117:6789/0 conn(0x7f35ec0b73d0 legacy=0x7f35ec0b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec0d3890 con 0x7f35ec0b5dc0 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f35dc001670 con 0x7f35ec0b5dc0 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 >> v1:172.21.15.66:6789/0 conn(0x7f35ec0b5dc0 legacy=0x7f35ec0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4003b70 0x7f35d4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 0x7f35d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d4004d40 0x7f35d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0d35b0 con 0x7f35d4004d40 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35d400ba70 con 0x7f35d4003b70 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.674+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f35d400bc70 con 0x7f35ec0ba090 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35ebfff640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d4004d40 0x7f35d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35eb7fe640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4003b70 0x7f35d4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35eaffd640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 0x7f35d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35ebfff640 1 -- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 msgr2=0x7f35d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35ebfff640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0ba090 0x7f35d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35ebfff640 1 -- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4003b70 msgr2=0x7f35d4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35ebfff640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4003b70 0x7f35d4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.675+0000 7f35ebfff640 1 -- 172.21.15.66:0/4014593704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d4003520 con 0x7f35d4004d40 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35ebfff640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d4004d40 0x7f35d40051c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f35e4001fd0 tx=0x7f35d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01258f2c8d5d7a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4001bf0 con 0x7f35d4004d40 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e4003b70 con 0x7f35d4004d40 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35ea7fc640 1 -- 172.21.15.66:0/4014593704 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40050d0 con 0x7f35d4004d40 2024-03-30T13:12:50.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35f2159640 1 -- 172.21.15.66:0/4014593704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d4004d40 msgr2=0x7f35d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35f2159640 1 --2- 172.21.15.66:0/4014593704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d4004d40 0x7f35d40051c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f35e4001fd0 tx=0x7f35d4005830 comp rx=0 tx=0).stop 2024-03-30T13:12:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35ebfff640 1 -- 172.21.15.66:0/4014593704 reap_dead start 2024-03-30T13:12:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35f2159640 1 -- 172.21.15.66:0/4014593704 shutdown_connections 2024-03-30T13:12:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35f2159640 1 -- 172.21.15.66:0/4014593704 >> 172.21.15.66:0/4014593704 conn(0x7f35ec0678f0 msgr2=0x7f35ec0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35f2159640 1 -- 172.21.15.66:0/4014593704 shutdown_connections 2024-03-30T13:12:50.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.676+0000 7f35f2159640 1 -- 172.21.15.66:0/4014593704 wait complete. 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.677+0000 7f35f2159640 1 Processor -- start 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 -- start start 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0c9fe0 0x7f35ec15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 0x7f35ec0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ec0b8930 0x7f35ec0b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec15e880 con 0x7f35ec160200 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0d4090 con 0x7f35ec0b8930 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35f2159640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f35ec15ea00 con 0x7f35ec0c9fe0 2024-03-30T13:12:50.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35eaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 0x7f35ec0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.678+0000 7f35eaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 0x7f35ec0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36702/0 (socket says 172.21.15.66:36702) 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 -- 172.21.15.66:0/222021258 learned_addr learned my addr 172.21.15.66:0/222021258 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eb7fe640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0c9fe0 0x7f35ec15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35ebfff640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ec0b8930 0x7f35ec0b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 -- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0c9fe0 msgr2=0x7f35ec15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0c9fe0 0x7f35ec15eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 -- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ec0b8930 msgr2=0x7f35ec0b8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ec0b8930 0x7f35ec0b8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 -- 172.21.15.66:0/222021258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec15f160 con 0x7f35ec160200 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35eaffd640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 0x7f35ec0cdc20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f35e0011910 tx=0x7f35e0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0019070 con 0x7f35ec160200 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0004240 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e000f8e0 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec15f440 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.679+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec15fa00 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.680+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec075890 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.681+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f35e001c070 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.681+0000 7f35d3fff640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f35d806c530 0x7f35d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.681+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f35e008bdb0 con 0x7f35ec160200 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.681+0000 7f35eb7fe640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f35d806c530 0x7f35d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.682+0000 7f35eb7fe640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f35d806c530 0x7f35d806e9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35dc004840 tx=0x7f35dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:50.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.685+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f35e005a210 con 0x7f35ec160200 2024-03-30T13:12:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.889+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f35ec0b6590 con 0x7f35ec160200 2024-03-30T13:12:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.889+0000 7f35d3fff640 1 -- 172.21.15.66:0/222021258 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f35e0059b10 con 0x7f35ec160200 2024-03-30T13:12:50.894 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f35d806c530 msgr2=0x7f35d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f35d806c530 0x7f35d806e9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35dc004840 tx=0x7f35dc009040 comp rx=0 tx=0).stop 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 msgr2=0x7f35ec0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 0x7f35ec0cdc20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f35e0011910 tx=0x7f35e0011dd0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 shutdown_connections 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f35d806c530 0x7f35d806e9f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35ec0b8930 0x7f35ec0b8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35ec160200 0x7f35ec0cdc20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 --2- 172.21.15.66:0/222021258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35ec0c9fe0 0x7f35ec15eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:50.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.891+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 >> 172.21.15.66:0/222021258 conn(0x7f35ec0678f0 msgr2=0x7f35ec0ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:50.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.892+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 shutdown_connections 2024-03-30T13:12:50.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:50.892+0000 7f35f2159640 1 -- 172.21.15.66:0/222021258 wait complete. 2024-03-30T13:12:50.906 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:51.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.013+0000 7f85ab074640 1 Processor -- start 2024-03-30T13:12:51.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.013+0000 7f85ab074640 1 -- start start 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85ab074640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 0x7f85a40cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85ab074640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a40cfab0 con 0x7f85a40cf620 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85ab074640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a415e950 con 0x7f85a415e570 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85ab074640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f85a40b94a0 con 0x7f85a40ba0b0 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85a9611640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f85a40cf620 0x7f85a40d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37820/0 (socket says 172.21.15.66:37820) 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85a9611640 1 -- 172.21.15.66:0/3257192578 learned_addr learned my addr 172.21.15.66:0/3257192578 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85a3fff640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 0x7f85a40cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.014+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638488660 0 0) 0x7f85a40cfab0 con 0x7f85a40cf620 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8594003520 con 0x7f85a40cf620 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 408349371 0 0) 0x7f8594003520 con 0x7f85a40cf620 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 msgr2=0x7f85a40cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 0x7f85a40cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 >> v1:172.21.15.117:6789/0 conn(0x7f85a415e570 legacy=0x7f85a40b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a40b8890 con 0x7f85a40cf620 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f859c0019f0 con 0x7f85a40cf620 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 >> v1:172.21.15.66:6789/0 conn(0x7f85a40cf620 legacy=0x7f85a40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 0x7f85940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8594003f20 0x7f8594004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8594004d40 0x7f85940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85a40b94a0 con 0x7f8594003f20 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f859400ba20 con 0x7f8594004d40 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f859400bc20 con 0x7f85a40ba0b0 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a3fff640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8594003f20 0x7f8594004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a9611640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8594004d40 0x7f85940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a8e10640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 0x7f85940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a3fff640 1 -- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 msgr2=0x7f85940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:51.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a3fff640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40ba0b0 0x7f85940039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a3fff640 1 -- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8594004d40 msgr2=0x7f85940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:51.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a3fff640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8594004d40 0x7f85940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.015+0000 7f85a3fff640 1 -- 172.21.15.66:0/3257192578 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8594003520 con 0x7f8594003f20 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85a3fff640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8594003f20 0x7f8594004800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ac0 tx=0x7f8594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5689802b608e543 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590039070 con 0x7f8594003f20 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859002c600 con 0x7f8594003f20 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85a37fe640 1 -- 172.21.15.66:0/3257192578 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590009690 con 0x7f8594003f20 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85ab074640 1 -- 172.21.15.66:0/3257192578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8594003f20 msgr2=0x7f8594004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85ab074640 1 --2- 172.21.15.66:0/3257192578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8594003f20 0x7f8594004800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ac0 tx=0x7f8594005830 comp rx=0 tx=0).stop 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85a9611640 1 -- 172.21.15.66:0/3257192578 reap_dead start 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85ab074640 1 -- 172.21.15.66:0/3257192578 shutdown_connections 2024-03-30T13:12:51.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85ab074640 1 -- 172.21.15.66:0/3257192578 >> 172.21.15.66:0/3257192578 conn(0x7f85a40676e0 msgr2=0x7f85a4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85ab074640 1 -- 172.21.15.66:0/3257192578 shutdown_connections 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.017+0000 7f85ab074640 1 -- 172.21.15.66:0/3257192578 wait complete. 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.018+0000 7f85ab074640 1 Processor -- start 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 -- start start 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8594003f20 0x7f85a414b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a4154240 0x7f85a41546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 0x7f85a40cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85a40b9130 con 0x7f85a40ccee0 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85a40b8e50 con 0x7f85a4154240 2024-03-30T13:12:51.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.019+0000 7f85ab074640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f85a40cf0e0 con 0x7f8594003f20 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 0x7f85a40cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 0x7f85a40cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36726/0 (socket says 172.21.15.66:36726) 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 -- 172.21.15.66:0/84401159 learned_addr learned my addr 172.21.15.66:0/84401159 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a3fff640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a4154240 0x7f85a41546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a8e10640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8594003f20 0x7f85a414b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 -- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8594003f20 msgr2=0x7f85a414b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8594003f20 0x7f85a414b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 -- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a4154240 msgr2=0x7f85a41546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a4154240 0x7f85a41546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 -- 172.21.15.66:0/84401159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a41f42c0 con 0x7f85a40ccee0 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a8e10640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8594003f20 0x7f85a414b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a3fff640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a4154240 0x7f85a41546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a9611640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 0x7f85a40cd360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f85a40d1390 tx=0x7f859c007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c00abc0 con 0x7f85a40ccee0 2024-03-30T13:12:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c019070 con 0x7f85a40ccee0 2024-03-30T13:12:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a41f4530 con 0x7f85a40ccee0 2024-03-30T13:12:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0133f0 con 0x7f85a40ccee0 2024-03-30T13:12:51.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.020+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41f4af0 con 0x7f85a40ccee0 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.022+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f859c013590 con 0x7f85a40ccee0 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.022+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a4075890 con 0x7f85a40ccee0 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.022+0000 7f85a17fa640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f858806c560 0x7f858806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.022+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f859c08be30 con 0x7f85a40ccee0 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.022+0000 7f85a8e10640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f858806c560 0x7f858806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.023+0000 7f85a8e10640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f858806c560 0x7f858806ea20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8598003700 tx=0x7f8598009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.027+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f859c05a290 con 0x7f85a40ccee0 2024-03-30T13:12:51.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.282+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85a415a760 con 0x7f85a40ccee0 2024-03-30T13:12:51.287 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.283+0000 7f85a17fa640 1 -- 172.21.15.66:0/84401159 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f859c059b90 con 0x7f85a40ccee0 2024-03-30T13:12:51.287 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f858806c560 msgr2=0x7f858806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f858806c560 0x7f858806ea20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8598003700 tx=0x7f8598009040 comp rx=0 tx=0).stop 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 msgr2=0x7f85a40cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 0x7f85a40cd360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f85a40d1390 tx=0x7f859c007b20 comp rx=0 tx=0).stop 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 shutdown_connections 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f858806c560 0x7f858806ea20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85a40ccee0 0x7f85a40cd360 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a4154240 0x7f85a41546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 --2- 172.21.15.66:0/84401159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8594003f20 0x7f85a414b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 >> 172.21.15.66:0/84401159 conn(0x7f85a40676e0 msgr2=0x7f85a40d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:51.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 shutdown_connections 2024-03-30T13:12:51.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.285+0000 7f85ab074640 1 -- 172.21.15.66:0/84401159 wait complete. 2024-03-30T13:12:51.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:51.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.405+0000 7f3e81f40640 1 Processor -- start 2024-03-30T13:12:51.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.405+0000 7f3e81f40640 1 -- start start 2024-03-30T13:12:51.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e81f40640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3e7c0d19c0 0x7f3e7c169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e81f40640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c0d3120 con 0x7f3e7c15e7a0 2024-03-30T13:12:51.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e81f40640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c0d3400 con 0x7f3e7c0cec60 2024-03-30T13:12:51.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e81f40640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3e7c0cfc40 con 0x7f3e7c0d19c0 2024-03-30T13:12:51.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7bfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3e7c0d19c0 0x7f3e7c169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7b7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f3e7c15e7a0 0x7f3e7c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:37828/0 (socket says 172.21.15.66:37828) 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7b7fe640 1 -- 172.21.15.66:0/627931830 learned_addr learned my addr 172.21.15.66:0/627931830 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7bfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3e7c0d19c0 0x7f3e7c169140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36740/0 (socket says 172.21.15.66:36740) 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7bfff640 1 -- 172.21.15.66:0/627931830 >> v1:172.21.15.117:6789/0 conn(0x7f3e7c0cec60 legacy=0x7f3e7c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7bfff640 1 -- 172.21.15.66:0/627931830 >> v1:172.21.15.66:6789/0 conn(0x7f3e7c15e7a0 legacy=0x7f3e7c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7a7fc640 1 -- 172.21.15.66:0/627931830 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581771262 0 0) 0x7f3e7c0d3120 con 0x7f3e7c15e7a0 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.406+0000 7f3e7bfff640 1 -- 172.21.15.66:0/627931830 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c0d36e0 con 0x7f3e7c0d19c0 2024-03-30T13:12:51.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.407+0000 7f3e7bfff640 1 --2- 172.21.15.66:0/627931830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c169140 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009d90 tx=0x7f3e7402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c4ed757c722795 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.407+0000 7f3e7a7fc640 1 -- 172.21.15.66:0/627931830 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74039070 con 0x7f3e7c0d19c0 2024-03-30T13:12:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.407+0000 7f3e7a7fc640 1 -- 172.21.15.66:0/627931830 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7402cbd0 con 0x7f3e7c0d19c0 2024-03-30T13:12:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.407+0000 7f3e7a7fc640 1 -- 172.21.15.66:0/627931830 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74034440 con 0x7f3e7c0d19c0 2024-03-30T13:12:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 -- 172.21.15.66:0/627931830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d19c0 msgr2=0x7f3e7c169140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 --2- 172.21.15.66:0/627931830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c169140 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009d90 tx=0x7f3e7402be20 comp rx=0 tx=0).stop 2024-03-30T13:12:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 -- 172.21.15.66:0/627931830 shutdown_connections 2024-03-30T13:12:51.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 --2- 172.21.15.66:0/627931830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c169140 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 -- 172.21.15.66:0/627931830 >> 172.21.15.66:0/627931830 conn(0x7f3e7c0676b0 msgr2=0x7f3e7c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:51.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 -- 172.21.15.66:0/627931830 shutdown_connections 2024-03-30T13:12:51.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.408+0000 7f3e81f40640 1 -- 172.21.15.66:0/627931830 wait complete. 2024-03-30T13:12:51.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.410+0000 7f3e81f40640 1 Processor -- start 2024-03-30T13:12:51.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.410+0000 7f3e81f40640 1 -- start start 2024-03-30T13:12:51.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e81f40640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e7c0cec60 0x7f3e7c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e81f40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c1f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e81f40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 0x7f3e7c0d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e81f40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d4060 con 0x7f3e7c0d4360 2024-03-30T13:12:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e81f40640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d3ee0 con 0x7f3e7c0d19c0 2024-03-30T13:12:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e81f40640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c0d41e0 con 0x7f3e7c0cec60 2024-03-30T13:12:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 0x7f3e7c0d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 0x7f3e7c0d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36752/0 (socket says 172.21.15.66:36752) 2024-03-30T13:12:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 -- 172.21.15.66:0/836631722 learned_addr learned my addr 172.21.15.66:0/836631722 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:51.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7affd640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c1f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7b7fe640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e7c0cec60 0x7f3e7c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 -- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e7c0cec60 msgr2=0x7f3e7c0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e7c0cec60 0x7f3e7c0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 -- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0d19c0 msgr2=0x7f3e7c1f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c1f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.412+0000 7f3e7bfff640 1 -- 172.21.15.66:0/836631722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c0b65b0 con 0x7f3e7c0d4360 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.413+0000 7f3e7bfff640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 0x7f3e7c0d47f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3e74037040 tx=0x7f3e74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.413+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74044030 con 0x7f3e7c0d4360 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.413+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7402c9d0 con 0x7f3e7c0d4360 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.413+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c0b6890 con 0x7f3e7c0d4360 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.413+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74039040 con 0x7f3e7c0d4360 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.413+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7c157e80 con 0x7f3e7c0d4360 2024-03-30T13:12:51.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.414+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c158080 con 0x7f3e7c0d4360 2024-03-30T13:12:51.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.414+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3e74047070 con 0x7f3e7c0d4360 2024-03-30T13:12:51.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.415+0000 7f3e63fff640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3e6806c530 0x7f3e6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.415+0000 7f3e7b7fe640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3e6806c530 0x7f3e6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.415+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f3e740adf50 con 0x7f3e7c0d4360 2024-03-30T13:12:51.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.416+0000 7f3e7b7fe640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3e6806c530 0x7f3e6806e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000f60 tx=0x7f3e6c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.419+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3e7c158080 con 0x7f3e7c0d4360 2024-03-30T13:12:51.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.617+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3e7c075900 con 0x7f3e7c0d4360 2024-03-30T13:12:51.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.618+0000 7f3e63fff640 1 -- 172.21.15.66:0/836631722 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f3e7404e070 con 0x7f3e7c0d4360 2024-03-30T13:12:51.622 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3e6806c530 msgr2=0x7f3e6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3e6806c530 0x7f3e6806e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000f60 tx=0x7f3e6c007d50 comp rx=0 tx=0).stop 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 msgr2=0x7f3e7c0d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 0x7f3e7c0d47f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3e74037040 tx=0x7f3e74004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 shutdown_connections 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f3e6806c530 0x7f3e6806e9f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e7c0d4360 0x7f3e7c0d47f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e7c0d19c0 0x7f3e7c1f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 --2- 172.21.15.66:0/836631722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e7c0cec60 0x7f3e7c0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 >> 172.21.15.66:0/836631722 conn(0x7f3e7c0676b0 msgr2=0x7f3e7c0b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:51.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 shutdown_connections 2024-03-30T13:12:51.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.620+0000 7f3e81f40640 1 -- 172.21.15.66:0/836631722 wait complete. 2024-03-30T13:12:51.635 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-30T13:12:51.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:51.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:51.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:12:51.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:51.778 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T13:12:51.793 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:12:51.793 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:12:51.793 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:51.810 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:12:51.810 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:51.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:51.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.971+0000 7f57d777c640 1 Processor -- start 2024-03-30T13:12:51.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.971+0000 7f57d777c640 1 -- start start 2024-03-30T13:12:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d777c640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57d015e950 0x7f57d00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d777c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57d00d48c0 0x7f57d00d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d777c640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d00d36f0 con 0x7f57d00d2f00 2024-03-30T13:12:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d777c640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f57d00d3cb0 con 0x7f57d00d48c0 2024-03-30T13:12:51.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d777c640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f57d00d4d00 con 0x7f57d015e950 2024-03-30T13:12:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d4d17640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57d00d48c0 0x7f57d00d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d5d19640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f57d00d2f00 0x7f57d00cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39638/0 (socket says 172.21.15.66:39638) 2024-03-30T13:12:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d5d19640 1 -- 172.21.15.66:0/813344024 learned_addr learned my addr 172.21.15.66:0/813344024 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d5518640 1 --2- 172.21.15.66:0/813344024 >> v2:172.21.15.117:3300/0 conn(0x7f57d015e950 0x7f57d00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d4d17640 1 -- 172.21.15.66:0/813344024 >> v2:172.21.15.117:3300/0 conn(0x7f57d015e950 msgr2=0x7f57d00d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:51.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d4d17640 1 --2- 172.21.15.66:0/813344024 >> v2:172.21.15.117:3300/0 conn(0x7f57d015e950 0x7f57d00d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d4d17640 1 -- 172.21.15.66:0/813344024 >> v1:172.21.15.66:6789/0 conn(0x7f57d00d2f00 legacy=0x7f57d00cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.972+0000 7f57d4d17640 1 -- 172.21.15.66:0/813344024 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d00d39d0 con 0x7f57d00d48c0 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.973+0000 7f57d4d17640 1 --2- 172.21.15.66:0/813344024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d00d29c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f57d00d3e30 tx=0x7f57c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8be4fb969d25686 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.973+0000 7f57c7fff640 1 -- 172.21.15.66:0/813344024 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0034070 con 0x7f57d00d48c0 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.973+0000 7f57c7fff640 1 -- 172.21.15.66:0/813344024 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0030610 con 0x7f57d00d48c0 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.973+0000 7f57c7fff640 1 -- 172.21.15.66:0/813344024 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00309e0 con 0x7f57d00d48c0 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 -- 172.21.15.66:0/813344024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 msgr2=0x7f57d00d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 --2- 172.21.15.66:0/813344024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d00d29c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f57d00d3e30 tx=0x7f57c002b960 comp rx=0 tx=0).stop 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 -- 172.21.15.66:0/813344024 shutdown_connections 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 --2- 172.21.15.66:0/813344024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d00d29c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 --2- 172.21.15.66:0/813344024 >> v2:172.21.15.117:3300/0 conn(0x7f57d015e950 0x7f57d00d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 -- 172.21.15.66:0/813344024 >> 172.21.15.66:0/813344024 conn(0x7f57d00676e0 msgr2=0x7f57d0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 -- 172.21.15.66:0/813344024 shutdown_connections 2024-03-30T13:12:51.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.974+0000 7f57d777c640 1 -- 172.21.15.66:0/813344024 wait complete. 2024-03-30T13:12:51.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.975+0000 7f57d777c640 1 Processor -- start 2024-03-30T13:12:51.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.975+0000 7f57d777c640 1 -- start start 2024-03-30T13:12:51.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d777c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57d00d2f00 0x7f57d01f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d777c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d01f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d777c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57d015e950 0x7f57d015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d777c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00d3390 con 0x7f57d00d48c0 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d777c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00b7da0 con 0x7f57d00d2f00 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d777c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f57d015ed30 con 0x7f57d015e950 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d01f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d01f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51982/0 (socket says 172.21.15.66:51982) 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 -- 172.21.15.66:0/2143760544 learned_addr learned my addr 172.21.15.66:0/2143760544 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d5518640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57d00d2f00 0x7f57d01f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d5d19640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57d015e950 0x7f57d015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 -- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57d015e950 msgr2=0x7f57d015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57d015e950 0x7f57d015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 -- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57d00d2f00 msgr2=0x7f57d01f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57d00d2f00 0x7f57d01f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 -- 172.21.15.66:0/2143760544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d00c9ed0 con 0x7f57d00d48c0 2024-03-30T13:12:51.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d5518640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57d00d2f00 0x7f57d01f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d5d19640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57d015e950 0x7f57d015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.976+0000 7f57d4d17640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d01f4c10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f57c002b9a0 tx=0x7f57c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.977+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0034040 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.977+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0030410 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.977+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d00ca1b0 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.977+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0039c30 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.977+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d00ca770 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.978+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d0075a10 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.979+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f57c0037070 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.979+0000 7f57c5ffb640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f57b406c530 0x7f57b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.979+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f57c00ad820 con 0x7f57d00d48c0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.979+0000 7f57d5518640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f57b406c530 0x7f57b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:51.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.980+0000 7f57d5518640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f57b406c530 0x7f57b406e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f57d00ce2f0 tx=0x7f57cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:51.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:51.986+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f57c007bc90 con 0x7f57d00d48c0 2024-03-30T13:12:52.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.173+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57d00b5c90 con 0x7f57d00d48c0 2024-03-30T13:12:52.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.174+0000 7f57c5ffb640 1 -- 172.21.15.66:0/2143760544 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16990 (secure 0 0 0) 0x7f57c007b590 con 0x7f57d00d48c0 2024-03-30T13:12:52.179 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:52.179 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":45,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:12:48.198771+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":44,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:52.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f57b406c530 msgr2=0x7f57b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f57b406c530 0x7f57b406e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f57d00ce2f0 tx=0x7f57cc008040 comp rx=0 tx=0).stop 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 msgr2=0x7f57d01f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d01f4c10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f57c002b9a0 tx=0x7f57c00046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 shutdown_connections 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f57b406c530 0x7f57b406e9f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57d015e950 0x7f57d015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d00d48c0 0x7f57d01f4c10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 --2- 172.21.15.66:0/2143760544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57d00d2f00 0x7f57d01f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 >> 172.21.15.66:0/2143760544 conn(0x7f57d00676e0 msgr2=0x7f57d00cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 shutdown_connections 2024-03-30T13:12:52.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.176+0000 7f57d777c640 1 -- 172.21.15.66:0/2143760544 wait complete. 2024-03-30T13:12:52.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:55.516141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:55.988604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:12:52.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:52.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.298+0000 7f5fed9e2640 1 Processor -- start 2024-03-30T13:12:52.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fed9e2640 1 -- start start 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fed9e2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fe80b9d60 0x7f5fe8162050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fed9e2640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5fe80b64d0 0x7f5fe80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fed9e2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe8162630 con 0x7f5fe815f4e0 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fed9e2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe8162910 con 0x7f5fe80b9d60 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fed9e2640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe8162bf0 con 0x7f5fe80b64d0 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.299+0000 7f5fe6ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fe80b9d60 0x7f5fe8162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe6ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fe80b9d60 0x7f5fe8162050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51992/0 (socket says 172.21.15.66:51992) 2024-03-30T13:12:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe6ffd640 1 -- 172.21.15.66:0/3901693977 learned_addr learned my addr 172.21.15.66:0/3901693977 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe77fe640 1 --2- 172.21.15.66:0/3901693977 >> v2:172.21.15.117:3300/0 conn(0x7f5fe80b64d0 0x7f5fe80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe6ffd640 1 -- 172.21.15.66:0/3901693977 >> v2:172.21.15.117:3300/0 conn(0x7f5fe80b64d0 msgr2=0x7f5fe80b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe6ffd640 1 --2- 172.21.15.66:0/3901693977 >> v2:172.21.15.117:3300/0 conn(0x7f5fe80b64d0 0x7f5fe80b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe6ffd640 1 -- 172.21.15.66:0/3901693977 >> v1:172.21.15.117:6789/0 conn(0x7f5fe815f4e0 legacy=0x7f5fe80b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:52.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.300+0000 7f5fe6ffd640 1 -- 172.21.15.66:0/3901693977 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe8162ed0 con 0x7f5fe80b9d60 2024-03-30T13:12:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fe6ffd640 1 --2- 172.21.15.66:0/3901693977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe80b9d60 0x7f5fe8162050 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009870 tx=0x7f5fdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816dc691965a2add server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fe5ffb640 1 -- 172.21.15.66:0/3901693977 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc035020 con 0x7f5fe80b9d60 2024-03-30T13:12:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fe5ffb640 1 -- 172.21.15.66:0/3901693977 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc031870 con 0x7f5fe80b9d60 2024-03-30T13:12:52.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fe5ffb640 1 -- 172.21.15.66:0/3901693977 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc031c40 con 0x7f5fe80b9d60 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3901693977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe80b9d60 msgr2=0x7f5fe8162050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3901693977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe80b9d60 0x7f5fe8162050 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009870 tx=0x7f5fdc02b910 comp rx=0 tx=0).stop 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3901693977 shutdown_connections 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3901693977 >> v2:172.21.15.117:3300/0 conn(0x7f5fe80b64d0 0x7f5fe80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3901693977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe80b9d60 0x7f5fe8162050 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3901693977 >> 172.21.15.66:0/3901693977 conn(0x7f5fe80678f0 msgr2=0x7f5fe80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:52.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3901693977 shutdown_connections 2024-03-30T13:12:52.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.301+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3901693977 wait complete. 2024-03-30T13:12:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.303+0000 7f5fed9e2640 1 Processor -- start 2024-03-30T13:12:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.304+0000 7f5fed9e2640 1 -- start start 2024-03-30T13:12:52.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fed9e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80b64d0 0x7f5fe8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fed9e2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fe80b9d60 0x7f5fe81e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fed9e2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 0x7f5fe80cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fed9e2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80ba140 con 0x7f5fe815f4e0 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fed9e2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe815f990 con 0x7f5fe80b64d0 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fed9e2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe80ba2c0 con 0x7f5fe80b9d60 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 0x7f5fe80cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 0x7f5fe80cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52004/0 (socket says 172.21.15.66:52004) 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 -- 172.21.15.66:0/3207374437 learned_addr learned my addr 172.21.15.66:0/3207374437 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe6ffd640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80b64d0 0x7f5fe8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe67fc640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fe80b9d60 0x7f5fe81e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 -- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fe80b9d60 msgr2=0x7f5fe81e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fe80b9d60 0x7f5fe81e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 -- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80b64d0 msgr2=0x7f5fe8153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80b64d0 0x7f5fe8153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.305+0000 7f5fe77fe640 1 -- 172.21.15.66:0/3207374437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe81e8720 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.306+0000 7f5fe77fe640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 0x7f5fe80cb8d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0007e40 tx=0x7f5fe000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.306+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0014620 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.306+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe81e8a00 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.306+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe0019070 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.306+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0014a20 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.306+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81e8fc0 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.307+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe8075890 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.307+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5fe0014bc0 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.308+0000 7f5fcffff640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5fd406c530 0x7f5fd406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.308+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f5fe008c040 con 0x7f5fe815f4e0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.308+0000 7f5fe6ffd640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5fd406c530 0x7f5fd406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.308+0000 7f5fe6ffd640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5fd406c530 0x7f5fd406e9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc036000 tx=0x7f5fdc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.312+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5fe0059e60 con 0x7f5fe815f4e0 2024-03-30T13:12:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.496+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fe80ce6c0 con 0x7f5fe815f4e0 2024-03-30T13:12:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.497+0000 7f5fcffff640 1 -- 172.21.15.66:0/3207374437 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f5fe005e160 con 0x7f5fe815f4e0 2024-03-30T13:12:52.501 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:12:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5fd406c530 msgr2=0x7f5fd406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5fd406c530 0x7f5fd406e9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc036000 tx=0x7f5fdc03c040 comp rx=0 tx=0).stop 2024-03-30T13:12:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 msgr2=0x7f5fe80cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 0x7f5fe80cb8d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0007e40 tx=0x7f5fe000e4b0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 shutdown_connections 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5fd406c530 0x7f5fd406e9f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe815f4e0 0x7f5fe80cb8d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fe80b9d60 0x7f5fe81e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 --2- 172.21.15.66:0/3207374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80b64d0 0x7f5fe8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 >> 172.21.15.66:0/3207374437 conn(0x7f5fe80678f0 msgr2=0x7f5fe80b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 shutdown_connections 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.500+0000 7f5fed9e2640 1 -- 172.21.15.66:0/3207374437 wait complete. 2024-03-30T13:12:52.515 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:52.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.621+0000 7fb1db70e640 1 Processor -- start 2024-03-30T13:12:52.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.621+0000 7fb1db70e640 1 -- start start 2024-03-30T13:12:52.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1db70e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40ba090 0x7fb1d40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1db70e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1d40b73d0 0x7fb1d40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1db70e640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d40d2ff0 con 0x7fb1d40b5dc0 2024-03-30T13:12:52.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1db70e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d40d32d0 con 0x7fb1d40b73d0 2024-03-30T13:12:52.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1db70e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d40d35b0 con 0x7fb1d40ba090 2024-03-30T13:12:52.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1d94aa640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb1d40b5dc0 0x7fb1d40b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39642/0 (socket says 172.21.15.66:39642) 2024-03-30T13:12:52.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1d94aa640 1 -- 172.21.15.66:0/853666675 learned_addr learned my addr 172.21.15.66:0/853666675 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:52.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1d9cab640 1 --2- 172.21.15.66:0/853666675 >> v2:172.21.15.117:3300/0 conn(0x7fb1d40b73d0 0x7fb1d40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1d8ca9640 1 --2- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40ba090 0x7fb1d40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.622+0000 7fb1c7fff640 1 -- 172.21.15.66:0/853666675 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813390399 0 0) 0x7fb1d40d2ff0 con 0x7fb1d40b5dc0 2024-03-30T13:12:52.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.623+0000 7fb1c7fff640 1 -- 172.21.15.66:0/853666675 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1c0003520 con 0x7fb1d40b5dc0 2024-03-30T13:12:52.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.623+0000 7fb1d9cab640 1 -- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40ba090 msgr2=0x7fb1d40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:52.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.623+0000 7fb1d9cab640 1 --2- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40ba090 0x7fb1d40d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.623+0000 7fb1d9cab640 1 -- 172.21.15.66:0/853666675 >> v1:172.21.15.66:6789/0 conn(0x7fb1d40b5dc0 legacy=0x7fb1d40b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.623+0000 7fb1d9cab640 1 -- 172.21.15.66:0/853666675 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d40d3890 con 0x7fb1d40b73d0 2024-03-30T13:12:52.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1d9cab640 1 --2- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d40b7880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009d90 tx=0x7fb1cc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d636ce7ff7869d38 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1c7fff640 1 -- 172.21.15.66:0/853666675 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc039070 con 0x7fb1d40b73d0 2024-03-30T13:12:52.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1c7fff640 1 -- 172.21.15.66:0/853666675 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1cc02cc50 con 0x7fb1d40b73d0 2024-03-30T13:12:52.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1c7fff640 1 -- 172.21.15.66:0/853666675 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc034440 con 0x7fb1d40b73d0 2024-03-30T13:12:52.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 -- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 msgr2=0x7fb1d40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 --2- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d40b7880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009d90 tx=0x7fb1cc02bea0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 -- 172.21.15.66:0/853666675 shutdown_connections 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 --2- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d40b7880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 --2- 172.21.15.66:0/853666675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40ba090 0x7fb1d40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 -- 172.21.15.66:0/853666675 >> 172.21.15.66:0/853666675 conn(0x7fb1d40678f0 msgr2=0x7fb1d40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 -- 172.21.15.66:0/853666675 shutdown_connections 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.624+0000 7fb1db70e640 1 -- 172.21.15.66:0/853666675 wait complete. 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 Processor -- start 2024-03-30T13:12:52.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 -- start start 2024-03-30T13:12:52.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40b5dc0 0x7fb1d4158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 0x7fb1d40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d415e880 con 0x7fb1d40ba090 2024-03-30T13:12:52.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d40d4090 con 0x7fb1d40b73d0 2024-03-30T13:12:52.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.626+0000 7fb1db70e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d415ea00 con 0x7fb1d40b5dc0 2024-03-30T13:12:52.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 0x7fb1d40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 0x7fb1d40cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52018/0 (socket says 172.21.15.66:52018) 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 -- 172.21.15.66:0/1100113775 learned_addr learned my addr 172.21.15.66:0/1100113775 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d94aa640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40b5dc0 0x7fb1d4158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d8ca9640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 -- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40b5dc0 msgr2=0x7fb1d4158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40b5dc0 0x7fb1d4158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 -- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 msgr2=0x7fb1d415eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d415eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 -- 172.21.15.66:0/1100113775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d415f640 con 0x7fb1d40ba090 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1d9cab640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 0x7fb1d40cd740 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009d90 tx=0x7fb1cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc044020 con 0x7fb1d40ba090 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1cc02ca50 con 0x7fb1d40ba090 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc039040 con 0x7fb1d40ba090 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.627+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d415f920 con 0x7fb1d40ba090 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.628+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d415fee0 con 0x7fb1d40ba090 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.629+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d4075890 con 0x7fb1d40ba090 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.629+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb1cc047020 con 0x7fb1d40ba090 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.630+0000 7fb1c5ffb640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1b806c530 0x7fb1b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.630+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fb1cc0adf00 con 0x7fb1d40ba090 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.630+0000 7fb1d94aa640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1b806c530 0x7fb1b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.630+0000 7fb1d94aa640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1b806c530 0x7fb1b806e9f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0005880 tx=0x7fb1d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.634+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb1cc04e030 con 0x7fb1d40ba090 2024-03-30T13:12:52.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.821+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb1d40b6520 con 0x7fb1d40ba090 2024-03-30T13:12:52.825 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.822+0000 7fb1c5ffb640 1 -- 172.21.15.66:0/1100113775 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7fb1cc07be60 con 0x7fb1d40ba090 2024-03-30T13:12:52.826 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1b806c530 msgr2=0x7fb1b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1b806c530 0x7fb1b806e9f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0005880 tx=0x7fb1d0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 msgr2=0x7fb1d40cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 0x7fb1d40cd740 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009d90 tx=0x7fb1cc004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 shutdown_connections 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb1b806c530 0x7fb1b806e9f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d40ba090 0x7fb1d40cd740 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d40b73d0 0x7fb1d415eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.823+0000 7fb1db70e640 1 --2- 172.21.15.66:0/1100113775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1d40b5dc0 0x7fb1d4158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.824+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 >> 172.21.15.66:0/1100113775 conn(0x7fb1d40678f0 msgr2=0x7fb1d40b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.824+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 shutdown_connections 2024-03-30T13:12:52.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.824+0000 7fb1db70e640 1 -- 172.21.15.66:0/1100113775 wait complete. 2024-03-30T13:12:52.839 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:52.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 Processor -- start 2024-03-30T13:12:52.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 -- start start 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa1f8071020 0x7fa1f806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1f809aee0 0x7fa1f8162830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f809b370 con 0x7fa1f809aa30 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f80d0770 con 0x7fa1f8071020 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.946+0000 7fa1fd50e640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f80d1c10 con 0x7fa1f809aee0 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f6ffd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa1f8071020 0x7fa1f806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f67fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa1f809aa30 0x7fa1f8162120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39658/0 (socket says 172.21.15.66:39658) 2024-03-30T13:12:52.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f67fc640 1 -- 172.21.15.66:0/497020979 learned_addr learned my addr 172.21.15.66:0/497020979 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f77fe640 1 --2- 172.21.15.66:0/497020979 >> v2:172.21.15.117:3300/0 conn(0x7fa1f809aee0 0x7fa1f8162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f5ffb640 1 -- 172.21.15.66:0/497020979 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758354765 0 0) 0x7fa1f809b370 con 0x7fa1f809aa30 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f5ffb640 1 -- 172.21.15.66:0/497020979 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e0003520 con 0x7fa1f809aa30 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f6ffd640 1 -- 172.21.15.66:0/497020979 >> v2:172.21.15.117:3300/0 conn(0x7fa1f809aee0 msgr2=0x7fa1f8162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f6ffd640 1 --2- 172.21.15.66:0/497020979 >> v2:172.21.15.117:3300/0 conn(0x7fa1f809aee0 0x7fa1f8162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f6ffd640 1 -- 172.21.15.66:0/497020979 >> v1:172.21.15.66:6789/0 conn(0x7fa1f809aa30 legacy=0x7fa1f8162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.947+0000 7fa1f6ffd640 1 -- 172.21.15.66:0/497020979 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f815f180 con 0x7fa1f8071020 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1f6ffd640 1 --2- 172.21.15.66:0/497020979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f8071020 0x7fa1f806c840 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80c9250 tx=0x7fa1e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28880bda5564a7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1f5ffb640 1 -- 172.21.15.66:0/497020979 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8035020 con 0x7fa1f8071020 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1f5ffb640 1 -- 172.21.15.66:0/497020979 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8031830 con 0x7fa1f8071020 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1f5ffb640 1 -- 172.21.15.66:0/497020979 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8031c00 con 0x7fa1f8071020 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1fd50e640 1 -- 172.21.15.66:0/497020979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f8071020 msgr2=0x7fa1f806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/497020979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f8071020 0x7fa1f806c840 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80c9250 tx=0x7fa1e802b8d0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1fd50e640 1 -- 172.21.15.66:0/497020979 shutdown_connections 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/497020979 >> v2:172.21.15.117:3300/0 conn(0x7fa1f809aee0 0x7fa1f8162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/497020979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f8071020 0x7fa1f806c840 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.948+0000 7fa1fd50e640 1 -- 172.21.15.66:0/497020979 >> 172.21.15.66:0/497020979 conn(0x7fa1f80b3ce0 msgr2=0x7fa1f80b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.949+0000 7fa1fd50e640 1 -- 172.21.15.66:0/497020979 shutdown_connections 2024-03-30T13:12:52.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.949+0000 7fa1fd50e640 1 -- 172.21.15.66:0/497020979 wait complete. 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.950+0000 7fa1fd50e640 1 Processor -- start 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.950+0000 7fa1fd50e640 1 -- start start 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1fd50e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f8071020 0x7fa1f8157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1fd50e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f809aa30 0x7fa1f815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1fd50e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 0x7fa1f8155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1fd50e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f815fa20 con 0x7fa1f809aee0 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1fd50e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f815f740 con 0x7fa1f809aa30 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1fd50e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f8071400 con 0x7fa1f8071020 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 0x7fa1f8155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 0x7fa1f8155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52036/0 (socket says 172.21.15.66:52036) 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 -- 172.21.15.66:0/3143433310 learned_addr learned my addr 172.21.15.66:0/3143433310 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:52.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f67fc640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f809aa30 0x7fa1f815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f6ffd640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f8071020 0x7fa1f8157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 -- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f8071020 msgr2=0x7fa1f8157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f8071020 0x7fa1f8157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 -- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f809aa30 msgr2=0x7fa1f815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f809aa30 0x7fa1f815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 -- 172.21.15.66:0/3143433310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f80c7b70 con 0x7fa1f809aee0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f67fc640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f809aa30 0x7fa1f815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f6ffd640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f8071020 0x7fa1f8157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.951+0000 7fa1f77fe640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 0x7fa1f8155870 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009dd0 tx=0x7fa1f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.952+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0018020 con 0x7fa1f809aee0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.952+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0014480 con 0x7fa1f809aee0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.952+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f80c7e50 con 0x7fa1f809aee0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.952+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0014870 con 0x7fa1f809aee0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.952+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f80c8410 con 0x7fa1f809aee0 2024-03-30T13:12:52.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.953+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f8075890 con 0x7fa1f809aee0 2024-03-30T13:12:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.953+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa1f001b020 con 0x7fa1f809aee0 2024-03-30T13:12:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.954+0000 7fa1df7fe640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa1e406c530 0x7fa1e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.954+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fa1f008c400 con 0x7fa1f809aee0 2024-03-30T13:12:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.954+0000 7fa1f6ffd640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa1e406c530 0x7fa1e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:52.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.954+0000 7fa1f6ffd640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa1e406c530 0x7fa1e406e9f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8036000 tx=0x7fa1e803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:52.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:52.958+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa1f0022030 con 0x7fa1f809aee0 2024-03-30T13:12:53.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.292+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1f80c8c30 con 0x7fa1f809aee0 2024-03-30T13:12:53.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.292+0000 7fa1df7fe640 1 -- 172.21.15.66:0/3143433310 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7fa1f0010040 con 0x7fa1f809aee0 2024-03-30T13:12:53.296 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:12:53.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa1e406c530 msgr2=0x7fa1e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa1e406c530 0x7fa1e406e9f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8036000 tx=0x7fa1e803c040 comp rx=0 tx=0).stop 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 msgr2=0x7fa1f8155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 0x7fa1f8155870 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009dd0 tx=0x7fa1f000c490 comp rx=0 tx=0).stop 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 shutdown_connections 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa1e406c530 0x7fa1e406e9f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1f809aee0 0x7fa1f8155870 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f809aa30 0x7fa1f815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 --2- 172.21.15.66:0/3143433310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f8071020 0x7fa1f8157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 >> 172.21.15.66:0/3143433310 conn(0x7fa1f80b3ce0 msgr2=0x7fa1f806fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 shutdown_connections 2024-03-30T13:12:53.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.295+0000 7fa1fd50e640 1 -- 172.21.15.66:0/3143433310 wait complete. 2024-03-30T13:12:53.311 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:53.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.415+0000 7f7357de2640 1 Processor -- start 2024-03-30T13:12:53.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.416+0000 7f7357de2640 1 -- start start 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.416+0000 7f7357de2640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73500cec60 0x7f73500d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.416+0000 7f7357de2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.416+0000 7f7357de2640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73500d3120 con 0x7f735015e7a0 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.416+0000 7f7357de2640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f73500cfc40 con 0x7f73500cec60 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.416+0000 7f7357de2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f73500d3400 con 0x7f73500d3ee0 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73500cec60 0x7f73500d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73500cec60 0x7f73500d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52048/0 (socket says 172.21.15.66:52048) 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 -- 172.21.15.66:0/2234768874 learned_addr learned my addr 172.21.15.66:0/2234768874 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735637f640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 -- 172.21.15.66:0/2234768874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 msgr2=0x7f73500d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 -- 172.21.15.66:0/2234768874 >> v1:172.21.15.117:6789/0 conn(0x7f735015e7a0 legacy=0x7f73500cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:53.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735537d640 1 -- 172.21.15.66:0/2234768874 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500d36e0 con 0x7f73500cec60 2024-03-30T13:12:53.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.417+0000 7f735637f640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f735537d640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f73500d12b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7340009870 tx=0x7f734002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e86befc26b08695 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7354b7c640 1 -- 172.21.15.66:0/2234768874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73400317c0 con 0x7f73500cec60 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7354b7c640 1 -- 172.21.15.66:0/2234768874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7340037070 con 0x7f73500cec60 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7354b7c640 1 -- 172.21.15.66:0/2234768874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340031b80 con 0x7f73500cec60 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 -- 172.21.15.66:0/2234768874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 msgr2=0x7f73500d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f73500d12b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7340009870 tx=0x7f734002b860 comp rx=0 tx=0).stop 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 -- 172.21.15.66:0/2234768874 shutdown_connections 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 --2- 172.21.15.66:0/2234768874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f73500d12b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 -- 172.21.15.66:0/2234768874 >> 172.21.15.66:0/2234768874 conn(0x7f73500676b0 msgr2=0x7f7350067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:53.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 -- 172.21.15.66:0/2234768874 shutdown_connections 2024-03-30T13:12:53.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.418+0000 7f7357de2640 1 -- 172.21.15.66:0/2234768874 wait complete. 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 Processor -- start 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 -- start start 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f735015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735014a3d0 0x7f73500b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73500b7720 con 0x7f73500cec60 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f73500b75a0 con 0x7f735014a3d0 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7357de2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f73500b78a0 con 0x7f73500d3ee0 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7355b7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f735015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7355b7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f735015ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52064/0 (socket says 172.21.15.66:52064) 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.420+0000 7f7355b7e640 1 -- 172.21.15.66:0/1580809496 learned_addr learned my addr 172.21.15.66:0/1580809496 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:53.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f735637f640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735014a3d0 0x7f73500b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f735537d640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7355b7e640 1 -- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 msgr2=0x7f73500b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7355b7e640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7355b7e640 1 -- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735014a3d0 msgr2=0x7f73500b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7355b7e640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735014a3d0 0x7f73500b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7355b7e640 1 -- 172.21.15.66:0/1580809496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73501e7950 con 0x7f73500cec60 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f735537d640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7355b7e640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f735015ec00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f734c00b3f0 tx=0x7f734c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c015020 con 0x7f73500cec60 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f734c011830 con 0x7f73500cec60 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501e7c30 con 0x7f73500cec60 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c011c20 con 0x7f73500cec60 2024-03-30T13:12:53.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.421+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735015f140 con 0x7f73500cec60 2024-03-30T13:12:53.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.422+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7350075900 con 0x7f73500cec60 2024-03-30T13:12:53.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.423+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f734c018020 con 0x7f73500cec60 2024-03-30T13:12:53.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.423+0000 7f73467fc640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f733406c530 0x7f733406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.423+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f734c08b580 con 0x7f73500cec60 2024-03-30T13:12:53.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.423+0000 7f735537d640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f733406c530 0x7f733406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.424+0000 7f735537d640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f733406c530 0x7f733406e9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7340036000 tx=0x7f7340008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.433+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f734c01f030 con 0x7f73500cec60 2024-03-30T13:12:53.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.630+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f735015d530 con 0x7f73500cec60 2024-03-30T13:12:53.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.631+0000 7f73467fc640 1 -- 172.21.15.66:0/1580809496 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f734c0172c0 con 0x7f73500cec60 2024-03-30T13:12:53.635 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-03-30T13:12:53.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f733406c530 msgr2=0x7f733406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f733406c530 0x7f733406e9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7340036000 tx=0x7f7340008030 comp rx=0 tx=0).stop 2024-03-30T13:12:53.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 msgr2=0x7f735015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f735015ec00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f734c00b3f0 tx=0x7f734c00b8b0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 shutdown_connections 2024-03-30T13:12:53.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f733406c530 0x7f733406e9f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f735014a3d0 0x7f73500b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73500d3ee0 0x7f73500b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 --2- 172.21.15.66:0/1580809496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73500cec60 0x7f735015ec00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 >> 172.21.15.66:0/1580809496 conn(0x7f73500676b0 msgr2=0x7f73500c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:53.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 shutdown_connections 2024-03-30T13:12:53.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.633+0000 7f7357de2640 1 -- 172.21.15.66:0/1580809496 wait complete. 2024-03-30T13:12:53.647 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-30T13:12:53.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:53.648 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:53.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:12:53.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:53.684 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T13:12:53.742 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:12:53.742 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:12:53.742 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:53.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056354,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2256 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T13:12:53.762 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:12:53.762 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:12:53.763 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:53.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=179 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T13:12:53.799 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:53.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.905+0000 7fddfef40640 1 Processor -- start 2024-03-30T13:12:53.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddfef40640 1 -- start start 2024-03-30T13:12:53.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddfef40640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fddf80d4a90 0x7fddf80d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddfef40640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf80d2f00 0x7fddf80b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddfef40640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf80d36f0 con 0x7fddf815e950 2024-03-30T13:12:53.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddfef40640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fddf80d3cb0 con 0x7fddf80d4a90 2024-03-30T13:12:53.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddfef40640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fddf80d39d0 con 0x7fddf80d2f00 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddf7fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fddf80d4a90 0x7fddf80d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.906+0000 7fddf7fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fddf80d4a90 0x7fddf80d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52076/0 (socket says 172.21.15.66:52076) 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddf7fff640 1 -- 172.21.15.66:0/1504968357 learned_addr learned my addr 172.21.15.66:0/1504968357 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddfd4dd640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf80d2f00 0x7fddf80b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddf7fff640 1 -- 172.21.15.66:0/1504968357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf80d2f00 msgr2=0x7fddf80b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddf7fff640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf80d2f00 0x7fddf80b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddf7fff640 1 -- 172.21.15.66:0/1504968357 >> v1:172.21.15.117:6789/0 conn(0x7fddf815e950 legacy=0x7fddf80d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddf7fff640 1 -- 172.21.15.66:0/1504968357 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf80b7ac0 con 0x7fddf80d4a90 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.907+0000 7fddfd4dd640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf80d2f00 0x7fddf80b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:53.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddf7fff640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d4a90 0x7fddf80d29c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009870 tx=0x7fdde402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445d9f182b7a91a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddf77fe640 1 -- 172.21.15.66:0/1504968357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde402f840 con 0x7fddf80d4a90 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddf77fe640 1 -- 172.21.15.66:0/1504968357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde402f9e0 con 0x7fddf80d4a90 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddf77fe640 1 -- 172.21.15.66:0/1504968357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde402fdb0 con 0x7fddf80d4a90 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 -- 172.21.15.66:0/1504968357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d4a90 msgr2=0x7fddf80d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d4a90 0x7fddf80d29c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009870 tx=0x7fdde402ba20 comp rx=0 tx=0).stop 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 -- 172.21.15.66:0/1504968357 shutdown_connections 2024-03-30T13:12:53.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf80d2f00 0x7fddf80b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 --2- 172.21.15.66:0/1504968357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d4a90 0x7fddf80d29c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 -- 172.21.15.66:0/1504968357 >> 172.21.15.66:0/1504968357 conn(0x7fddf80676e0 msgr2=0x7fddf8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 -- 172.21.15.66:0/1504968357 shutdown_connections 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.908+0000 7fddfef40640 1 -- 172.21.15.66:0/1504968357 wait complete. 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.909+0000 7fddfef40640 1 Processor -- start 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 -- start start 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 0x7fddf8161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf80d4a90 0x7fddf80aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf815e950 0x7fddf80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fddf815ed30 con 0x7fddf80d2f00 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddf80d33b0 con 0x7fddf80d4a90 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfef40640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fddf815eeb0 con 0x7fddf815e950 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.910+0000 7fddfccdc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 0x7fddf8161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 0x7fddf8161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52078/0 (socket says 172.21.15.66:52078) 2024-03-30T13:12:53.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 -- 172.21.15.66:0/2185062098 learned_addr learned my addr 172.21.15.66:0/2185062098 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:53.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfd4dd640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf815e950 0x7fddf80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddf7fff640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf80d4a90 0x7fddf80aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 -- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf815e950 msgr2=0x7fddf80b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf815e950 0x7fddf80b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 -- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf80d4a90 msgr2=0x7fddf80aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf80d4a90 0x7fddf80aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 -- 172.21.15.66:0/2185062098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf81608d0 con 0x7fddf80d2f00 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddf7fff640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf80d4a90 0x7fddf80aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfd4dd640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf815e950 0x7fddf80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfccdc640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 0x7fddf8161970 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fddec00c430 tx=0x7fddec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:53.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0052d0 con 0x7fddf80d2f00 2024-03-30T13:12:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec005490 con 0x7fddf80d2f00 2024-03-30T13:12:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf8160bb0 con 0x7fddf80d2f00 2024-03-30T13:12:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec007260 con 0x7fddf80d2f00 2024-03-30T13:12:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.911+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf8161170 con 0x7fddf80d2f00 2024-03-30T13:12:53.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.912+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf8075a10 con 0x7fddf80d2f00 2024-03-30T13:12:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.913+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fddec007400 con 0x7fddf80d2f00 2024-03-30T13:12:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.913+0000 7fddf57fa640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdddc06c530 0x7fdddc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.913+0000 7fddf7fff640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdddc06c530 0x7fdddc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.913+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fddec08ae20 con 0x7fddf80d2f00 2024-03-30T13:12:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.914+0000 7fddf7fff640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdddc06c530 0x7fdddc06e9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdde4036000 tx=0x7fdde4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:53.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:53.918+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fddec059290 con 0x7fddf80d2f00 2024-03-30T13:12:54.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.141+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf815df20 con 0x7fddf80d2f00 2024-03-30T13:12:54.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.142+0000 7fddf57fa640 1 -- 172.21.15.66:0/2185062098 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7fddec058b90 con 0x7fddf80d2f00 2024-03-30T13:12:54.146 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:54.146 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:57.041663+0000","modified":"2024-03-30T13:12:44.880334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1537031567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1537031567},{"type":"v1","addr":"172.21.15.66:6837","nonce":1537031567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-03-30T13:12:54.146 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 20 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdddc06c530 msgr2=0x7fdddc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdddc06c530 0x7fdddc06e9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdde4036000 tx=0x7fdde4008030 comp rx=0 tx=0).stop 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 msgr2=0x7fddf8161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 0x7fddf8161970 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fddec00c430 tx=0x7fddec00c8f0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 shutdown_connections 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fdddc06c530 0x7fdddc06e9f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf815e950 0x7fddf80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf80d4a90 0x7fddf80aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 --2- 172.21.15.66:0/2185062098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddf80d2f00 0x7fddf8161970 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 >> 172.21.15.66:0/2185062098 conn(0x7fddf80676e0 msgr2=0x7fddf80d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:54.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 shutdown_connections 2024-03-30T13:12:54.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.144+0000 7fddfef40640 1 -- 172.21.15.66:0/2185062098 wait complete. 2024-03-30T13:12:54.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:54.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.262+0000 7f7052382640 1 Processor -- start 2024-03-30T13:12:54.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.262+0000 7f7052382640 1 -- start start 2024-03-30T13:12:54.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f7052382640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f704c15e950 0x7f704c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f7052382640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f704c0d30d0 0x7f704c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f7052382640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704c0d36f0 con 0x7f704c0d48c0 2024-03-30T13:12:54.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f7052382640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f704c0d3cb0 con 0x7f704c15e950 2024-03-30T13:12:54.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f7052382640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f704c0d4d00 con 0x7f704c0d30d0 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704bfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f704c15e950 0x7f704c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704b7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f704c0d48c0 0x7f704c0d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39666/0 (socket says 172.21.15.66:39666) 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704b7fe640 1 -- 172.21.15.66:0/2223508992 learned_addr learned my addr 172.21.15.66:0/2223508992 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f705091f640 1 --2- 172.21.15.66:0/2223508992 >> v2:172.21.15.117:3300/0 conn(0x7f704c0d30d0 0x7f704c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704bfff640 1 -- 172.21.15.66:0/2223508992 >> v2:172.21.15.117:3300/0 conn(0x7f704c0d30d0 msgr2=0x7f704c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704bfff640 1 --2- 172.21.15.66:0/2223508992 >> v2:172.21.15.117:3300/0 conn(0x7f704c0d30d0 0x7f704c0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704bfff640 1 -- 172.21.15.66:0/2223508992 >> v1:172.21.15.66:6789/0 conn(0x7f704c0d48c0 legacy=0x7f704c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704affd640 1 -- 172.21.15.66:0/2223508992 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270208097 0 0) 0x7f704c0d36f0 con 0x7f704c0d48c0 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.263+0000 7f704bfff640 1 -- 172.21.15.66:0/2223508992 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c0d39d0 con 0x7f704c15e950 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.264+0000 7f704bfff640 1 --2- 172.21.15.66:0/2223508992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c15e950 0x7f704c0d4380 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f704c0d3e30 tx=0x7f703402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f697e7a90616d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.264+0000 7f704affd640 1 -- 172.21.15.66:0/2223508992 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034034070 con 0x7f704c15e950 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.264+0000 7f704affd640 1 -- 172.21.15.66:0/2223508992 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7034030610 con 0x7f704c15e950 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.264+0000 7f704affd640 1 -- 172.21.15.66:0/2223508992 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340309e0 con 0x7f704c15e950 2024-03-30T13:12:54.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 -- 172.21.15.66:0/2223508992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c15e950 msgr2=0x7f704c0d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 --2- 172.21.15.66:0/2223508992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c15e950 0x7f704c0d4380 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f704c0d3e30 tx=0x7f703402b960 comp rx=0 tx=0).stop 2024-03-30T13:12:54.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 -- 172.21.15.66:0/2223508992 shutdown_connections 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 --2- 172.21.15.66:0/2223508992 >> v2:172.21.15.117:3300/0 conn(0x7f704c0d30d0 0x7f704c0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 --2- 172.21.15.66:0/2223508992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c15e950 0x7f704c0d4380 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 -- 172.21.15.66:0/2223508992 >> 172.21.15.66:0/2223508992 conn(0x7f704c0676e0 msgr2=0x7f704c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 -- 172.21.15.66:0/2223508992 shutdown_connections 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.265+0000 7f7052382640 1 -- 172.21.15.66:0/2223508992 wait complete. 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.266+0000 7f7052382640 1 Processor -- start 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.266+0000 7f7052382640 1 -- start start 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f7052382640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 0x7f704c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f7052382640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c0d48c0 0x7f704c158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f7052382640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c15e950 0x7f704c0d1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f7052382640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f704c15ed30 con 0x7f704c0d30d0 2024-03-30T13:12:54.273 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f7052382640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f704c0b7da0 con 0x7f704c15e950 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f7052382640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f704c15eeb0 con 0x7f704c0d48c0 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 0x7f704c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 0x7f704c157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52100/0 (socket says 172.21.15.66:52100) 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 -- 172.21.15.66:0/2226720632 learned_addr learned my addr 172.21.15.66:0/2226720632 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f705091f640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c15e950 0x7f704c0d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704b7fe640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c0d48c0 0x7f704c158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 -- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c0d48c0 msgr2=0x7f704c158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c0d48c0 0x7f704c158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 -- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c15e950 msgr2=0x7f704c0d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.267+0000 7f704bfff640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c15e950 0x7f704c0d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f704bfff640 1 -- 172.21.15.66:0/2226720632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1589c0 con 0x7f704c0d30d0 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f704bfff640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 0x7f704c157790 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f703402b9a0 tx=0x7f70340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034047070 con 0x7f704c0d30d0 2024-03-30T13:12:54.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7034030410 con 0x7f704c0d30d0 2024-03-30T13:12:54.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c0c87a0 con 0x7f704c0d30d0 2024-03-30T13:12:54.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034034040 con 0x7f704c0d30d0 2024-03-30T13:12:54.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.268+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1586d0 con 0x7f704c0d30d0 2024-03-30T13:12:54.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.269+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c0c8ec0 con 0x7f704c0d30d0 2024-03-30T13:12:54.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.269+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f703404a070 con 0x7f704c0d30d0 2024-03-30T13:12:54.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.270+0000 7f7048ff9640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f702c06c530 0x7f702c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.270+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f70340ad1e0 con 0x7f704c0d30d0 2024-03-30T13:12:54.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.270+0000 7f704b7fe640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f702c06c530 0x7f702c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.271+0000 7f704b7fe640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f702c06c530 0x7f702c06e9f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f704c0c8230 tx=0x7f703c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.275+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f703407aff0 con 0x7f704c0d30d0 2024-03-30T13:12:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.486+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f704c075a10 con 0x7f704c0d30d0 2024-03-30T13:12:54.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.487+0000 7f7048ff9640 1 -- 172.21.15.66:0/2226720632 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16990 (secure 0 0 0) 0x7f703407f2f0 con 0x7f704c0d30d0 2024-03-30T13:12:54.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:54.492 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":45,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:12:48.198771+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":44,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f702c06c530 msgr2=0x7f702c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f702c06c530 0x7f702c06e9f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f704c0c8230 tx=0x7f703c007d50 comp rx=0 tx=0).stop 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 msgr2=0x7f704c157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 0x7f704c157790 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f703402b9a0 tx=0x7f70340046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 shutdown_connections 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f702c06c530 0x7f702c06e9f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c15e950 0x7f704c0d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c0d48c0 0x7f704c158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 --2- 172.21.15.66:0/2226720632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f704c0d30d0 0x7f704c157790 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 >> 172.21.15.66:0/2226720632 conn(0x7f704c0676e0 msgr2=0x7f704c0b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:54.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 shutdown_connections 2024-03-30T13:12:54.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.490+0000 7f7052382640 1 -- 172.21.15.66:0/2226720632 wait complete. 2024-03-30T13:12:54.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:09:28.807377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:55.516141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:55.988604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:12:54.506 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:54.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.611+0000 7fb584b5f640 1 Processor -- start 2024-03-30T13:12:54.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.611+0000 7fb584b5f640 1 -- start start 2024-03-30T13:12:54.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.612+0000 7fb584b5f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800b5dc0 0x7fb5800b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.612+0000 7fb584b5f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5800ba090 0x7fb5800d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.612+0000 7fb584b5f640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800b66e0 con 0x7fb58006c620 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.612+0000 7fb584b5f640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb5800d2ff0 con 0x7fb5800ba090 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.612+0000 7fb584b5f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800d32d0 con 0x7fb5800b5dc0 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5800ba090 0x7fb5800d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5800ba090 0x7fb5800d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52104/0 (socket says 172.21.15.66:52104) 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57ed9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb58006c620 0x7fb58006cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39682/0 (socket says 172.21.15.66:39682) 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 -- 172.21.15.66:0/1464679149 learned_addr learned my addr 172.21.15.66:0/1464679149 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57e59c640 1 --2- 172.21.15.66:0/1464679149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800b5dc0 0x7fb5800b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 -- 172.21.15.66:0/1464679149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800b5dc0 msgr2=0x7fb5800b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 --2- 172.21.15.66:0/1464679149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800b5dc0 0x7fb5800b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 -- 172.21.15.66:0/1464679149 >> v1:172.21.15.66:6789/0 conn(0x7fb58006c620 legacy=0x7fb58006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.613+0000 7fb57dd9b640 1 -- 172.21.15.66:0/1464679149 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5800d35b0 con 0x7fb5800ba090 2024-03-30T13:12:54.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb57dd9b640 1 --2- 172.21.15.66:0/1464679149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5800ba090 0x7fb5800d2680 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb574009870 tx=0x7fb57402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b720411a70a6ef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb57d59a640 1 -- 172.21.15.66:0/1464679149 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574034070 con 0x7fb5800ba090 2024-03-30T13:12:54.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb57d59a640 1 -- 172.21.15.66:0/1464679149 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb574030840 con 0x7fb5800ba090 2024-03-30T13:12:54.619 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb57d59a640 1 -- 172.21.15.66:0/1464679149 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574030c10 con 0x7fb5800ba090 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb584b5f640 1 -- 172.21.15.66:0/1464679149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5800ba090 msgr2=0x7fb5800d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb584b5f640 1 --2- 172.21.15.66:0/1464679149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5800ba090 0x7fb5800d2680 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb574009870 tx=0x7fb57402b980 comp rx=0 tx=0).stop 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb584b5f640 1 -- 172.21.15.66:0/1464679149 shutdown_connections 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.614+0000 7fb584b5f640 1 --2- 172.21.15.66:0/1464679149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5800ba090 0x7fb5800d2680 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.615+0000 7fb584b5f640 1 --2- 172.21.15.66:0/1464679149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800b5dc0 0x7fb5800b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.615+0000 7fb584b5f640 1 -- 172.21.15.66:0/1464679149 >> 172.21.15.66:0/1464679149 conn(0x7fb5800678f0 msgr2=0x7fb5800d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.615+0000 7fb584b5f640 1 -- 172.21.15.66:0/1464679149 shutdown_connections 2024-03-30T13:12:54.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.615+0000 7fb584b5f640 1 -- 172.21.15.66:0/1464679149 wait complete. 2024-03-30T13:12:54.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.617+0000 7fb584b5f640 1 Processor -- start 2024-03-30T13:12:54.621 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.617+0000 7fb584b5f640 1 -- start start 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb584b5f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 0x7fb58014b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb584b5f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5800b5dc0 0x7fb5800c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb584b5f640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800ba090 0x7fb5801e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb584b5f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800d4090 con 0x7fb58006c620 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb584b5f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800d3b70 con 0x7fb5800b5dc0 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb584b5f640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800b83e0 con 0x7fb5800ba090 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 0x7fb58014b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 0x7fb58014b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52120/0 (socket says 172.21.15.66:52120) 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 -- 172.21.15.66:0/2058174636 learned_addr learned my addr 172.21.15.66:0/2058174636 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57dd9b640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5800b5dc0 0x7fb5800c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57ed9d640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800ba090 0x7fb5801e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 -- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800ba090 msgr2=0x7fb5801e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800ba090 0x7fb5801e7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 -- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5800b5dc0 msgr2=0x7fb5800c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.618+0000 7fb57e59c640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5800b5dc0 0x7fb5800c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb57e59c640 1 -- 172.21.15.66:0/2058174636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580147fa0 con 0x7fb58006c620 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb57dd9b640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5800b5dc0 0x7fb5800c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb57e59c640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 0x7fb58014b490 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9e0 tx=0x7fb57000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570017070 con 0x7fb58006c620 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57000c9d0 con 0x7fb58006c620 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb580148280 con 0x7fb58006c620 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000cdc0 con 0x7fb58006c620 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.619+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb580148840 con 0x7fb58006c620 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.620+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb580075890 con 0x7fb58006c620 2024-03-30T13:12:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.621+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb57001a070 con 0x7fb58006c620 2024-03-30T13:12:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.621+0000 7fb56effd640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb56006c560 0x7fb56006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.621+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fb57008bc20 con 0x7fb58006c620 2024-03-30T13:12:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.621+0000 7fb57dd9b640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb56006c560 0x7fb56006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.622+0000 7fb57dd9b640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb56006c560 0x7fb56006ea20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb574036000 tx=0x7fb57403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.631+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb57005a080 con 0x7fb58006c620 2024-03-30T13:12:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.816+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5800b8d00 con 0x7fb58006c620 2024-03-30T13:12:54.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.817+0000 7fb56effd640 1 -- 172.21.15.66:0/2058174636 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7fb570059980 con 0x7fb58006c620 2024-03-30T13:12:54.821 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb56006c560 msgr2=0x7fb56006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb56006c560 0x7fb56006ea20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb574036000 tx=0x7fb57403b040 comp rx=0 tx=0).stop 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 msgr2=0x7fb58014b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 0x7fb58014b490 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9e0 tx=0x7fb57000eea0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 shutdown_connections 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fb56006c560 0x7fb56006ea20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5800ba090 0x7fb5801e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5800b5dc0 0x7fb5800c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 --2- 172.21.15.66:0/2058174636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb58006c620 0x7fb58014b490 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.818+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 >> 172.21.15.66:0/2058174636 conn(0x7fb5800678f0 msgr2=0x7fb58015dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.819+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 shutdown_connections 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.819+0000 7fb584b5f640 1 -- 172.21.15.66:0/2058174636 wait complete. 2024-03-30T13:12:54.833 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.939+0000 7efef80b7640 1 Processor -- start 2024-03-30T13:12:54.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef80b7640 1 -- start start 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef80b7640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efef0167120 0x7efef0169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef80b7640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef00d2ff0 con 0x7efef00b5dc0 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef80b7640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef00d32d0 con 0x7efef00ba090 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef80b7640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7efef00d2d90 con 0x7efef0167120 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef5e53640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efef00b5dc0 0x7efef00b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39690/0 (socket says 172.21.15.66:39690) 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.940+0000 7efef5e53640 1 -- 172.21.15.66:0/1116802852 learned_addr learned my addr 172.21.15.66:0/1116802852 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.941+0000 7efef6654640 1 --2- 172.21.15.66:0/1116802852 >> v2:172.21.15.117:3300/0 conn(0x7efef0167120 0x7efef0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.941+0000 7efef4e51640 1 -- 172.21.15.66:0/1116802852 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170278004 0 0) 0x7efef00d2ff0 con 0x7efef00b5dc0 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.941+0000 7efef4e51640 1 -- 172.21.15.66:0/1116802852 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efedc003520 con 0x7efef00b5dc0 2024-03-30T13:12:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.941+0000 7efef6654640 1 -- 172.21.15.66:0/1116802852 >> v1:172.21.15.117:6789/0 conn(0x7efef00ba090 legacy=0x7efef00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.941+0000 7efef6654640 1 -- 172.21.15.66:0/1116802852 >> v1:172.21.15.66:6789/0 conn(0x7efef00b5dc0 legacy=0x7efef00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.941+0000 7efef6654640 1 -- 172.21.15.66:0/1116802852 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef00d35b0 con 0x7efef0167120 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.942+0000 7efef6654640 1 --2- 172.21.15.66:0/1116802852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0169510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efee8009dc0 tx=0x7efee802bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d8166af9f08925 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.942+0000 7efef4e51640 1 -- 172.21.15.66:0/1116802852 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8039070 con 0x7efef0167120 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.942+0000 7efef4e51640 1 -- 172.21.15.66:0/1116802852 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee802ccd0 con 0x7efef0167120 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.942+0000 7efef4e51640 1 -- 172.21.15.66:0/1116802852 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8034640 con 0x7efef0167120 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.942+0000 7efef80b7640 1 -- 172.21.15.66:0/1116802852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 msgr2=0x7efef0169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.942+0000 7efef80b7640 1 --2- 172.21.15.66:0/1116802852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0169510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efee8009dc0 tx=0x7efee802bf20 comp rx=0 tx=0).stop 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.943+0000 7efef80b7640 1 -- 172.21.15.66:0/1116802852 shutdown_connections 2024-03-30T13:12:54.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.943+0000 7efef80b7640 1 --2- 172.21.15.66:0/1116802852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0169510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.943+0000 7efef80b7640 1 -- 172.21.15.66:0/1116802852 >> 172.21.15.66:0/1116802852 conn(0x7efef00678f0 msgr2=0x7efef00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.943+0000 7efef80b7640 1 -- 172.21.15.66:0/1116802852 shutdown_connections 2024-03-30T13:12:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.943+0000 7efef80b7640 1 -- 172.21.15.66:0/1116802852 wait complete. 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.944+0000 7efef80b7640 1 Processor -- start 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.944+0000 7efef80b7640 1 -- start start 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.944+0000 7efef80b7640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efef00b5dc0 0x7efef0159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.944+0000 7efef80b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 0x7efef0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.944+0000 7efef80b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.944+0000 7efef80b7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efef00d4090 con 0x7efef00ba090 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef80b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efef00d3b70 con 0x7efef0167120 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef80b7640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efef00b83e0 con 0x7efef00b5dc0 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 0x7efef0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 0x7efef0156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52134/0 (socket says 172.21.15.66:52134) 2024-03-30T13:12:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 -- 172.21.15.66:0/997937863 learned_addr learned my addr 172.21.15.66:0/997937863 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef6654640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 -- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efef00b5dc0 msgr2=0x7efef0159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5e53640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efef00b5dc0 0x7efef0159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efef00b5dc0 0x7efef0159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 -- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 msgr2=0x7efef0160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 -- 172.21.15.66:0/997937863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef0157870 con 0x7efef00ba090 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef6654640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef5652640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 0x7efef0156850 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7efee000c430 tx=0x7efee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00052d0 con 0x7efef00ba090 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee0005490 con 0x7efef00ba090 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0007260 con 0x7efef00ba090 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef0159ca0 con 0x7efef00ba090 2024-03-30T13:12:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.945+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef015a260 con 0x7efef00ba090 2024-03-30T13:12:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.946+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef0075890 con 0x7efef00ba090 2024-03-30T13:12:54.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.947+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efee0007400 con 0x7efef00ba090 2024-03-30T13:12:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.947+0000 7efee67fc640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efed406c530 0x7efed406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.947+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7efee008adf0 con 0x7efef00ba090 2024-03-30T13:12:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.948+0000 7efef5e53640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efed406c530 0x7efed406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:54.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.948+0000 7efef5e53640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efed406c530 0x7efed406e9f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efeec0011e0 tx=0x7efeec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:54.952+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efee0059260 con 0x7efef00ba090 2024-03-30T13:12:55.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.138+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efef00b9b10 con 0x7efef00ba090 2024-03-30T13:12:55.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.138+0000 7efee67fc640 1 -- 172.21.15.66:0/997937863 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7efee0058b60 con 0x7efef00ba090 2024-03-30T13:12:55.142 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.140+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efed406c530 msgr2=0x7efed406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.140+0000 7efef80b7640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efed406c530 0x7efed406e9f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efeec0011e0 tx=0x7efeec009040 comp rx=0 tx=0).stop 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.140+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 msgr2=0x7efef0156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.140+0000 7efef80b7640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 0x7efef0156850 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7efee000c430 tx=0x7efee000c8f0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 shutdown_connections 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efed406c530 0x7efed406e9f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef0167120 0x7efef0160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efef00ba090 0x7efef0156850 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 --2- 172.21.15.66:0/997937863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efef00b5dc0 0x7efef0159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 >> 172.21.15.66:0/997937863 conn(0x7efef00678f0 msgr2=0x7efef00ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:55.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 shutdown_connections 2024-03-30T13:12:55.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.141+0000 7efef80b7640 1 -- 172.21.15.66:0/997937863 wait complete. 2024-03-30T13:12:55.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:55.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.262+0000 7f1073559640 1 Processor -- start 2024-03-30T13:12:55.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.262+0000 7f1073559640 1 -- start start 2024-03-30T13:12:55.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1073559640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f106c0ba090 0x7f106c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1073559640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106c0d2ff0 con 0x7f106c0b73d0 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1073559640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106c0d32d0 con 0x7f106c0b5dc0 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1073559640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f106c0d35b0 con 0x7f106c0ba090 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1071af6640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f106c0b73d0 0x7f106c0b7820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39706/0 (socket says 172.21.15.66:39706) 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1071af6640 1 -- 172.21.15.66:0/1567640493 learned_addr learned my addr 172.21.15.66:0/1567640493 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1070af4640 1 --2- 172.21.15.66:0/1567640493 >> v2:172.21.15.117:3300/0 conn(0x7f106c0ba090 0x7f106c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1063fff640 1 -- 172.21.15.66:0/1567640493 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699270090 0 0) 0x7f106c0d2ff0 con 0x7f106c0b73d0 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1063fff640 1 -- 172.21.15.66:0/1567640493 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1058003520 con 0x7f106c0b73d0 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1070af4640 1 -- 172.21.15.66:0/1567640493 >> v1:172.21.15.117:6789/0 conn(0x7f106c0b5dc0 legacy=0x7f106c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1063fff640 1 -- 172.21.15.66:0/1567640493 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731082753 0 0) 0x7f106c0d32d0 con 0x7f106c0b5dc0 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1070af4640 1 -- 172.21.15.66:0/1567640493 >> v1:172.21.15.66:6789/0 conn(0x7f106c0b73d0 legacy=0x7f106c0b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.263+0000 7f1070af4640 1 -- 172.21.15.66:0/1567640493 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c0d3890 con 0x7f106c0ba090 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.264+0000 7f1070af4640 1 --2- 172.21.15.66:0/1567640493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c0d2680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f105c0097d0 tx=0x7f105c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc49519db1f8e41f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.264+0000 7f1063fff640 1 -- 172.21.15.66:0/1567640493 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c037070 con 0x7f106c0ba090 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.264+0000 7f1063fff640 1 -- 172.21.15.66:0/1567640493 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c030510 con 0x7f106c0ba090 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1063fff640 1 -- 172.21.15.66:0/1567640493 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640019f0 con 0x7f106c0ba090 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 -- 172.21.15.66:0/1567640493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 msgr2=0x7f106c0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 --2- 172.21.15.66:0/1567640493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c0d2680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f105c0097d0 tx=0x7f105c02b790 comp rx=0 tx=0).stop 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 -- 172.21.15.66:0/1567640493 shutdown_connections 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 --2- 172.21.15.66:0/1567640493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c0d2680 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 -- 172.21.15.66:0/1567640493 >> 172.21.15.66:0/1567640493 conn(0x7f106c0678f0 msgr2=0x7f106c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:55.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 -- 172.21.15.66:0/1567640493 shutdown_connections 2024-03-30T13:12:55.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.265+0000 7f1073559640 1 -- 172.21.15.66:0/1567640493 wait complete. 2024-03-30T13:12:55.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.267+0000 7f1073559640 1 Processor -- start 2024-03-30T13:12:55.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.267+0000 7f1073559640 1 -- start start 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1073559640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0b5dc0 0x7f106c0b9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1073559640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 0x7f106c160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1073559640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1073559640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f106c15e880 con 0x7f106c0b73d0 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1073559640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f106c0d4090 con 0x7f106c0ba090 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1073559640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f106c15ea00 con 0x7f106c0b5dc0 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.268+0000 7f1070af4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 0x7f106c160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 0x7f106c160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52136/0 (socket says 172.21.15.66:52136) 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 -- 172.21.15.66:0/2717884943 learned_addr learned my addr 172.21.15.66:0/2717884943 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:55.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f10712f5640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0b5dc0 0x7f106c0b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1071af6640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 -- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0b5dc0 msgr2=0x7f106c0b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0b5dc0 0x7f106c0b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 -- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 msgr2=0x7f106c153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 -- 172.21.15.66:0/2717884943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c15fe70 con 0x7f106c0b73d0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1070af4640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 0x7f106c160c30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f105c035040 tx=0x7f105c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c037040 con 0x7f106c0b73d0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c004270 con 0x7f106c0b73d0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c160150 con 0x7f106c0b73d0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.269+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c03d650 con 0x7f106c0b73d0 2024-03-30T13:12:55.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.270+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c15fb20 con 0x7f106c0b73d0 2024-03-30T13:12:55.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.270+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c075890 con 0x7f106c0b73d0 2024-03-30T13:12:55.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.272+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f105c03d7f0 con 0x7f106c0b73d0 2024-03-30T13:12:55.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.272+0000 7f1061ffb640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f105006c530 0x7f105006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.272+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f105c0ad380 con 0x7f106c0b73d0 2024-03-30T13:12:55.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.272+0000 7f10712f5640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f105006c530 0x7f105006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.273+0000 7f10712f5640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f105006c530 0x7f105006e9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1068000f60 tx=0x7f1068009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:55.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.277+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f105c042020 con 0x7f106c0b73d0 2024-03-30T13:12:55.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.461+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f106c0b9e70 con 0x7f106c0b73d0 2024-03-30T13:12:55.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.461+0000 7f1061ffb640 1 -- 172.21.15.66:0/2717884943 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f105c07b870 con 0x7f106c0b73d0 2024-03-30T13:12:55.466 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f105006c530 msgr2=0x7f105006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f105006c530 0x7f105006e9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1068000f60 tx=0x7f1068009040 comp rx=0 tx=0).stop 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 msgr2=0x7f106c160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 0x7f106c160c30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f105c035040 tx=0x7f105c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 shutdown_connections 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f105006c530 0x7f105006e9f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106c0ba090 0x7f106c153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f106c0b73d0 0x7f106c160c30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 --2- 172.21.15.66:0/2717884943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f106c0b5dc0 0x7f106c0b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.463+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 >> 172.21.15.66:0/2717884943 conn(0x7f106c0678f0 msgr2=0x7f106c0b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:55.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.464+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 shutdown_connections 2024-03-30T13:12:55.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.464+0000 7f1073559640 1 -- 172.21.15.66:0/2717884943 wait complete. 2024-03-30T13:12:55.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:55.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.586+0000 7fabcc93d640 1 Processor -- start 2024-03-30T13:12:55.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.586+0000 7fabcc93d640 1 -- start start 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabcc93d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fabc80d4a90 0x7fabc80d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabcc93d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc80d2f00 0x7fabc80b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabcc93d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80d36f0 con 0x7fabc815e950 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabcc93d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fabc80d3cb0 con 0x7fabc80d4a90 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabcc93d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fabc80d39d0 con 0x7fabc80d2f00 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc5d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fabc80d4a90 0x7fabc80d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc659c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fabc815e950 0x7fabc80d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39710/0 (socket says 172.21.15.66:39710) 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc659c640 1 -- 172.21.15.66:0/1164972514 learned_addr learned my addr 172.21.15.66:0/1164972514 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:55.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc6d9d640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc80d2f00 0x7fabc80b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc5d9b640 1 -- 172.21.15.66:0/1164972514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc80d2f00 msgr2=0x7fabc80b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc5d9b640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc80d2f00 0x7fabc80b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc5d9b640 1 -- 172.21.15.66:0/1164972514 >> v1:172.21.15.66:6789/0 conn(0x7fabc815e950 legacy=0x7fabc80d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc5d9b640 1 -- 172.21.15.66:0/1164972514 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80b7ac0 con 0x7fabc80d4a90 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.587+0000 7fabc6d9d640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc80d2f00 0x7fabc80b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.588+0000 7fabc5d9b640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80d29c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097a0 tx=0x7fabbc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d4e13752cbee6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:55.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.588+0000 7fabc559a640 1 -- 172.21.15.66:0/1164972514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc034070 con 0x7fabc80d4a90 2024-03-30T13:12:55.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.588+0000 7fabc559a640 1 -- 172.21.15.66:0/1164972514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc030610 con 0x7fabc80d4a90 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.588+0000 7fabc559a640 1 -- 172.21.15.66:0/1164972514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0309e0 con 0x7fabc80d4a90 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 -- 172.21.15.66:0/1164972514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 msgr2=0x7fabc80d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80d29c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097a0 tx=0x7fabbc02b960 comp rx=0 tx=0).stop 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 -- 172.21.15.66:0/1164972514 shutdown_connections 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc80d2f00 0x7fabc80b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 --2- 172.21.15.66:0/1164972514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80d29c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 -- 172.21.15.66:0/1164972514 >> 172.21.15.66:0/1164972514 conn(0x7fabc80676e0 msgr2=0x7fabc8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 -- 172.21.15.66:0/1164972514 shutdown_connections 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.589+0000 7fabcc93d640 1 -- 172.21.15.66:0/1164972514 wait complete. 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.590+0000 7fabcc93d640 1 Processor -- start 2024-03-30T13:12:55.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.590+0000 7fabcc93d640 1 -- start start 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabcc93d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabc80d2f00 0x7fabc815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabcc93d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80ac760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabcc93d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc815e950 0x7fabc8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabcc93d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fabc815ed30 con 0x7fabc80d4a90 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabcc93d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80d33b0 con 0x7fabc80d2f00 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabcc93d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fabc815eeb0 con 0x7fabc815e950 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80ac760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52156/0 (socket says 172.21.15.66:52156) 2024-03-30T13:12:55.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 -- 172.21.15.66:0/3305594466 learned_addr learned my addr 172.21.15.66:0/3305594466 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc6d9d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc815e950 0x7fabc8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc659c640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabc80d2f00 0x7fabc815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 -- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc815e950 msgr2=0x7fabc8160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc815e950 0x7fabc8160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 -- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabc80d2f00 msgr2=0x7fabc815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabc80d2f00 0x7fabc815a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.591+0000 7fabc5d9b640 1 -- 172.21.15.66:0/3305594466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80ad190 con 0x7fabc80d4a90 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.592+0000 7fabc5d9b640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80ac760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009890 tx=0x7fabbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.592+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc034040 con 0x7fabc80d4a90 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.592+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc039a10 con 0x7fabc80d4a90 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.592+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc80ad470 con 0x7fabc80d4a90 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.592+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc039de0 con 0x7fabc80d4a90 2024-03-30T13:12:55.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.592+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc80ada30 con 0x7fabc80d4a90 2024-03-30T13:12:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.593+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc8075a10 con 0x7fabc80d4a90 2024-03-30T13:12:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.594+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fabbc037070 con 0x7fabc80d4a90 2024-03-30T13:12:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.595+0000 7fabb6ffd640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faba806c530 0x7faba806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:55.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.595+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fabbc0ad730 con 0x7fabc80d4a90 2024-03-30T13:12:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.595+0000 7fabc659c640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faba806c530 0x7faba806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:55.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.595+0000 7fabc659c640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faba806c530 0x7faba806e9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fabb8007fd0 tx=0x7fabb8007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:55.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.600+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fabbc07b5d0 con 0x7fabc80d4a90 2024-03-30T13:12:55.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.974+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabc80cc340 con 0x7fabc80d4a90 2024-03-30T13:12:55.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.974+0000 7fabb6ffd640 1 -- 172.21.15.66:0/3305594466 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7fabbc07f8d0 con 0x7fabc80d4a90 2024-03-30T13:12:55.978 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:55.979 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:57.041663+0000","modified":"2024-03-30T13:12:44.880334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1537031567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1537031567},{"type":"v1","addr":"172.21.15.66:6837","nonce":1537031567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-03-30T13:12:55.979 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 20 2024-03-30T13:12:55.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faba806c530 msgr2=0x7faba806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faba806c530 0x7faba806e9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fabb8007fd0 tx=0x7fabb8007e80 comp rx=0 tx=0).stop 2024-03-30T13:12:55.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 msgr2=0x7fabc80ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:55.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80ac760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009890 tx=0x7fabbc0046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 shutdown_connections 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7faba806c530 0x7faba806e9f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabc815e950 0x7fabc8160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc80d4a90 0x7fabc80ac760 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 --2- 172.21.15.66:0/3305594466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabc80d2f00 0x7fabc815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 >> 172.21.15.66:0/3305594466 conn(0x7fabc80676e0 msgr2=0x7fabc80d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 shutdown_connections 2024-03-30T13:12:55.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:55.977+0000 7fabcc93d640 1 -- 172.21.15.66:0/3305594466 wait complete. 2024-03-30T13:12:55.996 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.099+0000 7fe52680d640 1 Processor -- start 2024-03-30T13:12:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.099+0000 7fe52680d640 1 -- start start 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.100+0000 7fe52680d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5200d4a90 0x7fe5200d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.100+0000 7fe52680d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5200d2f00 0x7fe5200b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.100+0000 7fe52680d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5200d36f0 con 0x7fe52015e950 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.100+0000 7fe52680d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fe5200d3cb0 con 0x7fe5200d4a90 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.100+0000 7fe52680d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe5200d39d0 con 0x7fe5200d2f00 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe51f7fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5200d4a90 0x7fe5200d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe524daa640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5200d2f00 0x7fe5200b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe51ffff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe52015e950 0x7fe5200d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:44034/0 (socket says 172.21.15.66:44034) 2024-03-30T13:12:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe51ffff640 1 -- 172.21.15.66:0/3065601513 learned_addr learned my addr 172.21.15.66:0/3065601513 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe524daa640 1 -- 172.21.15.66:0/3065601513 >> v2:172.21.15.117:3300/0 conn(0x7fe5200d4a90 msgr2=0x7fe5200d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe524daa640 1 --2- 172.21.15.66:0/3065601513 >> v2:172.21.15.117:3300/0 conn(0x7fe5200d4a90 0x7fe5200d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe524daa640 1 -- 172.21.15.66:0/3065601513 >> v1:172.21.15.117:6789/0 conn(0x7fe52015e950 legacy=0x7fe5200d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.101+0000 7fe524daa640 1 -- 172.21.15.66:0/3065601513 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5200b7ac0 con 0x7fe5200d2f00 2024-03-30T13:12:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe524daa640 1 --2- 172.21.15.66:0/3065601513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5200d2f00 0x7fe5200b82c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe518009d90 tx=0x7fe51802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476a8a87a9b0b185 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe51effd640 1 -- 172.21.15.66:0/3065601513 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518039070 con 0x7fe5200d2f00 2024-03-30T13:12:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe51effd640 1 -- 172.21.15.66:0/3065601513 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51802cce0 con 0x7fe5200d2f00 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe51effd640 1 -- 172.21.15.66:0/3065601513 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518034660 con 0x7fe5200d2f00 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe52680d640 1 -- 172.21.15.66:0/3065601513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5200d2f00 msgr2=0x7fe5200b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe52680d640 1 --2- 172.21.15.66:0/3065601513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5200d2f00 0x7fe5200b82c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe518009d90 tx=0x7fe51802bf30 comp rx=0 tx=0).stop 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe52680d640 1 -- 172.21.15.66:0/3065601513 shutdown_connections 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe52680d640 1 --2- 172.21.15.66:0/3065601513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5200d2f00 0x7fe5200b82c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe52680d640 1 --2- 172.21.15.66:0/3065601513 >> v2:172.21.15.117:3300/0 conn(0x7fe5200d4a90 0x7fe5200d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.102+0000 7fe52680d640 1 -- 172.21.15.66:0/3065601513 >> 172.21.15.66:0/3065601513 conn(0x7fe5200676e0 msgr2=0x7fe520067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.103+0000 7fe52680d640 1 -- 172.21.15.66:0/3065601513 shutdown_connections 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.103+0000 7fe52680d640 1 -- 172.21.15.66:0/3065601513 wait complete. 2024-03-30T13:12:56.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.104+0000 7fe52680d640 1 Processor -- start 2024-03-30T13:12:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.104+0000 7fe52680d640 1 -- start start 2024-03-30T13:12:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe52680d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 0x7fe5200c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe52680d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5200d4a90 0x7fe52014c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe52680d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe52015e950 0x7fe520156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe52680d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe52015ed30 con 0x7fe5200d2f00 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe52680d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5200d33b0 con 0x7fe5200d4a90 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe52680d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe52015eeb0 con 0x7fe52015e950 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 0x7fe5200c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 0x7fe5200c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52160/0 (socket says 172.21.15.66:52160) 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 -- 172.21.15.66:0/1913349398 learned_addr learned my addr 172.21.15.66:0/1913349398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51f7fe640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5200d4a90 0x7fe52014c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe524daa640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe52015e950 0x7fe520156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 -- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe52015e950 msgr2=0x7fe520156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe52015e950 0x7fe520156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 -- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5200d4a90 msgr2=0x7fe52014c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5200d4a90 0x7fe52014c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 -- 172.21.15.66:0/1913349398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5200c9030 con 0x7fe5200d2f00 2024-03-30T13:12:56.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51ffff640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 0x7fe5200c8930 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c400 tx=0x7fe51000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510017070 con 0x7fe5200d2f00 2024-03-30T13:12:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.105+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe510005ce0 con 0x7fe5200d2f00 2024-03-30T13:12:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.106+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5200c9310 con 0x7fe5200d2f00 2024-03-30T13:12:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.106+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100072f0 con 0x7fe5200d2f00 2024-03-30T13:12:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.106+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5200c98d0 con 0x7fe5200d2f00 2024-03-30T13:12:56.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.106+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe520075a10 con 0x7fe5200d2f00 2024-03-30T13:12:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.107+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe51001a070 con 0x7fe5200d2f00 2024-03-30T13:12:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.108+0000 7fe51cff9640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe50c06c530 0x7fe50c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.108+0000 7fe51f7fe640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe50c06c530 0x7fe50c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.108+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fe51008ab70 con 0x7fe5200d2f00 2024-03-30T13:12:56.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.108+0000 7fe51f7fe640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe50c06c530 0x7fe50c06e9f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe520067e10 tx=0x7fe514009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.112+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe510058a10 con 0x7fe5200d2f00 2024-03-30T13:12:56.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.319+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe52015df50 con 0x7fe5200d2f00 2024-03-30T13:12:56.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.320+0000 7fe51cff9640 1 -- 172.21.15.66:0/1913349398 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16990 (secure 0 0 0) 0x7fe51005cd10 con 0x7fe5200d2f00 2024-03-30T13:12:56.325 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:56.325 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":45,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:12:48.198771+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":44,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.322+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe50c06c530 msgr2=0x7fe50c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.322+0000 7fe52680d640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe50c06c530 0x7fe50c06e9f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe520067e10 tx=0x7fe514009190 comp rx=0 tx=0).stop 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.322+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 msgr2=0x7fe5200c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.322+0000 7fe52680d640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 0x7fe5200c8930 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c400 tx=0x7fe51000c8c0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 shutdown_connections 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe50c06c530 0x7fe50c06e9f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe52015e950 0x7fe520156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5200d4a90 0x7fe52014c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 --2- 172.21.15.66:0/1913349398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5200d2f00 0x7fe5200c8930 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 >> 172.21.15.66:0/1913349398 conn(0x7fe5200676e0 msgr2=0x7fe5200b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 shutdown_connections 2024-03-30T13:12:56.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.323+0000 7fe52680d640 1 -- 172.21.15.66:0/1913349398 wait complete. 2024-03-30T13:12:56.338 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T13:12:56.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:56.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.441+0000 7f80c40b1640 1 Processor -- start 2024-03-30T13:12:56.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.441+0000 7f80c40b1640 1 -- start start 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c40b1640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80bc15e950 0x7f80bc0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c40b1640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80bc0d48c0 0x7f80bc0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c40b1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d2f00 0x7f80bc0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c40b1640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d3cb0 con 0x7f80bc0d48c0 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c40b1640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d4d00 con 0x7f80bc15e950 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c40b1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0d36f0 con 0x7f80bc0d2f00 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c164c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80bc0d48c0 0x7f80bc0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c164c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80bc0d48c0 0x7f80bc0d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52166/0 (socket says 172.21.15.66:52166) 2024-03-30T13:12:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.442+0000 7f80c164c640 1 -- 172.21.15.66:0/1681208994 learned_addr learned my addr 172.21.15.66:0/1681208994 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c264e640 1 --2- 172.21.15.66:0/1681208994 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d2f00 0x7f80bc0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c1e4d640 1 --2- 172.21.15.66:0/1681208994 >> v2:172.21.15.117:3300/0 conn(0x7f80bc15e950 0x7f80bc0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c164c640 1 -- 172.21.15.66:0/1681208994 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d2f00 msgr2=0x7f80bc0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c164c640 1 --2- 172.21.15.66:0/1681208994 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d2f00 0x7f80bc0cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c164c640 1 -- 172.21.15.66:0/1681208994 >> v2:172.21.15.117:3300/0 conn(0x7f80bc15e950 msgr2=0x7f80bc0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c164c640 1 --2- 172.21.15.66:0/1681208994 >> v2:172.21.15.117:3300/0 conn(0x7f80bc15e950 0x7f80bc0d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.443+0000 7f80c164c640 1 -- 172.21.15.66:0/1681208994 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0d39d0 con 0x7f80bc0d48c0 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.444+0000 7f80c164c640 1 --2- 172.21.15.66:0/1681208994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d48c0 0x7f80bc0d29c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0d5510 tx=0x7f80b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec9db503193f801 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.444+0000 7f80c0e4b640 1 -- 172.21.15.66:0/1681208994 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8034070 con 0x7f80bc0d48c0 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.444+0000 7f80c0e4b640 1 -- 172.21.15.66:0/1681208994 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b8030860 con 0x7f80bc0d48c0 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.444+0000 7f80c0e4b640 1 -- 172.21.15.66:0/1681208994 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8030c30 con 0x7f80bc0d48c0 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 -- 172.21.15.66:0/1681208994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d48c0 msgr2=0x7f80bc0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 --2- 172.21.15.66:0/1681208994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d48c0 0x7f80bc0d29c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0d5510 tx=0x7f80b802b9a0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 -- 172.21.15.66:0/1681208994 shutdown_connections 2024-03-30T13:12:56.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 --2- 172.21.15.66:0/1681208994 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d2f00 0x7f80bc0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 --2- 172.21.15.66:0/1681208994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d48c0 0x7f80bc0d29c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 --2- 172.21.15.66:0/1681208994 >> v2:172.21.15.117:3300/0 conn(0x7f80bc15e950 0x7f80bc0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 -- 172.21.15.66:0/1681208994 >> 172.21.15.66:0/1681208994 conn(0x7f80bc0676e0 msgr2=0x7f80bc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 -- 172.21.15.66:0/1681208994 shutdown_connections 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.445+0000 7f80c40b1640 1 -- 172.21.15.66:0/1681208994 wait complete. 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.446+0000 7f80c40b1640 1 Processor -- start 2024-03-30T13:12:56.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.446+0000 7f80c40b1640 1 -- start start 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c40b1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 0x7f80bc14d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c40b1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d48c0 0x7f80bc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c40b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80bc15e950 0x7f80bc0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c40b1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0d3390 con 0x7f80bc0d2f00 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c40b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b7da0 con 0x7f80bc15e950 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c40b1640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc15ed30 con 0x7f80bc0d48c0 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 0x7f80bc14d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 0x7f80bc14d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52180/0 (socket says 172.21.15.66:52180) 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 -- 172.21.15.66:0/2217286454 learned_addr learned my addr 172.21.15.66:0/2217286454 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c164c640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d48c0 0x7f80bc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c264e640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80bc15e950 0x7f80bc0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 -- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d48c0 msgr2=0x7f80bc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d48c0 0x7f80bc15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 -- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80bc15e950 msgr2=0x7f80bc0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80bc15e950 0x7f80bc0cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 -- 172.21.15.66:0/2217286454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc1602b0 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.447+0000 7f80c1e4d640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 0x7f80bc14d940 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c7f0 tx=0x7f80ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.448+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac015070 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.448+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac004240 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.448+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc160590 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.448+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac007620 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.448+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc160b50 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.449+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc075a10 con 0x7f80bc0d2f00 2024-03-30T13:12:56.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.449+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f80ac0079d0 con 0x7f80bc0d2f00 2024-03-30T13:12:56.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.450+0000 7f80b27fc640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f80a006c530 0x7f80a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.450+0000 7f80c164c640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f80a006c530 0x7f80a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.450+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f80ac08bb40 con 0x7f80bc0d2f00 2024-03-30T13:12:56.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.450+0000 7f80c164c640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f80a006c530 0x7f80a006e9f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80b8036000 tx=0x7f80b803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.454+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f80ac0599e0 con 0x7f80bc0d2f00 2024-03-30T13:12:56.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.676+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80bc0b67c0 con 0x7f80bc0d2f00 2024-03-30T13:12:56.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.676+0000 7f80b27fc640 1 -- 172.21.15.66:0/2217286454 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7f80ac05dce0 con 0x7f80bc0d2f00 2024-03-30T13:12:56.680 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:56.680 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:57.041663+0000","modified":"2024-03-30T13:12:44.880334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1537031567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1537031567},{"type":"v1","addr":"172.21.15.66:6837","nonce":1537031567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-03-30T13:12:56.681 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 20 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f80a006c530 msgr2=0x7f80a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f80a006c530 0x7f80a006e9f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80b8036000 tx=0x7f80b803b040 comp rx=0 tx=0).stop 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 msgr2=0x7f80bc14d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 0x7f80bc14d940 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c7f0 tx=0x7f80ac00ccb0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 shutdown_connections 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f80a006c530 0x7f80a006e9f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80bc15e950 0x7f80bc0cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80bc0d48c0 0x7f80bc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 --2- 172.21.15.66:0/2217286454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0d2f00 0x7f80bc14d940 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 >> 172.21.15.66:0/2217286454 conn(0x7f80bc0676e0 msgr2=0x7f80bc15db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.678+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 shutdown_connections 2024-03-30T13:12:56.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.679+0000 7f80c40b1640 1 -- 172.21.15.66:0/2217286454 wait complete. 2024-03-30T13:12:56.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:56.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.799+0000 7fd5e8a0d640 1 Processor -- start 2024-03-30T13:12:56.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.799+0000 7fd5e8a0d640 1 -- start start 2024-03-30T13:12:56.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e8a0d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd5e40d4a90 0x7fd5e40d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e8a0d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40d36f0 con 0x7fd5e415e950 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e8a0d640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40d39d0 con 0x7fd5e40d2f00 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e8a0d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40d3cb0 con 0x7fd5e40d4a90 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e1d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd5e40d4a90 0x7fd5e40d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e259c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd5e415e950 0x7fd5e40d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39726/0 (socket says 172.21.15.66:39726) 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e259c640 1 -- 172.21.15.66:0/1747523698 learned_addr learned my addr 172.21.15.66:0/1747523698 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.800+0000 7fd5e1d9b640 1 -- 172.21.15.66:0/1747523698 >> v1:172.21.15.117:6789/0 conn(0x7fd5e40d2f00 legacy=0x7fd5e40b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.801+0000 7fd5e1d9b640 1 -- 172.21.15.66:0/1747523698 >> v1:172.21.15.66:6789/0 conn(0x7fd5e415e950 legacy=0x7fd5e40d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.801+0000 7fd5e159a640 1 -- 172.21.15.66:0/1747523698 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138074939 0 0) 0x7fd5e40d36f0 con 0x7fd5e415e950 2024-03-30T13:12:56.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.801+0000 7fd5e1d9b640 1 -- 172.21.15.66:0/1747523698 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40b7ac0 con 0x7fd5e40d4a90 2024-03-30T13:12:56.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.801+0000 7fd5e1d9b640 1 --2- 172.21.15.66:0/1747523698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e40d4a90 0x7fd5e40d29c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40097a0 tx=0x7fd5d402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cc2690dcc115e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.801+0000 7fd5e159a640 1 -- 172.21.15.66:0/1747523698 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4034070 con 0x7fd5e40d4a90 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e159a640 1 -- 172.21.15.66:0/1747523698 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d4030610 con 0x7fd5e40d4a90 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e159a640 1 -- 172.21.15.66:0/1747523698 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40309e0 con 0x7fd5e40d4a90 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/1747523698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e40d4a90 msgr2=0x7fd5e40d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/1747523698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e40d4a90 0x7fd5e40d29c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40097a0 tx=0x7fd5d402b960 comp rx=0 tx=0).stop 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/1747523698 shutdown_connections 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/1747523698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e40d4a90 0x7fd5e40d29c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/1747523698 >> 172.21.15.66:0/1747523698 conn(0x7fd5e40676e0 msgr2=0x7fd5e4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/1747523698 shutdown_connections 2024-03-30T13:12:56.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.802+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/1747523698 wait complete. 2024-03-30T13:12:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 Processor -- start 2024-03-30T13:12:56.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 -- start start 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40d2f00 0x7fd5e41f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40d4a90 0x7fd5e4158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 0x7fd5e40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e415ed30 con 0x7fd5e415e950 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40d33b0 con 0x7fd5e40d2f00 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e8a0d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e415eeb0 con 0x7fd5e40d4a90 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 0x7fd5e40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 0x7fd5e40aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52198/0 (socket says 172.21.15.66:52198) 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.804+0000 7fd5e2d9d640 1 -- 172.21.15.66:0/2871430479 learned_addr learned my addr 172.21.15.66:0/2871430479 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e259c640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40d2f00 0x7fd5e41f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e1d9b640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40d4a90 0x7fd5e4158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e2d9d640 1 -- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40d4a90 msgr2=0x7fd5e4158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e2d9d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40d4a90 0x7fd5e4158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e2d9d640 1 -- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40d2f00 msgr2=0x7fd5e41f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e2d9d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40d2f00 0x7fd5e41f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e2d9d640 1 -- 172.21.15.66:0/2871430479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41589c0 con 0x7fd5e415e950 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e2d9d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 0x7fd5e40aadf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c780 tx=0x7fd5dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004b60 con 0x7fd5e415e950 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc004d00 con 0x7fd5e415e950 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e4158ca0 con 0x7fd5e415e950 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00aa90 con 0x7fd5e415e950 2024-03-30T13:12:56.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.805+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41586d0 con 0x7fd5e415e950 2024-03-30T13:12:56.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.806+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4075a10 con 0x7fd5e415e950 2024-03-30T13:12:56.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.807+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd5dc00ac30 con 0x7fd5e415e950 2024-03-30T13:12:56.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.808+0000 7fd5d2ffd640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5c406c530 0x7fd5c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:56.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.808+0000 7fd5e259c640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5c406c530 0x7fd5c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:56.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.808+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7fd5dc08bbc0 con 0x7fd5e415e950 2024-03-30T13:12:56.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.808+0000 7fd5e259c640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5c406c530 0x7fd5c406e9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8000f60 tx=0x7fd5d8007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:56.812+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd5dc020070 con 0x7fd5e415e950 2024-03-30T13:12:57.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.036+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e40cf660 con 0x7fd5e415e950 2024-03-30T13:12:57.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.036+0000 7fd5d2ffd640 1 -- 172.21.15.66:0/2871430479 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5184 (secure 0 0 0) 0x7fd5dc05a020 con 0x7fd5e415e950 2024-03-30T13:12:57.041 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:57.041 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:57.041663+0000","modified":"2024-03-30T13:12:44.880334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6837/1537031567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":1537031567},{"type":"v1","addr":"172.21.15.66:6837","nonce":1537031567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-03-30T13:12:57.041 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 20 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.038+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5c406c530 msgr2=0x7fd5c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.038+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5c406c530 0x7fd5c406e9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8000f60 tx=0x7fd5d8007c00 comp rx=0 tx=0).stop 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.038+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 msgr2=0x7fd5e40aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.038+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 0x7fd5e40aadf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c780 tx=0x7fd5dc00cc40 comp rx=0 tx=0).stop 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.038+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 shutdown_connections 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.038+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fd5c406c530 0x7fd5c406e9f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.039+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e415e950 0x7fd5e40aadf0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.039+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40d4a90 0x7fd5e4158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.039+0000 7fd5e8a0d640 1 --2- 172.21.15.66:0/2871430479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40d2f00 0x7fd5e41f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.039+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 >> 172.21.15.66:0/2871430479 conn(0x7fd5e40676e0 msgr2=0x7fd5e40d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:57.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.039+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 shutdown_connections 2024-03-30T13:12:57.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.039+0000 7fd5e8a0d640 1 -- 172.21.15.66:0/2871430479 wait complete. 2024-03-30T13:12:57.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:57.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.157+0000 7f586c100640 1 Processor -- start 2024-03-30T13:12:57.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.157+0000 7f586c100640 1 -- start start 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.158+0000 7f586c100640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f586415e4f0 0x7f58640cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.158+0000 7f586c100640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58640b7530 0x7f58640d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.158+0000 7f586c100640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58640b9d20 con 0x7f58640b7990 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f586c100640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f58640ba000 con 0x7f586415e4f0 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f586c100640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f58640ba2e0 con 0x7f58640b7530 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f586415e4f0 0x7f58640cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f586415e4f0 0x7f58640cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52206/0 (socket says 172.21.15.66:52206) 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 -- 172.21.15.66:0/174876067 learned_addr learned my addr 172.21.15.66:0/174876067 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:57.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f586969b640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58640b7530 0x7f58640d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 -- 172.21.15.66:0/174876067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58640b7530 msgr2=0x7f58640d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58640b7530 0x7f58640d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 -- 172.21.15.66:0/174876067 >> v1:172.21.15.117:6789/0 conn(0x7f58640b7990 legacy=0x7f58640b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f5869e9c640 1 -- 172.21.15.66:0/174876067 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58640ceaf0 con 0x7f586415e4f0 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.159+0000 7f586969b640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58640b7530 0x7f58640d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.160+0000 7f5869e9c640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586415e4f0 0x7f58640cf460 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f586002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdf751f318758e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.160+0000 7f5868e9a640 1 -- 172.21.15.66:0/174876067 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600317c0 con 0x7f586415e4f0 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.160+0000 7f5868e9a640 1 -- 172.21.15.66:0/174876067 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5860037070 con 0x7f586415e4f0 2024-03-30T13:12:57.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.160+0000 7f5868e9a640 1 -- 172.21.15.66:0/174876067 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860031b80 con 0x7f586415e4f0 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 -- 172.21.15.66:0/174876067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586415e4f0 msgr2=0x7f58640cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586415e4f0 0x7f58640cf460 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f586002b860 comp rx=0 tx=0).stop 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 -- 172.21.15.66:0/174876067 shutdown_connections 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58640b7530 0x7f58640d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 --2- 172.21.15.66:0/174876067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f586415e4f0 0x7f58640cf460 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 -- 172.21.15.66:0/174876067 >> 172.21.15.66:0/174876067 conn(0x7f58640676e0 msgr2=0x7f5864067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 -- 172.21.15.66:0/174876067 shutdown_connections 2024-03-30T13:12:57.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.161+0000 7f586c100640 1 -- 172.21.15.66:0/174876067 wait complete. 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.162+0000 7f586c100640 1 Processor -- start 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.162+0000 7f586c100640 1 -- start start 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586c100640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 0x7f58641615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586c100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58640b7990 0x7f586414a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586c100640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586415e4f0 0x7f58640c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586c100640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f586415ea50 con 0x7f58640b7530 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586c100640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f586415e8d0 con 0x7f58640b7990 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586c100640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f58640d2ef0 con 0x7f586415e4f0 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 0x7f58641615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 0x7f58641615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52220/0 (socket says 172.21.15.66:52220) 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 -- 172.21.15.66:0/947596982 learned_addr learned my addr 172.21.15.66:0/947596982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586969b640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58640b7990 0x7f586414a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586a69d640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586415e4f0 0x7f58640c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 -- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586415e4f0 msgr2=0x7f58640c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586415e4f0 0x7f58640c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 -- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58640b7990 msgr2=0x7f586414a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58640b7990 0x7f586414a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 -- 172.21.15.66:0/947596982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58640ca750 con 0x7f58640b7530 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f586a69d640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586415e4f0 0x7f58640c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.163+0000 7f5869e9c640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 0x7f58641615b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f58600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.164+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860032860 con 0x7f58640b7530 2024-03-30T13:12:57.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.164+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5860032a00 con 0x7f58640b7530 2024-03-30T13:12:57.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.164+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58640caa30 con 0x7f58640b7530 2024-03-30T13:12:57.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.164+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860032df0 con 0x7f58640b7530 2024-03-30T13:12:57.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.164+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58640b4ee0 con 0x7f58640b7530 2024-03-30T13:12:57.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.165+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5864075a10 con 0x7f58640b7530 2024-03-30T13:12:57.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.166+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f586004d3f0 con 0x7f58640b7530 2024-03-30T13:12:57.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.166+0000 7f585a7fc640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f584806c530 0x7f584806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.166+0000 7f586969b640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f584806c530 0x7f584806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.166+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f58600ad690 con 0x7f58640b7530 2024-03-30T13:12:57.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.167+0000 7f586969b640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f584806c530 0x7f584806e9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58640d1c20 tx=0x7f5854008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.171+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5860040070 con 0x7f58640b7530 2024-03-30T13:12:57.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.357+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58640ce810 con 0x7f58640b7530 2024-03-30T13:12:57.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.358+0000 7f585a7fc640 1 -- 172.21.15.66:0/947596982 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16990 (secure 0 0 0) 0x7f58600abe20 con 0x7f58640b7530 2024-03-30T13:12:57.362 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:12:57.363 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":45,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:12:48.198771+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:55.516141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:55.988604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":44,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:57.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f584806c530 msgr2=0x7f584806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f584806c530 0x7f584806e9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58640d1c20 tx=0x7f5854008040 comp rx=0 tx=0).stop 2024-03-30T13:12:57.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 msgr2=0x7f58641615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 0x7f58641615b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f58600046c0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 shutdown_connections 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f584806c530 0x7f584806e9f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586415e4f0 0x7f58640c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58640b7990 0x7f586414a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 --2- 172.21.15.66:0/947596982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58640b7530 0x7f58641615b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 >> 172.21.15.66:0/947596982 conn(0x7f58640676e0 msgr2=0x7f5864157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 shutdown_connections 2024-03-30T13:12:57.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.360+0000 7f586c100640 1 -- 172.21.15.66:0/947596982 wait complete. 2024-03-30T13:12:57.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T13:12:57.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.479+0000 7f58bbed9640 1 Processor -- start 2024-03-30T13:12:57.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.479+0000 7f58bbed9640 1 -- start start 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58bbed9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58b415e7a0 0x7f58b40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58bbed9640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58b40d19c0 0x7f58b40b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58bbed9640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40d3120 con 0x7f58b40cec60 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58bbed9640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f58b40cfc40 con 0x7f58b40d19c0 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58bbed9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f58b40d3400 con 0x7f58b415e7a0 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58b9474640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f58b40cec60 0x7f58b40d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39728/0 (socket says 172.21.15.66:39728) 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58ba476640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58b40d19c0 0x7f58b40b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.480+0000 7f58b9474640 1 -- 172.21.15.66:0/1755020850 learned_addr learned my addr 172.21.15.66:0/1755020850 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58b9c75640 1 --2- 172.21.15.66:0/1755020850 >> v2:172.21.15.117:3300/0 conn(0x7f58b415e7a0 0x7f58b40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58ba476640 1 -- 172.21.15.66:0/1755020850 >> v2:172.21.15.117:3300/0 conn(0x7f58b415e7a0 msgr2=0x7f58b40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58ba476640 1 --2- 172.21.15.66:0/1755020850 >> v2:172.21.15.117:3300/0 conn(0x7f58b415e7a0 0x7f58b40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58ba476640 1 -- 172.21.15.66:0/1755020850 >> v1:172.21.15.66:6789/0 conn(0x7f58b40cec60 legacy=0x7f58b40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58ba476640 1 -- 172.21.15.66:0/1755020850 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40d36e0 con 0x7f58b40d19c0 2024-03-30T13:12:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58b8c73640 1 -- 172.21.15.66:0/1755020850 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581388418 0 0) 0x7f58b40d3120 con 0x7f58b40cec60 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.481+0000 7f58ba476640 1 --2- 172.21.15.66:0/1755020850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40d19c0 0x7f58b40b75a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f58b406d8f0 tx=0x7f58ac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d935374f84b31ac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58b8c73640 1 -- 172.21.15.66:0/1755020850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac039070 con 0x7f58b40d19c0 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58b8c73640 1 -- 172.21.15.66:0/1755020850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac02cbf0 con 0x7f58b40d19c0 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58b8c73640 1 -- 172.21.15.66:0/1755020850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac034440 con 0x7f58b40d19c0 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 -- 172.21.15.66:0/1755020850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40d19c0 msgr2=0x7f58b40b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 --2- 172.21.15.66:0/1755020850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40d19c0 0x7f58b40b75a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f58b406d8f0 tx=0x7f58ac02be40 comp rx=0 tx=0).stop 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 -- 172.21.15.66:0/1755020850 shutdown_connections 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 --2- 172.21.15.66:0/1755020850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40d19c0 0x7f58b40b75a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 --2- 172.21.15.66:0/1755020850 >> v2:172.21.15.117:3300/0 conn(0x7f58b415e7a0 0x7f58b40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 -- 172.21.15.66:0/1755020850 >> 172.21.15.66:0/1755020850 conn(0x7f58b40676b0 msgr2=0x7f58b4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:57.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 -- 172.21.15.66:0/1755020850 shutdown_connections 2024-03-30T13:12:57.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.482+0000 7f58bbed9640 1 -- 172.21.15.66:0/1755020850 wait complete. 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 Processor -- start 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 -- start start 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 0x7f58b4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d19c0 0x7f58b40b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4160c50 0x7f58b40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40d3ee0 con 0x7f58b40cec60 2024-03-30T13:12:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40cf930 con 0x7f58b4160c50 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58bbed9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40d4060 con 0x7f58b40d19c0 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58b9c75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 0x7f58b4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58b9c75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 0x7f58b4154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52238/0 (socket says 172.21.15.66:52238) 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.484+0000 7f58b9c75640 1 -- 172.21.15.66:0/2673137341 learned_addr learned my addr 172.21.15.66:0/2673137341 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58ba476640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4160c50 0x7f58b40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9474640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d19c0 0x7f58b40b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9c75640 1 -- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d19c0 msgr2=0x7f58b40b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9c75640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d19c0 0x7f58b40b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9c75640 1 -- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4160c50 msgr2=0x7f58b40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9c75640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4160c50 0x7f58b40cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9c75640 1 -- 172.21.15.66:0/2673137341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b41587d0 con 0x7f58b40cec60 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58b9c75640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 0x7f58b4154700 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f58b000d800 tx=0x7f58b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000b6c0 con 0x7f58b40cec60 2024-03-30T13:12:57.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b0019070 con 0x7f58b40cec60 2024-03-30T13:12:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000bac0 con 0x7f58b40cec60 2024-03-30T13:12:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b40b4350 con 0x7f58b40cec60 2024-03-30T13:12:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.485+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b40b4910 con 0x7f58b40cec60 2024-03-30T13:12:57.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.486+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b40b4ae0 con 0x7f58b40cec60 2024-03-30T13:12:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.487+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f58b000bc60 con 0x7f58b40cec60 2024-03-30T13:12:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.487+0000 7f58aa7fc640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589806c560 0x7f589806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.487+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6158+0+0 (secure 0 0 0) 0x7f58b008bd40 con 0x7f58b40cec60 2024-03-30T13:12:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.487+0000 7f58b9474640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589806c560 0x7f589806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.488+0000 7f58b9474640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589806c560 0x7f589806ea20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f58b4154c40 tx=0x7f58a4007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.491+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f58b005a1a0 con 0x7f58b40cec60 2024-03-30T13:12:57.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.722+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f58b4161aa0 con 0x7f58b40cec60 2024-03-30T13:12:57.727 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-03-30T13:12:57.723+0000 7f1a04250640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T13:12:57.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.724+0000 7f58aa7fc640 1 -- 172.21.15.66:0/2673137341 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f58b0059aa0 con 0x7f58b40cec60 2024-03-30T13:12:57.728 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T13:12:57.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589806c560 msgr2=0x7f589806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589806c560 0x7f589806ea20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f58b4154c40 tx=0x7f58a4007c00 comp rx=0 tx=0).stop 2024-03-30T13:12:57.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 msgr2=0x7f58b4154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 0x7f58b4154700 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f58b000d800 tx=0x7f58b000dcc0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 shutdown_connections 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f589806c560 0x7f589806ea20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4160c50 0x7f58b40cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d19c0 0x7f58b40b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 --2- 172.21.15.66:0/2673137341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58b40cec60 0x7f58b4154700 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 >> 172.21.15.66:0/2673137341 conn(0x7f58b40676b0 msgr2=0x7f58b415da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 shutdown_connections 2024-03-30T13:12:57.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.727+0000 7f58bbed9640 1 -- 172.21.15.66:0/2673137341 wait complete. 2024-03-30T13:12:57.743 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T13:12:57.771 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-03-30T13:12:57.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.850+0000 7f7d787f4640 1 Processor -- start 2024-03-30T13:12:57.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.850+0000 7f7d787f4640 1 -- start start 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d787f4640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d700ba090 0x7f7d700d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d787f4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7d700b73d0 0x7f7d700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d787f4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d700d2ff0 con 0x7f7d700b5dc0 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d787f4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7d700d32d0 con 0x7f7d700b73d0 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d787f4640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f7d700d35b0 con 0x7f7d700ba090 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d76590640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f7d700b5dc0 0x7f7d700b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39746/0 (socket says 172.21.15.66:39746) 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d76590640 1 -- 172.21.15.66:0/132959178 learned_addr learned my addr 172.21.15.66:0/132959178 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d76d91640 1 --2- 172.21.15.66:0/132959178 >> v2:172.21.15.66:3300/0 conn(0x7f7d700b73d0 0x7f7d700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.851+0000 7f7d75d8f640 1 --2- 172.21.15.66:0/132959178 >> v2:172.21.15.117:3300/0 conn(0x7f7d700ba090 0x7f7d700d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d7558e640 1 -- 172.21.15.66:0/132959178 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2530859454 0 0) 0x7f7d700d2ff0 con 0x7f7d700b5dc0 2024-03-30T13:12:57.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d7558e640 1 -- 172.21.15.66:0/132959178 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d58003520 con 0x7f7d700b5dc0 2024-03-30T13:12:57.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d76d91640 1 -- 172.21.15.66:0/132959178 >> v2:172.21.15.117:3300/0 conn(0x7f7d700ba090 msgr2=0x7f7d700d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:57.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d76d91640 1 --2- 172.21.15.66:0/132959178 >> v2:172.21.15.117:3300/0 conn(0x7f7d700ba090 0x7f7d700d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d76d91640 1 -- 172.21.15.66:0/132959178 >> v1:172.21.15.66:6789/0 conn(0x7f7d700b5dc0 legacy=0x7f7d700b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d76d91640 1 -- 172.21.15.66:0/132959178 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d700d3890 con 0x7f7d700b73d0 2024-03-30T13:12:57.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.852+0000 7f7d76d91640 1 --2- 172.21.15.66:0/132959178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b73d0 0x7f7d700b7820 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009d90 tx=0x7f7d6002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934be191e85f4cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d7558e640 1 -- 172.21.15.66:0/132959178 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60039070 con 0x7f7d700b73d0 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d7558e640 1 -- 172.21.15.66:0/132959178 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6002cc60 con 0x7f7d700b73d0 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d7558e640 1 -- 172.21.15.66:0/132959178 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60034660 con 0x7f7d700b73d0 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 -- 172.21.15.66:0/132959178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b73d0 msgr2=0x7f7d700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 --2- 172.21.15.66:0/132959178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b73d0 0x7f7d700b7820 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009d90 tx=0x7f7d6002beb0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 -- 172.21.15.66:0/132959178 shutdown_connections 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 --2- 172.21.15.66:0/132959178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b73d0 0x7f7d700b7820 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 --2- 172.21.15.66:0/132959178 >> v2:172.21.15.117:3300/0 conn(0x7f7d700ba090 0x7f7d700d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 -- 172.21.15.66:0/132959178 >> 172.21.15.66:0/132959178 conn(0x7f7d700678f0 msgr2=0x7f7d700d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 -- 172.21.15.66:0/132959178 shutdown_connections 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.853+0000 7f7d787f4640 1 -- 172.21.15.66:0/132959178 wait complete. 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.854+0000 7f7d787f4640 1 Processor -- start 2024-03-30T13:12:57.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 -- start start 2024-03-30T13:12:57.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 0x7f7d701548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d700b73d0 0x7f7d7015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d700ba090 0x7f7d700b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7015e880 con 0x7f7d700b5dc0 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d700d4090 con 0x7f7d700ba090 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d787f4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7015ea00 con 0x7f7d700b73d0 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d76590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 0x7f7d701548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d76590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 0x7f7d701548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52274/0 (socket says 172.21.15.66:52274) 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.855+0000 7f7d76590640 1 -- 172.21.15.66:0/3765505554 learned_addr learned my addr 172.21.15.66:0/3765505554 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d75d8f640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d700b73d0 0x7f7d7015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76d91640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d700ba090 0x7f7d700b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76590640 1 -- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d700b73d0 msgr2=0x7f7d7015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76590640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d700b73d0 0x7f7d7015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76590640 1 -- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d700ba090 msgr2=0x7f7d700b8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76590640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d700ba090 0x7f7d700b8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76590640 1 -- 172.21.15.66:0/3765505554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7015b770 con 0x7f7d700b5dc0 2024-03-30T13:12:57.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d76590640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 0x7f7d701548e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c460 tx=0x7f7d6800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68017070 con 0x7f7d700b5dc0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d68005ce0 con 0x7f7d700b5dc0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d701f3dc0 con 0x7f7d700b5dc0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68007290 con 0x7f7d700b5dc0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.856+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701f4380 con 0x7f7d700b5dc0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.857+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d70075890 con 0x7f7d700b5dc0 2024-03-30T13:12:57.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.858+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7d6801a070 con 0x7f7d700b5dc0 2024-03-30T13:12:57.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.858+0000 7f7d6effd640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d5c06c530 0x7f7d5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:57.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.858+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6216+0+0 (secure 0 0 0) 0x7f7d6808ac60 con 0x7f7d700b5dc0 2024-03-30T13:12:57.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.858+0000 7f7d75d8f640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d5c06c530 0x7f7d5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:57.864 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.859+0000 7f7d75d8f640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d5c06c530 0x7f7d5c06e9f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d7015f880 tx=0x7f7d64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:57.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:57.862+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7d68059080 con 0x7f7d700b5dc0 2024-03-30T13:12:58.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.084+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7d701f4580 con 0x7f7d700b5dc0 2024-03-30T13:12:58.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.726+0000 7f7d6effd640 1 -- 172.21.15.66:0/3765505554 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f7d68058980 con 0x7f7d700b5dc0 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d5c06c530 msgr2=0x7f7d5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d5c06c530 0x7f7d5c06e9f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7d7015f880 tx=0x7f7d64008040 comp rx=0 tx=0).stop 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 msgr2=0x7f7d701548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 0x7f7d701548e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c460 tx=0x7f7d6800c920 comp rx=0 tx=0).stop 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 shutdown_connections 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f7d5c06c530 0x7f7d5c06e9f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d700ba090 0x7f7d700b8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d700b73d0 0x7f7d7015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 --2- 172.21.15.66:0/3765505554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d700b5dc0 0x7f7d701548e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.729+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 >> 172.21.15.66:0/3765505554 conn(0x7f7d700678f0 msgr2=0x7f7d700b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.730+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 shutdown_connections 2024-03-30T13:12:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.730+0000 7f7d787f4640 1 -- 172.21.15.66:0/3765505554 wait complete. 2024-03-30T13:12:58.747 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T13:12:58.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.904+0000 7f0d0d122640 1 Processor -- start 2024-03-30T13:12:58.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.904+0000 7f0d0d122640 1 -- start start 2024-03-30T13:12:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.904+0000 7f0d0d122640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d080cef20 0x7f0d080b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.904+0000 7f0d0d122640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0d080b66f0 0x7f0d080d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:58.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0d122640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d080cf3d0 con 0x7f0d0815e710 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0d122640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0806d5a0 con 0x7f0d080b66f0 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0d122640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f0d080b6b60 con 0x7f0d080cef20 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0759e640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0d080b66f0 0x7f0d080d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d06d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0d0815e710 0x7f0d0806ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39760/0 (socket says 172.21.15.66:39760) 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d06d9d640 1 -- 172.21.15.66:0/1392374373 learned_addr learned my addr 172.21.15.66:0/1392374373 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0759e640 1 -- 172.21.15.66:0/1392374373 >> v2:172.21.15.117:3300/0 conn(0x7f0d080cef20 msgr2=0x7f0d080b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0cfffff640 1 --2- 172.21.15.66:0/1392374373 >> v2:172.21.15.117:3300/0 conn(0x7f0d080cef20 0x7f0d080b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0759e640 1 --2- 172.21.15.66:0/1392374373 >> v2:172.21.15.117:3300/0 conn(0x7f0d080cef20 0x7f0d080b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0759e640 1 -- 172.21.15.66:0/1392374373 >> v1:172.21.15.66:6789/0 conn(0x7f0d0815e710 legacy=0x7f0d0806ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0659c640 1 -- 172.21.15.66:0/1392374373 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355756085 0 0) 0x7f0d080cf3d0 con 0x7f0d0815e710 2024-03-30T13:12:58.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.905+0000 7f0d0759e640 1 -- 172.21.15.66:0/1392374373 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d080cf6b0 con 0x7f0d080b66f0 2024-03-30T13:12:58.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.906+0000 7f0d0759e640 1 --2- 172.21.15.66:0/1392374373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080b66f0 0x7f0d080d2c30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0d080d5450 tx=0x7f0d0002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acd5266d08525a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:58.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.906+0000 7f0d0659c640 1 -- 172.21.15.66:0/1392374373 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00039070 con 0x7f0d080b66f0 2024-03-30T13:12:58.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.906+0000 7f0d0659c640 1 -- 172.21.15.66:0/1392374373 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0002cd00 con 0x7f0d080b66f0 2024-03-30T13:12:58.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.906+0000 7f0d0659c640 1 -- 172.21.15.66:0/1392374373 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00034660 con 0x7f0d080b66f0 2024-03-30T13:12:58.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.906+0000 7f0d0d122640 1 -- 172.21.15.66:0/1392374373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080b66f0 msgr2=0x7f0d080d2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:58.912 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.906+0000 7f0d0d122640 1 --2- 172.21.15.66:0/1392374373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080b66f0 0x7f0d080d2c30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0d080d5450 tx=0x7f0d0002bf50 comp rx=0 tx=0).stop 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.907+0000 7f0d0d122640 1 -- 172.21.15.66:0/1392374373 shutdown_connections 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.907+0000 7f0d0d122640 1 --2- 172.21.15.66:0/1392374373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d080b66f0 0x7f0d080d2c30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.907+0000 7f0d0d122640 1 --2- 172.21.15.66:0/1392374373 >> v2:172.21.15.117:3300/0 conn(0x7f0d080cef20 0x7f0d080b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.907+0000 7f0d0d122640 1 -- 172.21.15.66:0/1392374373 >> 172.21.15.66:0/1392374373 conn(0x7f0d080d1210 msgr2=0x7f0d080d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.907+0000 7f0d0d122640 1 -- 172.21.15.66:0/1392374373 shutdown_connections 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.907+0000 7f0d0d122640 1 -- 172.21.15.66:0/1392374373 wait complete. 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.908+0000 7f0d0d122640 1 Processor -- start 2024-03-30T13:12:58.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.908+0000 7f0d0d122640 1 -- start start 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0d122640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d080b66f0 0x7f0d0815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0d122640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d080cef20 0x7f0d0814a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0d122640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 0x7f0d081f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0d122640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d080d2710 con 0x7f0d0815e710 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0d122640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d080cfc70 con 0x7f0d080b66f0 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0d122640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0815eaf0 con 0x7f0d080cef20 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 0x7f0d081f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:58.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 0x7f0d081f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52296/0 (socket says 172.21.15.66:52296) 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 -- 172.21.15.66:0/2990784462 learned_addr learned my addr 172.21.15.66:0/2990784462 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0cfffff640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d080cef20 0x7f0d0814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d06d9d640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d080b66f0 0x7f0d0815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 -- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d080cef20 msgr2=0x7f0d0814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d080cef20 0x7f0d0814a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 -- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d080b66f0 msgr2=0x7f0d0815edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d080b66f0 0x7f0d0815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:58.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 -- 172.21.15.66:0/2990784462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d080ca8b0 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.909+0000 7f0d0759e640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 0x7f0d081f8420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009e80 tx=0x7f0d00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.910+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00039050 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.910+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d00045070 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.910+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d080cab90 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.910+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0003e830 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.910+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081f7c20 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.911+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d080d5190 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.911+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0d0003e9d0 con 0x7f0d0815e710 2024-03-30T13:12:58.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.912+0000 7f0cff7fe640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0ce806c530 0x7f0ce806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:58.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.912+0000 7f0d06d9d640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0ce806c530 0x7f0ce806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:58.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.912+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6216+0+0 (secure 0 0 0) 0x7f0d000ae6c0 con 0x7f0d0815e710 2024-03-30T13:12:58.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.912+0000 7f0d06d9d640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0ce806c530 0x7f0ce806e9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8000f60 tx=0x7f0cf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:58.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:58.916+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0d080d5190 con 0x7f0d0815e710 2024-03-30T13:12:59.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.100+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0d080d2140 con 0x7f0d0815e710 2024-03-30T13:12:59.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.732+0000 7f0cff7fe640 1 -- 172.21.15.66:0/2990784462 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7f0d0007cc00 con 0x7f0d0815e710 2024-03-30T13:12:59.737 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-03-30T13:12:59.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0ce806c530 msgr2=0x7f0ce806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0ce806c530 0x7f0ce806e9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8000f60 tx=0x7f0cf8009040 comp rx=0 tx=0).stop 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 msgr2=0x7f0d081f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 0x7f0d081f8420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009e80 tx=0x7f0d00004a40 comp rx=0 tx=0).stop 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 shutdown_connections 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f0ce806c530 0x7f0ce806e9f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0815e710 0x7f0d081f8420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d080cef20 0x7f0d0814a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 --2- 172.21.15.66:0/2990784462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d080b66f0 0x7f0d0815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 >> 172.21.15.66:0/2990784462 conn(0x7f0d080d1210 msgr2=0x7f0d0815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 shutdown_connections 2024-03-30T13:12:59.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.736+0000 7f0d0d122640 1 -- 172.21.15.66:0/2990784462 wait complete. 2024-03-30T13:12:59.753 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T13:12:59.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.865+0000 7f2e92d50640 1 Processor -- start 2024-03-30T13:12:59.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.865+0000 7f2e92d50640 1 -- start start 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e92d50640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 0x7f2e8c1624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e92d50640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c0cc900 con 0x7f2e8c071070 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e92d50640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c0ccbe0 con 0x7f2e8c0d1b80 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e92d50640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8c0ccec0 con 0x7f2e8c0ce760 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e90aec640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2e8c071070 0x7f2e8c071450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39772/0 (socket says 172.21.15.66:39772) 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e90aec640 1 -- 172.21.15.66:0/387254278 learned_addr learned my addr 172.21.15.66:0/387254278 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e8bfff640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 0x7f2e8c1624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354863728 0 0) 0x7f2e8c0cc900 con 0x7f2e8c071070 2024-03-30T13:12:59.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.866+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e74003520 con 0x7f2e8c071070 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1563622036 0 0) 0x7f2e74003520 con 0x7f2e8c071070 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 msgr2=0x7f2e8c1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 0x7f2e8c1624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 >> v1:172.21.15.117:6789/0 conn(0x7f2e8c0d1b80 legacy=0x7f2e8c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c070290 con 0x7f2e8c071070 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f2e7c001670 con 0x7f2e8c071070 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 >> v1:172.21.15.66:6789/0 conn(0x7f2e8c071070 legacy=0x7f2e8c071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 0x7f2e74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e74004d40 0x7f2e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c0ccec0 con 0x7f2e740039e0 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7400ba70 con 0x7f2e74004d40 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7400bc70 con 0x7f2e8c0ce760 2024-03-30T13:12:59.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e90aec640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e8bfff640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 0x7f2e74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e912ed640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e74004d40 0x7f2e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e90aec640 1 -- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 msgr2=0x7f2e74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e90aec640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0ce760 0x7f2e74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e90aec640 1 -- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e74004d40 msgr2=0x7f2e740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e90aec640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e74004d40 0x7f2e740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e90aec640 1 -- 172.21.15.66:0/387254278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e74003520 con 0x7f2e740039e0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.867+0000 7f2e912ed640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e74004d40 0x7f2e740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.868+0000 7f2e90aec640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e74003e00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c001eb0 tx=0x7f2e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d08f9d11c795e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.868+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c007d30 con 0x7f2e740039e0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.868+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c030070 con 0x7f2e740039e0 2024-03-30T13:12:59.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.868+0000 7f2e8b7fe640 1 -- 172.21.15.66:0/387254278 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c005d30 con 0x7f2e740039e0 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e92d50640 1 -- 172.21.15.66:0/387254278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 msgr2=0x7f2e74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e92d50640 1 --2- 172.21.15.66:0/387254278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e74003e00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c001eb0 tx=0x7f2e74005830 comp rx=0 tx=0).stop 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e912ed640 1 -- 172.21.15.66:0/387254278 reap_dead start 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e92d50640 1 -- 172.21.15.66:0/387254278 shutdown_connections 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e92d50640 1 -- 172.21.15.66:0/387254278 >> 172.21.15.66:0/387254278 conn(0x7f2e8c0d2110 msgr2=0x7f2e8c0d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e92d50640 1 -- 172.21.15.66:0/387254278 shutdown_connections 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.869+0000 7f2e92d50640 1 -- 172.21.15.66:0/387254278 wait complete. 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.870+0000 7f2e92d50640 1 Processor -- start 2024-03-30T13:12:59.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.870+0000 7f2e92d50640 1 -- start start 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e92d50640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e8c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e92d50640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0c9bf0 0x7f2e8c15fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e92d50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c150160 0x7f2e8c1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e92d50640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c15f210 con 0x7f2e740039e0 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e92d50640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c15f090 con 0x7f2e8c150160 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e92d50640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8c0704c0 con 0x7f2e8c0c9bf0 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e8c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e8c0cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52310/0 (socket says 172.21.15.66:52310) 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 -- 172.21.15.66:0/3600741198 learned_addr learned my addr 172.21.15.66:0/3600741198 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e912ed640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c150160 0x7f2e8c1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e8bfff640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0c9bf0 0x7f2e8c15fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 -- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0c9bf0 msgr2=0x7f2e8c15fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0c9bf0 0x7f2e8c15fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 -- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c150160 msgr2=0x7f2e8c1505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c150160 0x7f2e8c1505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 -- 172.21.15.66:0/3600741198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c0caae0 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.871+0000 7f2e90aec640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e8c0cbb80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.872+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c004bf0 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.872+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c004d90 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.872+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0072b0 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.872+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c0cadc0 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.872+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c0cb380 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.873+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8c075900 con 0x7f2e740039e0 2024-03-30T13:12:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.873+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e7c048020 con 0x7f2e740039e0 2024-03-30T13:12:59.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.873+0000 7f2e897fa640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e7806c560 0x7f2e7806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:59.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.874+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5778+0+0 (secure 0 0 0) 0x7f2e7c0acab0 con 0x7f2e740039e0 2024-03-30T13:12:59.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.874+0000 7f2e8bfff640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e7806c560 0x7f2e7806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:59.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.874+0000 7f2e8bfff640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e7806c560 0x7f2e7806ea20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0ca730 tx=0x7f2e80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:59.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:12:59.878+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e7c07b080 con 0x7f2e740039e0 2024-03-30T13:13:00.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.060+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e8c075900 con 0x7f2e740039e0 2024-03-30T13:13:00.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.734+0000 7f2e897fa640 1 -- 172.21.15.66:0/3600741198 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) ==== 169+0+0 (secure 0 0 0) 0x7f2e7c07a980 con 0x7f2e740039e0 2024-03-30T13:13:00.738 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-03-30T13:13:00.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e7806c560 msgr2=0x7f2e7806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e7806c560 0x7f2e7806ea20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0ca730 tx=0x7f2e80008040 comp rx=0 tx=0).stop 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 msgr2=0x7f2e8c0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e8c0cbb80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c001a40 comp rx=0 tx=0).stop 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 shutdown_connections 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2e7806c560 0x7f2e7806ea20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e8c150160 0x7f2e8c1505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e8c0c9bf0 0x7f2e8c15fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 --2- 172.21.15.66:0/3600741198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e740039e0 0x7f2e8c0cbb80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 >> 172.21.15.66:0/3600741198 conn(0x7f2e8c0d2110 msgr2=0x7f2e8c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 shutdown_connections 2024-03-30T13:13:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.736+0000 7f2e92d50640 1 -- 172.21.15.66:0/3600741198 wait complete. 2024-03-30T13:13:00.756 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T13:13:00.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.862+0000 7fa8299f4640 1 Processor -- start 2024-03-30T13:13:00.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.862+0000 7fa8299f4640 1 -- start start 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa8299f4640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8240b43c0 0x7fa82415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa8299f4640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa824162ac0 con 0x7fa8240d03f0 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa8299f4640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa824162da0 con 0x7fa824162150 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa8299f4640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa82415eae0 con 0x7fa8240b43c0 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa8237fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8240b43c0 0x7fa82415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa822ffd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa8240d03f0 0x7fa82415f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:39774/0 (socket says 172.21.15.66:39774) 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa822ffd640 1 -- 172.21.15.66:0/3351638982 learned_addr learned my addr 172.21.15.66:0/3351638982 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.863+0000 7fa8237fe640 1 --2- 172.21.15.66:0/3351638982 >> v2:172.21.15.66:3300/0 conn(0x7fa8240b43c0 0x7fa82415ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52322/0 (socket says 172.21.15.66:52322) 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.864+0000 7fa8237fe640 1 -- 172.21.15.66:0/3351638982 >> v1:172.21.15.117:6789/0 conn(0x7fa824162150 legacy=0x7fa8240b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.864+0000 7fa8237fe640 1 -- 172.21.15.66:0/3351638982 >> v1:172.21.15.66:6789/0 conn(0x7fa8240d03f0 legacy=0x7fa82415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.864+0000 7fa8237fe640 1 -- 172.21.15.66:0/3351638982 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82415ed00 con 0x7fa8240b43c0 2024-03-30T13:13:00.868 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.864+0000 7fa821ffb640 1 -- 172.21.15.66:0/3351638982 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045848525 0 0) 0x7fa824162ac0 con 0x7fa8240d03f0 2024-03-30T13:13:00.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.864+0000 7fa8237fe640 1 --2- 172.21.15.66:0/3351638982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240b43c0 0x7fa82415ff30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa814009d90 tx=0x7fa81402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67de677ed577331 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa821ffb640 1 -- 172.21.15.66:0/3351638982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814039070 con 0x7fa8240b43c0 2024-03-30T13:13:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa821ffb640 1 -- 172.21.15.66:0/3351638982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81402cbd0 con 0x7fa8240b43c0 2024-03-30T13:13:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa821ffb640 1 -- 172.21.15.66:0/3351638982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814034440 con 0x7fa8240b43c0 2024-03-30T13:13:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 -- 172.21.15.66:0/3351638982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240b43c0 msgr2=0x7fa82415ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 --2- 172.21.15.66:0/3351638982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240b43c0 0x7fa82415ff30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa814009d90 tx=0x7fa81402be20 comp rx=0 tx=0).stop 2024-03-30T13:13:00.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 -- 172.21.15.66:0/3351638982 shutdown_connections 2024-03-30T13:13:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 --2- 172.21.15.66:0/3351638982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240b43c0 0x7fa82415ff30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 -- 172.21.15.66:0/3351638982 >> 172.21.15.66:0/3351638982 conn(0x7fa8240672c0 msgr2=0x7fa8240676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 -- 172.21.15.66:0/3351638982 shutdown_connections 2024-03-30T13:13:00.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.865+0000 7fa8299f4640 1 -- 172.21.15.66:0/3351638982 wait complete. 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.866+0000 7fa8299f4640 1 Processor -- start 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 -- start start 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240b43c0 0x7fa8240c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 0x7fa8240b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa824162150 0x7fa8240c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa824163080 con 0x7fa8240d03f0 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240cfb90 con 0x7fa8240b43c0 2024-03-30T13:13:00.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8299f4640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa82415f070 con 0x7fa824162150 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8227fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 0x7fa8240b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8227fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 0x7fa8240b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52326/0 (socket says 172.21.15.66:52326) 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8227fc640 1 -- 172.21.15.66:0/2785433748 learned_addr learned my addr 172.21.15.66:0/2785433748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa822ffd640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240b43c0 0x7fa8240c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.867+0000 7fa8237fe640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa824162150 0x7fa8240c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8227fc640 1 -- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa824162150 msgr2=0x7fa8240c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8227fc640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa824162150 0x7fa8240c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8227fc640 1 -- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240b43c0 msgr2=0x7fa8240c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8227fc640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240b43c0 0x7fa8240c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8227fc640 1 -- 172.21.15.66:0/2785433748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa824148220 con 0x7fa8240d03f0 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8227fc640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 0x7fa8240b7570 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c400 tx=0x7fa81800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa8240d03f0 2024-03-30T13:13:00.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8180052d0 con 0x7fa8240d03f0 2024-03-30T13:13:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818003830 con 0x7fa8240d03f0 2024-03-30T13:13:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82415a3f0 con 0x7fa8240d03f0 2024-03-30T13:13:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.868+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241487a0 con 0x7fa8240d03f0 2024-03-30T13:13:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.869+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa824075890 con 0x7fa8240d03f0 2024-03-30T13:13:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.869+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa81801a070 con 0x7fa8240d03f0 2024-03-30T13:13:00.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.870+0000 7fa80bfff640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa81006c530 0x7fa81006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:00.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.870+0000 7fa822ffd640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa81006c530 0x7fa81006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:00.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.870+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fa81808a820 con 0x7fa8240d03f0 2024-03-30T13:13:00.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.870+0000 7fa822ffd640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa81006c530 0x7fa81006e9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007f70 tx=0x7fa81c007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:00.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:00.874+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa818058960 con 0x7fa8240d03f0 2024-03-30T13:13:01.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.062+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa824160860 con 0x7fa8240d03f0 2024-03-30T13:13:01.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.065+0000 7fa80bfff640 1 -- 172.21.15.66:0/2785433748 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fa81805cc60 con 0x7fa8240d03f0 2024-03-30T13:13:01.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa81006c530 msgr2=0x7fa81006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa81006c530 0x7fa81006e9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007f70 tx=0x7fa81c007f00 comp rx=0 tx=0).stop 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 msgr2=0x7fa8240b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 0x7fa8240b7570 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c400 tx=0x7fa81800c8c0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 shutdown_connections 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa81006c530 0x7fa81006e9f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa824162150 0x7fa8240c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8240d03f0 0x7fa8240b7570 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 --2- 172.21.15.66:0/2785433748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8240b43c0 0x7fa8240c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 >> 172.21.15.66:0/2785433748 conn(0x7fa8240672c0 msgr2=0x7fa82415ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 shutdown_connections 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.067+0000 7fa8299f4640 1 -- 172.21.15.66:0/2785433748 wait complete. 2024-03-30T13:13:01.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-30T13:13:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 Processor -- start 2024-03-30T13:13:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 -- start start 2024-03-30T13:13:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f01445e0 0x7f64f00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f64f0072340 0x7f64f00cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f64f00727a0 0x7f64f00cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f64f00cd610 con 0x7f64f0072340 2024-03-30T13:13:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f64f01449c0 con 0x7f64f00727a0 2024-03-30T13:13:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.186+0000 7f64f5924640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f64f00cf5b0 con 0x7f64f01445e0 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f64f0072340 0x7f64f00cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f64f0072340 0x7f64f00cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52328/0 (socket says 172.21.15.66:52328) 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 -- 172.21.15.66:0/363682921 learned_addr learned my addr 172.21.15.66:0/363682921 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 -- 172.21.15.66:0/363682921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f01445e0 msgr2=0x7f64f00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64ef7fe640 1 --2- 172.21.15.66:0/363682921 >> v2:172.21.15.117:3300/0 conn(0x7f64f00727a0 0x7f64f00cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64eeffd640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f01445e0 0x7f64f00cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f01445e0 0x7f64f00cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 -- 172.21.15.66:0/363682921 >> v2:172.21.15.117:3300/0 conn(0x7f64f00727a0 msgr2=0x7f64f00cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 --2- 172.21.15.66:0/363682921 >> v2:172.21.15.117:3300/0 conn(0x7f64f00727a0 0x7f64f00cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64e67fc640 1 -- 172.21.15.66:0/363682921 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f0144e10 con 0x7f64f0072340 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64eeffd640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f01445e0 0x7f64f00cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.187+0000 7f64ef7fe640 1 --2- 172.21.15.66:0/363682921 >> v2:172.21.15.117:3300/0 conn(0x7f64f00727a0 0x7f64f00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64e67fc640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f0072340 0x7f64f00cb690 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f64f0074110 tx=0x7f64d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b2a7063f1cee29 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64ee7fc640 1 -- 172.21.15.66:0/363682921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8035020 con 0x7f64f0072340 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64ee7fc640 1 -- 172.21.15.66:0/363682921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d8031640 con 0x7f64f0072340 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64ee7fc640 1 -- 172.21.15.66:0/363682921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d802d040 con 0x7f64f0072340 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 -- 172.21.15.66:0/363682921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f0072340 msgr2=0x7f64f00cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f0072340 0x7f64f00cb690 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f64f0074110 tx=0x7f64d802b8f0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 -- 172.21.15.66:0/363682921 shutdown_connections 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 --2- 172.21.15.66:0/363682921 >> v2:172.21.15.117:3300/0 conn(0x7f64f00727a0 0x7f64f00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f0072340 0x7f64f00cb690 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 --2- 172.21.15.66:0/363682921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f01445e0 0x7f64f00cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.188+0000 7f64f5924640 1 -- 172.21.15.66:0/363682921 >> 172.21.15.66:0/363682921 conn(0x7f64f00b1c90 msgr2=0x7f64f00b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.189+0000 7f64f5924640 1 -- 172.21.15.66:0/363682921 shutdown_connections 2024-03-30T13:13:01.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.189+0000 7f64f5924640 1 -- 172.21.15.66:0/363682921 wait complete. 2024-03-30T13:13:01.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.190+0000 7f64f5924640 1 Processor -- start 2024-03-30T13:13:01.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.190+0000 7f64f5924640 1 -- start start 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f0072340 0x7f64f0137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f00727a0 0x7f64f00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 0x7f64f00b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64f00cad60 con 0x7f64f01445e0 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64f015edb0 con 0x7f64f00727a0 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f64f00cb040 con 0x7f64f0072340 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 0x7f64f00b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 0x7f64f00b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52338/0 (socket says 172.21.15.66:52338) 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 -- 172.21.15.66:0/2895063614 learned_addr learned my addr 172.21.15.66:0/2895063614 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64e67fc640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f00727a0 0x7f64f00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64eeffd640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f0072340 0x7f64f0137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 -- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f0072340 msgr2=0x7f64f0137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f0072340 0x7f64f0137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 -- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f00727a0 msgr2=0x7f64f00b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f00727a0 0x7f64f00b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 -- 172.21.15.66:0/2895063614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f0137da0 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ef7fe640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 0x7f64f00b4a10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f64e800cef0 tx=0x7f64e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e801a070 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e800d840 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f015b630 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e800dc30 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.191+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f015bbf0 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.193+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f0075a10 con 0x7f64f01445e0 2024-03-30T13:13:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.193+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f64e801d070 con 0x7f64f01445e0 2024-03-30T13:13:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.193+0000 7f64ecff9640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f64d006c530 0x7f64d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.193+0000 7f64eeffd640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f64d006c530 0x7f64d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.193+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f64e808bbd0 con 0x7f64f01445e0 2024-03-30T13:13:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.194+0000 7f64eeffd640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f64d006c530 0x7f64d006e9f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64e000aac0 tx=0x7f64e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:01.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.198+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f64e8059da0 con 0x7f64f01445e0 2024-03-30T13:13:01.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.457+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f64f00b6060 con 0x7f64f01445e0 2024-03-30T13:13:01.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.741+0000 7f64ecff9640 1 -- 172.21.15.66:0/2895063614 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v172) ==== 147+0+0 (secure 0 0 0) 0x7f64e805e0a0 con 0x7f64f01445e0 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f64d006c530 msgr2=0x7f64d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f64d006c530 0x7f64d006e9f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64e000aac0 tx=0x7f64e0008040 comp rx=0 tx=0).stop 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 msgr2=0x7f64f00b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 0x7f64f00b4a10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f64e800cef0 tx=0x7f64e800a710 comp rx=0 tx=0).stop 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 shutdown_connections 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f64d006c530 0x7f64d006e9f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f01445e0 0x7f64f00b4a10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64f00727a0 0x7f64f00b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 --2- 172.21.15.66:0/2895063614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64f0072340 0x7f64f0137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:01.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 >> 172.21.15.66:0/2895063614 conn(0x7f64f00b1c90 msgr2=0x7f64f0073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:01.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 shutdown_connections 2024-03-30T13:13:01.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:01.743+0000 7f64f5924640 1 -- 172.21.15.66:0/2895063614 wait complete. 2024-03-30T13:13:01.761 INFO:tasks.cephfs_test_runner:test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) ... ERROR 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 219, in run_test_worker 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: self.fs.mds_tell(['cache', 'drop'], i) 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1313, in mds_tell 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: return self.rank_tell(*args, **kwargs) 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-30T13:13:01.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-30T13:13:01.763 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:Ran 1 test in 150.485s 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 219, in run_test_worker 2024-03-30T13:13:01.764 INFO:tasks.cephfs_test_runner: self.fs.mds_tell(['cache', 'drop'], i) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1313, in mds_tell 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: return self.rank_tell(*args, **kwargs) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-30T13:13:01.765 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-30T13:13:01.766 INFO:tasks.cephfs_test_runner: 2024-03-30T13:13:01.766 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:13:02.018 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=79f3f4b5ec5841e8b898c6ef751a59d2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:13:02.020 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-30T13:13:02.029 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-30T13:13:02.038 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-30T13:13:02.038 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:13:02.039 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:13:02.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056354,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2256 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T13:13:02.059 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-30T13:13:02.059 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:13:02.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056354,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2256 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T13:13:02.116 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:13:02.116 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:13:02.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=179 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T13:13:02.135 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-30T13:13:02.136 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:13:02.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=179 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T13:13:02.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.194 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:13:02.210 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T13:13:02.211 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T13:13:02.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:13:02.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:13:02.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-03-30T13:13:02.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.453 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-03-30T13:13:02.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.469 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-03-30T13:13:02.543 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-03-30T13:13:02.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:13:02.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.546 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:13:02.564 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-30T13:13:02.564 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T13:13:02.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.565 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:13:02.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.636 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:13:02.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.708 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-30T13:13:02.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.811 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-03-30T13:13:02.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:02.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-03-30T13:13:02.899 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-03-30T13:13:02.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:13:02.900 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-30T13:13:02.910 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T13:13:02.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:13:03.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.018+0000 7f33023f0640 1 Processor -- start 2024-03-30T13:13:03.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.018+0000 7f33023f0640 1 -- start start 2024-03-30T13:13:03.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f33023f0640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f32fc145c60 0x7f32fc071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f33023f0640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32fc160830 0x7f32fc0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f33023f0640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32fc146040 con 0x7f32fc071a60 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f33023f0640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f32fc071ed0 con 0x7f32fc160830 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f33023f0640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f32fc160e50 con 0x7f32fc145c60 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f330098d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32fc160830 0x7f32fc0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f330098d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f32fc160830 0x7f32fc0ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59720/0 (socket says 172.21.15.66:59720) 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f330098d640 1 -- 172.21.15.66:0/3215233332 learned_addr learned my addr 172.21.15.66:0/3215233332 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:03.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.019+0000 7f32fbfff640 1 --2- 172.21.15.66:0/3215233332 >> v2:172.21.15.117:3300/0 conn(0x7f32fc145c60 0x7f32fc071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.020+0000 7f330098d640 1 -- 172.21.15.66:0/3215233332 >> v2:172.21.15.117:3300/0 conn(0x7f32fc145c60 msgr2=0x7f32fc071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:03.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.020+0000 7f330098d640 1 --2- 172.21.15.66:0/3215233332 >> v2:172.21.15.117:3300/0 conn(0x7f32fc145c60 0x7f32fc071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.020+0000 7f330098d640 1 -- 172.21.15.66:0/3215233332 >> v1:172.21.15.117:6789/0 conn(0x7f32fc071a60 legacy=0x7f32fc160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.020+0000 7f330098d640 1 -- 172.21.15.66:0/3215233332 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc15d8b0 con 0x7f32fc160830 2024-03-30T13:13:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.020+0000 7f330098d640 1 --2- 172.21.15.66:0/3215233332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc160830 0x7f32fc0ca830 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f32fc06e580 tx=0x7f32f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e832ac450da909 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f32faffd640 1 -- 172.21.15.66:0/3215233332 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f402ca80 con 0x7f32fc160830 2024-03-30T13:13:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f32faffd640 1 -- 172.21.15.66:0/3215233332 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f402cc20 con 0x7f32fc160830 2024-03-30T13:13:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f32faffd640 1 -- 172.21.15.66:0/3215233332 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40343f0 con 0x7f32fc160830 2024-03-30T13:13:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 -- 172.21.15.66:0/3215233332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc160830 msgr2=0x7f32fc0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 --2- 172.21.15.66:0/3215233332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc160830 0x7f32fc0ca830 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f32fc06e580 tx=0x7f32f402bee0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 -- 172.21.15.66:0/3215233332 shutdown_connections 2024-03-30T13:13:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 --2- 172.21.15.66:0/3215233332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc160830 0x7f32fc0ca830 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 --2- 172.21.15.66:0/3215233332 >> v2:172.21.15.117:3300/0 conn(0x7f32fc145c60 0x7f32fc071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 -- 172.21.15.66:0/3215233332 >> 172.21.15.66:0/3215233332 conn(0x7f32fc0b1c90 msgr2=0x7f32fc0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 -- 172.21.15.66:0/3215233332 shutdown_connections 2024-03-30T13:13:03.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.021+0000 7f33023f0640 1 -- 172.21.15.66:0/3215233332 wait complete. 2024-03-30T13:13:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.022+0000 7f33023f0640 1 Processor -- start 2024-03-30T13:13:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 -- start start 2024-03-30T13:13:03.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 0x7f32fc13e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32fc145c60 0x7f32fc13bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32fc160830 0x7f32fc14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc15d0b0 con 0x7f32fc071a60 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc15cdd0 con 0x7f32fc145c60 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f33023f0640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc0cad70 con 0x7f32fc160830 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 0x7f32fc13e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 0x7f32fc13e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59732/0 (socket says 172.21.15.66:59732) 2024-03-30T13:13:03.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.023+0000 7f32fbfff640 1 -- 172.21.15.66:0/701931209 learned_addr learned my addr 172.21.15.66:0/701931209 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f330098d640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32fc160830 0x7f32fc14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fb7fe640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32fc145c60 0x7f32fc13bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fbfff640 1 -- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32fc160830 msgr2=0x7f32fc14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fbfff640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32fc160830 0x7f32fc14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fbfff640 1 -- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32fc145c60 msgr2=0x7f32fc13bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fbfff640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32fc145c60 0x7f32fc13bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fbfff640 1 -- 172.21.15.66:0/701931209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc13c300 con 0x7f32fc071a60 2024-03-30T13:13:03.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fb7fe640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32fc145c60 0x7f32fc13bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32fbfff640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 0x7f32fc13e3e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f32f000e9b0 tx=0x7f32f000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0017070 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f000c9a0 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc14ea20 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f000cd90 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.024+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc13ed40 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.025+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32fc143960 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.026+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f32f001a070 con 0x7f32fc071a60 2024-03-30T13:13:03.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.026+0000 7f32f8ff9640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f32e806c530 0x7f32e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.026+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f32f008b610 con 0x7f32fc071a60 2024-03-30T13:13:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.026+0000 7f32fb7fe640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f32e806c530 0x7f32e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.026+0000 7f32fb7fe640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f32e806c530 0x7f32e806e9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f32fc141c60 tx=0x7f32ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.030+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f32fc143960 con 0x7f32fc071a60 2024-03-30T13:13:03.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.213+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32fc075a10 con 0x7f32e806c530 2024-03-30T13:13:03.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.214+0000 7f32f8ff9640 1 -- 172.21.15.66:0/701931209 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7f32fc075a10 con 0x7f32e806c530 2024-03-30T13:13:03.218 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:03.218 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.215+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f32e806c530 msgr2=0x7f32e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.215+0000 7f33023f0640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f32e806c530 0x7f32e806e9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f32fc141c60 tx=0x7f32ec009040 comp rx=0 tx=0).stop 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.215+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 msgr2=0x7f32fc13e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.215+0000 7f33023f0640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 0x7f32fc13e3e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f32f000e9b0 tx=0x7f32f000ee70 comp rx=0 tx=0).stop 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 shutdown_connections 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f32e806c530 0x7f32e806e9f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32fc160830 0x7f32fc14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32fc145c60 0x7f32fc13bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 --2- 172.21.15.66:0/701931209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32fc071a60 0x7f32fc13e3e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 >> 172.21.15.66:0/701931209 conn(0x7f32fc0b1c90 msgr2=0x7f32fc143250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 shutdown_connections 2024-03-30T13:13:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.216+0000 7f33023f0640 1 -- 172.21.15.66:0/701931209 wait complete. 2024-03-30T13:13:03.230 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-03-30T13:13:02.832684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":585192,"kb_used_data":352400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754389528,"statfs":{"total":773094113280,"available":772494876672,"internally_reserved":0,"allocated":360857600,"data_stored":358118768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-177493347,"num_objects":-139,"num_object_clones":0,"num_object_copies":-278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-139,"num_whiteouts":0,"num_read":-198,"num_read_kb":-168593,"num_write":-924,"num_write_kb":-176170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-03-30T13:12:59.738376+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:12:59.738376+0000","last_peered":"2024-03-30T13:12:59.738376+0000","last_clean":"2024-03-30T13:12:59.738376+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:12:59.738376+0000","last_undegraded":"2024-03-30T13:12:59.738376+0000","last_fullsized":"2024-03-30T13:12:59.738376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97612,"kb_used_data":66432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94274228,"statfs":{"total":96636764160,"available":96536809472,"internally_reserved":0,"allocated":68026368,"data_stored":67667611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69332,"kb_used_data":42312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94302508,"statfs":{"total":96636764160,"available":96565768192,"internally_reserved":0,"allocated":43327488,"data_stored":43002752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44284,"kb_used_data":17264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327556,"statfs":{"total":96636764160,"available":96591417344,"internally_reserved":0,"allocated":17678336,"data_stored":17349269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89368,"kb_used_data":58192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282472,"statfs":{"total":96636764160,"available":96545251328,"internally_reserved":0,"allocated":59588608,"data_stored":59231621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73512,"kb_used_data":42332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298328,"statfs":{"total":96636764160,"available":96561487872,"internally_reserved":0,"allocated":43347968,"data_stored":43005212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77116,"kb_used_data":50100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294724,"statfs":{"total":96636764160,"available":96557797376,"internally_reserved":0,"allocated":51302400,"data_stored":50970547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":13228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":13545472,"data_stored":13208734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93720,"kb_used_data":62540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94278120,"statfs":{"total":96636764160,"available":96540794880,"internally_reserved":0,"allocated":64040960,"data_stored":63683022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:13:03.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:13:03.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.335+0000 7f5c16a37640 1 Processor -- start 2024-03-30T13:13:03.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.335+0000 7f5c16a37640 1 -- start start 2024-03-30T13:13:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c16a37640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5c10155990 0x7f5c1006c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c16a37640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5c100b4050 0x7f5c100cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c16a37640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c101624f0 con 0x7f5c100d0300 2024-03-30T13:13:03.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c16a37640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5c100d0750 con 0x7f5c100b4050 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c16a37640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f5c101622a0 con 0x7f5c10155990 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c0f7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5c100b4050 0x7f5c100cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c14fd4640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5c100d0300 0x7f5c1015eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60184/0 (socket says 172.21.15.66:60184) 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.336+0000 7f5c14fd4640 1 -- 172.21.15.66:0/2997317102 learned_addr learned my addr 172.21.15.66:0/2997317102 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0ffff640 1 --2- 172.21.15.66:0/2997317102 >> v2:172.21.15.117:3300/0 conn(0x7f5c10155990 0x7f5c1006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0effd640 1 -- 172.21.15.66:0/2997317102 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571850617 0 0) 0x7f5c101624f0 con 0x7f5c100d0300 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0effd640 1 -- 172.21.15.66:0/2997317102 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bf8003520 con 0x7f5c100d0300 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0f7fe640 1 -- 172.21.15.66:0/2997317102 >> v2:172.21.15.117:3300/0 conn(0x7f5c10155990 msgr2=0x7f5c1006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0f7fe640 1 --2- 172.21.15.66:0/2997317102 >> v2:172.21.15.117:3300/0 conn(0x7f5c10155990 0x7f5c1006c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0f7fe640 1 -- 172.21.15.66:0/2997317102 >> v1:172.21.15.66:6789/0 conn(0x7f5c100d0300 legacy=0x7f5c1015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.337+0000 7f5c0f7fe640 1 -- 172.21.15.66:0/2997317102 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c101627d0 con 0x7f5c100b4050 2024-03-30T13:13:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c0f7fe640 1 --2- 172.21.15.66:0/2997317102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c100cfdc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5c100680e0 tx=0x7f5c0402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3578a46c3103c703 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c0effd640 1 -- 172.21.15.66:0/2997317102 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040315a0 con 0x7f5c100b4050 2024-03-30T13:13:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c0effd640 1 -- 172.21.15.66:0/2997317102 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c04037070 con 0x7f5c100b4050 2024-03-30T13:13:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c0effd640 1 -- 172.21.15.66:0/2997317102 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04031960 con 0x7f5c100b4050 2024-03-30T13:13:03.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 -- 172.21.15.66:0/2997317102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 msgr2=0x7f5c100cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2997317102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c100cfdc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5c100680e0 tx=0x7f5c0402b840 comp rx=0 tx=0).stop 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 -- 172.21.15.66:0/2997317102 shutdown_connections 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2997317102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c100cfdc0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2997317102 >> v2:172.21.15.117:3300/0 conn(0x7f5c10155990 0x7f5c1006c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 -- 172.21.15.66:0/2997317102 >> 172.21.15.66:0/2997317102 conn(0x7f5c100672c0 msgr2=0x7f5c100676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 -- 172.21.15.66:0/2997317102 shutdown_connections 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.338+0000 7f5c16a37640 1 -- 172.21.15.66:0/2997317102 wait complete. 2024-03-30T13:13:03.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.339+0000 7f5c16a37640 1 Processor -- start 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 -- start start 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c10157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c100d0300 0x7f5c10147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10155990 0x7f5c1015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c100cf9a0 con 0x7f5c100b4050 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10162d90 con 0x7f5c10155990 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c16a37640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5c100cfb20 con 0x7f5c100d0300 2024-03-30T13:13:03.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c10157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c10157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59740/0 (socket says 172.21.15.66:59740) 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.340+0000 7f5c0ffff640 1 -- 172.21.15.66:0/2289609762 learned_addr learned my addr 172.21.15.66:0/2289609762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0f7fe640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c100d0300 0x7f5c10147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c14fd4640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10155990 0x7f5c1015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0ffff640 1 -- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c100d0300 msgr2=0x7f5c10147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0ffff640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c100d0300 0x7f5c10147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0ffff640 1 -- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10155990 msgr2=0x7f5c1015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0ffff640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10155990 0x7f5c1015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0ffff640 1 -- 172.21.15.66:0/2289609762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c100ce510 con 0x7f5c100b4050 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c14fd4640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10155990 0x7f5c1015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0ffff640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c10157540 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e9e0 tx=0x7f5c0000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00017070 con 0x7f5c100b4050 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0000c9d0 con 0x7f5c100b4050 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1015c190 con 0x7f5c100b4050 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000cdc0 con 0x7f5c100b4050 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.341+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101563c0 con 0x7f5c100b4050 2024-03-30T13:13:03.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.342+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c100b4500 con 0x7f5c100b4050 2024-03-30T13:13:03.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.343+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5c0001a070 con 0x7f5c100b4050 2024-03-30T13:13:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.343+0000 7f5c0cff9640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5bfc06c530 0x7f5bfc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.343+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f5c0008b570 con 0x7f5c100b4050 2024-03-30T13:13:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.343+0000 7f5c0f7fe640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5bfc06c530 0x7f5bfc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.343+0000 7f5c0f7fe640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5bfc06c530 0x7f5bfc06e9f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c04036000 tx=0x7f5c0403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.348+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5c100b4500 con 0x7f5c100b4050 2024-03-30T13:13:03.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.531+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c10075a10 con 0x7f5bfc06c530 2024-03-30T13:13:03.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.531+0000 7f5c0cff9640 1 -- 172.21.15.66:0/2289609762 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7f5c10075a10 con 0x7f5bfc06c530 2024-03-30T13:13:03.536 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:03.536 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:13:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5bfc06c530 msgr2=0x7f5bfc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5bfc06c530 0x7f5bfc06e9f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c04036000 tx=0x7f5c0403d040 comp rx=0 tx=0).stop 2024-03-30T13:13:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 msgr2=0x7f5c10157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c10157540 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000e9e0 tx=0x7f5c0000eea0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 shutdown_connections 2024-03-30T13:13:03.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5bfc06c530 0x7f5bfc06e9f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10155990 0x7f5c1015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c100d0300 0x7f5c10147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 --2- 172.21.15.66:0/2289609762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c100b4050 0x7f5c10157540 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 >> 172.21.15.66:0/2289609762 conn(0x7f5c100672c0 msgr2=0x7f5c100b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 shutdown_connections 2024-03-30T13:13:03.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.534+0000 7f5c16a37640 1 -- 172.21.15.66:0/2289609762 wait complete. 2024-03-30T13:13:03.549 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-03-30T13:13:02.832684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":585192,"kb_used_data":352400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754389528,"statfs":{"total":773094113280,"available":772494876672,"internally_reserved":0,"allocated":360857600,"data_stored":358118768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-177493347,"num_objects":-139,"num_object_clones":0,"num_object_copies":-278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-139,"num_whiteouts":0,"num_read":-198,"num_read_kb":-168593,"num_write":-924,"num_write_kb":-176170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-03-30T13:12:59.738376+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:12:59.738376+0000","last_peered":"2024-03-30T13:12:59.738376+0000","last_clean":"2024-03-30T13:12:59.738376+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:12:59.738376+0000","last_undegraded":"2024-03-30T13:12:59.738376+0000","last_fullsized":"2024-03-30T13:12:59.738376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97612,"kb_used_data":66432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94274228,"statfs":{"total":96636764160,"available":96536809472,"internally_reserved":0,"allocated":68026368,"data_stored":67667611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69332,"kb_used_data":42312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94302508,"statfs":{"total":96636764160,"available":96565768192,"internally_reserved":0,"allocated":43327488,"data_stored":43002752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44284,"kb_used_data":17264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327556,"statfs":{"total":96636764160,"available":96591417344,"internally_reserved":0,"allocated":17678336,"data_stored":17349269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89368,"kb_used_data":58192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282472,"statfs":{"total":96636764160,"available":96545251328,"internally_reserved":0,"allocated":59588608,"data_stored":59231621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73512,"kb_used_data":42332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298328,"statfs":{"total":96636764160,"available":96561487872,"internally_reserved":0,"allocated":43347968,"data_stored":43005212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77116,"kb_used_data":50100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294724,"statfs":{"total":96636764160,"available":96557797376,"internally_reserved":0,"allocated":51302400,"data_stored":50970547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":13228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":13545472,"data_stored":13208734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93720,"kb_used_data":62540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94278120,"statfs":{"total":96636764160,"available":96540794880,"internally_reserved":0,"allocated":64040960,"data_stored":63683022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:13:03.549 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T13:13:03.550 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:13:03.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.652+0000 7f8b00ed3640 1 Processor -- start 2024-03-30T13:13:03.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.652+0000 7f8b00ed3640 1 -- start start 2024-03-30T13:13:03.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.653+0000 7f8b00ed3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc15e950 0x7f8afc0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.653+0000 7f8b00ed3640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc0d36f0 con 0x7f8afc0d30d0 2024-03-30T13:13:03.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.653+0000 7f8b00ed3640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc0d39d0 con 0x7f8afc0d48c0 2024-03-30T13:13:03.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.653+0000 7f8b00ed3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc0d3cb0 con 0x7f8afc15e950 2024-03-30T13:13:03.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.653+0000 7f8afad9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8afc0d30d0 0x7f8afc0b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60188/0 (socket says 172.21.15.66:60188) 2024-03-30T13:13:03.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8afad9d640 1 -- 172.21.15.66:0/2820130542 learned_addr learned my addr 172.21.15.66:0/2820130542 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8afa59c640 1 --2- 172.21.15.66:0/2820130542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc15e950 0x7f8afc0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8af959a640 1 -- 172.21.15.66:0/2820130542 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68511699 0 0) 0x7f8afc0d36f0 con 0x7f8afc0d30d0 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8af959a640 1 -- 172.21.15.66:0/2820130542 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ae8003520 con 0x7f8afc0d30d0 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8afa59c640 1 -- 172.21.15.66:0/2820130542 >> v1:172.21.15.117:6789/0 conn(0x7f8afc0d48c0 legacy=0x7f8afc0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8afa59c640 1 -- 172.21.15.66:0/2820130542 >> v1:172.21.15.66:6789/0 conn(0x7f8afc0d30d0 legacy=0x7f8afc0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8af959a640 1 -- 172.21.15.66:0/2820130542 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1569085836 0 0) 0x7f8ae8003520 con 0x7f8afc0d30d0 2024-03-30T13:13:03.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.654+0000 7f8afa59c640 1 -- 172.21.15.66:0/2820130542 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc0b7ac0 con 0x7f8afc15e950 2024-03-30T13:13:03.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.655+0000 7f8afa59c640 1 --2- 172.21.15.66:0/2820130542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc15e950 0x7f8afc0d4380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8af40097d0 tx=0x7f8af402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e750375ee992579 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.655+0000 7f8af959a640 1 -- 172.21.15.66:0/2820130542 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0019f0 con 0x7f8afc15e950 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.655+0000 7f8af959a640 1 -- 172.21.15.66:0/2820130542 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4037070 con 0x7f8afc15e950 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.655+0000 7f8af959a640 1 -- 172.21.15.66:0/2820130542 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af40317a0 con 0x7f8afc15e950 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 -- 172.21.15.66:0/2820130542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc15e950 msgr2=0x7f8afc0d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/2820130542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc15e950 0x7f8afc0d4380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8af40097d0 tx=0x7f8af402b8a0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 -- 172.21.15.66:0/2820130542 shutdown_connections 2024-03-30T13:13:03.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/2820130542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc15e950 0x7f8afc0d4380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 -- 172.21.15.66:0/2820130542 >> 172.21.15.66:0/2820130542 conn(0x7f8afc0676e0 msgr2=0x7f8afc067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 -- 172.21.15.66:0/2820130542 shutdown_connections 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.656+0000 7f8b00ed3640 1 -- 172.21.15.66:0/2820130542 wait complete. 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.657+0000 7f8b00ed3640 1 Processor -- start 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.657+0000 7f8b00ed3640 1 -- start start 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8b00ed3640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc0d30d0 0x7f8afc14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8b00ed3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 0x7f8afc15f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8b00ed3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc15e950 0x7f8afc15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8b00ed3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc15ed30 con 0x7f8afc0d48c0 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8b00ed3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0d4d00 con 0x7f8afc15e950 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8b00ed3640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc15eeb0 con 0x7f8afc0d30d0 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 0x7f8afc15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 0x7f8afc15f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59750/0 (socket says 172.21.15.66:59750) 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 -- 172.21.15.66:0/3239060630 learned_addr learned my addr 172.21.15.66:0/3239060630 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:03.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8afa59c640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc0d30d0 0x7f8afc14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8afad9d640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc15e950 0x7f8afc15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 -- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc0d30d0 msgr2=0x7f8afc14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc0d30d0 0x7f8afc14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 -- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc15e950 msgr2=0x7f8afc15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc15e950 0x7f8afc15f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 -- 172.21.15.66:0/3239060630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc160d80 con 0x7f8afc0d48c0 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8afa59c640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc0d30d0 0x7f8afc14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:03.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8afad9d640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc15e950 0x7f8afc15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.658+0000 7f8af9d9b640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 0x7f8afc15f090 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c450 tx=0x7f8ae400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.659+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40052d0 con 0x7f8afc0d48c0 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.659+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4005470 con 0x7f8afc0d48c0 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.659+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc15fce0 con 0x7f8afc0d48c0 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.659+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4012870 con 0x7f8afc0d48c0 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.659+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc1602a0 con 0x7f8afc0d48c0 2024-03-30T13:13:03.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.660+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8afc159110 con 0x7f8afc0d48c0 2024-03-30T13:13:03.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.660+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8ae4012a10 con 0x7f8afc0d48c0 2024-03-30T13:13:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.661+0000 7f8af2ffd640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8adc06c530 0x7f8adc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.661+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f8ae408b170 con 0x7f8afc0d48c0 2024-03-30T13:13:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.661+0000 7f8afa59c640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8adc06c530 0x7f8adc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.661+0000 7f8afa59c640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8adc06c530 0x7f8adc06e9f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8af4036000 tx=0x7f8af403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.665+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8ae4059910 con 0x7f8afc0d48c0 2024-03-30T13:13:03.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.849+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8afc0cf710 con 0x7f8adc06c530 2024-03-30T13:13:03.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.850+0000 7f8af2ffd640 1 -- 172.21.15.66:0/3239060630 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7f8af404d4d0 con 0x7f8adc06c530 2024-03-30T13:13:03.854 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:03.855 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:13:03.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8adc06c530 msgr2=0x7f8adc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8adc06c530 0x7f8adc06e9f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8af4036000 tx=0x7f8af403d040 comp rx=0 tx=0).stop 2024-03-30T13:13:03.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 msgr2=0x7f8afc15f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 0x7f8afc15f090 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c450 tx=0x7f8ae400c910 comp rx=0 tx=0).stop 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 shutdown_connections 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8adc06c530 0x7f8adc06e9f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc15e950 0x7f8afc15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8afc0d48c0 0x7f8afc15f090 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 --2- 172.21.15.66:0/3239060630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc0d30d0 0x7f8afc14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 >> 172.21.15.66:0/3239060630 conn(0x7f8afc0676e0 msgr2=0x7f8afc15dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 shutdown_connections 2024-03-30T13:13:03.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.852+0000 7f8b00ed3640 1 -- 172.21.15.66:0/3239060630 wait complete. 2024-03-30T13:13:03.866 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-03-30T13:13:02.832684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":585192,"kb_used_data":352400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754389528,"statfs":{"total":773094113280,"available":772494876672,"internally_reserved":0,"allocated":360857600,"data_stored":358118768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-177493347,"num_objects":-139,"num_object_clones":0,"num_object_copies":-278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-139,"num_whiteouts":0,"num_read":-198,"num_read_kb":-168593,"num_write":-924,"num_write_kb":-176170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":220,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-03-30T13:12:59.738376+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:12:59.738376+0000","last_peered":"2024-03-30T13:12:59.738376+0000","last_clean":"2024-03-30T13:12:59.738376+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:12:59.738376+0000","last_undegraded":"2024-03-30T13:12:59.738376+0000","last_fullsized":"2024-03-30T13:12:59.738376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97612,"kb_used_data":66432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94274228,"statfs":{"total":96636764160,"available":96536809472,"internally_reserved":0,"allocated":68026368,"data_stored":67667611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69332,"kb_used_data":42312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94302508,"statfs":{"total":96636764160,"available":96565768192,"internally_reserved":0,"allocated":43327488,"data_stored":43002752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44284,"kb_used_data":17264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327556,"statfs":{"total":96636764160,"available":96591417344,"internally_reserved":0,"allocated":17678336,"data_stored":17349269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89368,"kb_used_data":58192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282472,"statfs":{"total":96636764160,"available":96545251328,"internally_reserved":0,"allocated":59588608,"data_stored":59231621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73512,"kb_used_data":42332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298328,"statfs":{"total":96636764160,"available":96561487872,"internally_reserved":0,"allocated":43347968,"data_stored":43005212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77116,"kb_used_data":50100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294724,"statfs":{"total":96636764160,"available":96557797376,"internally_reserved":0,"allocated":51302400,"data_stored":50970547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":13228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":13545472,"data_stored":13208734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93720,"kb_used_data":62540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94278120,"statfs":{"total":96636764160,"available":96540794880,"internally_reserved":0,"allocated":64040960,"data_stored":63683022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:13:03.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:13:03.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 Processor -- start 2024-03-30T13:13:03.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 -- start start 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f91880d4a90 0x7f91880d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d2f00 0x7f91880b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91880d36f0 con 0x7f918815e950 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f91880d3cb0 con 0x7f91880d4a90 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.971+0000 7f918ee5b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f91880d39d0 con 0x7f91880d2f00 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f91880d4a90 0x7f91880d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f91880d4a90 0x7f91880d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59762/0 (socket says 172.21.15.66:59762) 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 -- 172.21.15.66:0/3706139649 learned_addr learned my addr 172.21.15.66:0/3706139649 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f918d3f8640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d2f00 0x7f91880b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 -- 172.21.15.66:0/3706139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d2f00 msgr2=0x7f91880b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d2f00 0x7f91880b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 -- 172.21.15.66:0/3706139649 >> v1:172.21.15.117:6789/0 conn(0x7f918815e950 legacy=0x7f91880d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f9187fff640 1 -- 172.21.15.66:0/3706139649 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91880b7ac0 con 0x7f91880d4a90 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.972+0000 7f918d3f8640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d2f00 0x7f91880b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:03.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.973+0000 7f9187fff640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91880d4a90 0x7f91880d29c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9178009870 tx=0x7f917802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e5191a6815390 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.973+0000 7f91877fe640 1 -- 172.21.15.66:0/3706139649 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917802f840 con 0x7f91880d4a90 2024-03-30T13:13:03.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.973+0000 7f91877fe640 1 -- 172.21.15.66:0/3706139649 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917802f9e0 con 0x7f91880d4a90 2024-03-30T13:13:03.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.973+0000 7f91877fe640 1 -- 172.21.15.66:0/3706139649 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917802fdb0 con 0x7f91880d4a90 2024-03-30T13:13:03.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.973+0000 7f918ee5b640 1 -- 172.21.15.66:0/3706139649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91880d4a90 msgr2=0x7f91880d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.973+0000 7f918ee5b640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91880d4a90 0x7f91880d29c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9178009870 tx=0x7f917802ba20 comp rx=0 tx=0).stop 2024-03-30T13:13:03.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.974+0000 7f918ee5b640 1 -- 172.21.15.66:0/3706139649 shutdown_connections 2024-03-30T13:13:03.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.974+0000 7f918ee5b640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d2f00 0x7f91880b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.974+0000 7f918ee5b640 1 --2- 172.21.15.66:0/3706139649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91880d4a90 0x7f91880d29c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.974+0000 7f918ee5b640 1 -- 172.21.15.66:0/3706139649 >> 172.21.15.66:0/3706139649 conn(0x7f91880676e0 msgr2=0x7f9188067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.974+0000 7f918ee5b640 1 -- 172.21.15.66:0/3706139649 shutdown_connections 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.974+0000 7f918ee5b640 1 -- 172.21.15.66:0/3706139649 wait complete. 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.975+0000 7f918ee5b640 1 Processor -- start 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.975+0000 7f918ee5b640 1 -- start start 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880d2f00 0x7f91881f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d4a90 0x7f9188156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 0x7f91881579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f918815ed30 con 0x7f918815e950 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91880d33b0 con 0x7f91880d2f00 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f918815eeb0 con 0x7f91880d4a90 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 0x7f91881579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 0x7f91881579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59768/0 (socket says 172.21.15.66:59768) 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 -- 172.21.15.66:0/2680816627 learned_addr learned my addr 172.21.15.66:0/2680816627 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f9187fff640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d4a90 0x7f9188156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918cbf7640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880d2f00 0x7f91881f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 -- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d4a90 msgr2=0x7f9188156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d4a90 0x7f9188156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 -- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880d2f00 msgr2=0x7f91881f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880d2f00 0x7f91881f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 -- 172.21.15.66:0/2680816627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9188156a50 con 0x7f918815e950 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918cbf7640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880d2f00 0x7f91881f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f9187fff640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d4a90 0x7f9188156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918d3f8640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 0x7f91881579b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f918000cd80 tx=0x7f918000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918000f680 con 0x7f918815e950 2024-03-30T13:13:03.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918000f840 con 0x7f918815e950 2024-03-30T13:13:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9188156d30 con 0x7f918815e950 2024-03-30T13:13:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.976+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918000fc30 con 0x7f918815e950 2024-03-30T13:13:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.977+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91881572f0 con 0x7f918815e950 2024-03-30T13:13:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.978+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9188075a10 con 0x7f918815e950 2024-03-30T13:13:03.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.978+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f91800213f0 con 0x7f918815e950 2024-03-30T13:13:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.978+0000 7f91857fa640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f916806c530 0x7f916806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.979+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f918008c300 con 0x7f918815e950 2024-03-30T13:13:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.979+0000 7f918cbf7640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f916806c530 0x7f916806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.979+0000 7f918cbf7640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f916806c530 0x7f916806e9f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9170001700 tx=0x7f9170009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:03.983+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9180014030 con 0x7f918815e950 2024-03-30T13:13:04.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.167+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91880b8960 con 0x7f918815e950 2024-03-30T13:13:04.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.168+0000 7f91857fa640 1 -- 172.21.15.66:0/2680816627 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13281 (secure 0 0 0) 0x7f918005aa90 con 0x7f918815e950 2024-03-30T13:13:04.172 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:04.173 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":48,"fsid":"03992909-781c-43b4-940b-d3d6d831de0b","created":"2024-03-30T13:09:18.390141+0000","modified":"2024-03-30T13:13:00.733642+0000","last_up_change":"2024-03-30T13:09:29.065006+0000","last_in_change":"2024-03-30T13:09:20.620223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:09:28.807377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ed397a5b-f6cc-47da-901b-1503da0fb2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6817","nonce":3495913091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6819","nonce":3495913091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6823","nonce":3495913091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3495913091},{"type":"v1","addr":"172.21.15.66:6821","nonce":3495913091}]},"public_addr":"172.21.15.66:6817/3495913091","cluster_addr":"172.21.15.66:6819/3495913091","heartbeat_back_addr":"172.21.15.66:6823/3495913091","heartbeat_front_addr":"172.21.15.66:6821/3495913091","state":["exists","up"]},{"osd":1,"uuid":"7d89d47e-4ac4-4b2d-9197-96f97a2229e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6809","nonce":1885904429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6811","nonce":1885904429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6815","nonce":1885904429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1885904429},{"type":"v1","addr":"172.21.15.66:6813","nonce":1885904429}]},"public_addr":"172.21.15.66:6809/1885904429","cluster_addr":"172.21.15.66:6811/1885904429","heartbeat_back_addr":"172.21.15.66:6815/1885904429","heartbeat_front_addr":"172.21.15.66:6813/1885904429","state":["exists","up"]},{"osd":2,"uuid":"f1273686-748a-40d1-9fab-ef964bbe5595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6801","nonce":948131126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6803","nonce":948131126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6807","nonce":948131126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":948131126},{"type":"v1","addr":"172.21.15.66:6805","nonce":948131126}]},"public_addr":"172.21.15.66:6801/948131126","cluster_addr":"172.21.15.66:6803/948131126","heartbeat_back_addr":"172.21.15.66:6807/948131126","heartbeat_front_addr":"172.21.15.66:6805/948131126","state":["exists","up"]},{"osd":3,"uuid":"13f783a3-d401-4d87-bb74-7ba74809c37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6825","nonce":2124064569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6827","nonce":2124064569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6831","nonce":2124064569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2124064569},{"type":"v1","addr":"172.21.15.66:6829","nonce":2124064569}]},"public_addr":"172.21.15.66:6825/2124064569","cluster_addr":"172.21.15.66:6827/2124064569","heartbeat_back_addr":"172.21.15.66:6831/2124064569","heartbeat_front_addr":"172.21.15.66:6829/2124064569","state":["exists","up"]},{"osd":4,"uuid":"dbc2a753-dc2b-4e4f-affd-3819d14fe865","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6825","nonce":2408712778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6827","nonce":2408712778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6831","nonce":2408712778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2408712778},{"type":"v1","addr":"172.21.15.117:6829","nonce":2408712778}]},"public_addr":"172.21.15.117:6825/2408712778","cluster_addr":"172.21.15.117:6827/2408712778","heartbeat_back_addr":"172.21.15.117:6831/2408712778","heartbeat_front_addr":"172.21.15.117:6829/2408712778","state":["exists","up"]},{"osd":5,"uuid":"7c3189d7-05b3-4e03-8797-ce35e70a9db1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6804","nonce":1603026034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6808","nonce":1603026034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6815","nonce":1603026034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1603026034},{"type":"v1","addr":"172.21.15.117:6812","nonce":1603026034}]},"public_addr":"172.21.15.117:6804/1603026034","cluster_addr":"172.21.15.117:6808/1603026034","heartbeat_back_addr":"172.21.15.117:6815/1603026034","heartbeat_front_addr":"172.21.15.117:6812/1603026034","state":["exists","up"]},{"osd":6,"uuid":"b3a28f78-833e-4970-b45f-8527734c119d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6801","nonce":698765583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6805","nonce":698765583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6813","nonce":698765583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":698765583},{"type":"v1","addr":"172.21.15.117:6809","nonce":698765583}]},"public_addr":"172.21.15.117:6801/698765583","cluster_addr":"172.21.15.117:6805/698765583","heartbeat_back_addr":"172.21.15.117:6813/698765583","heartbeat_front_addr":"172.21.15.117:6809/698765583","state":["exists","up"]},{"osd":7,"uuid":"92d584f2-2882-4eaf-962f-045287f91202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6817","nonce":3946720435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6819","nonce":3946720435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6823","nonce":3946720435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3946720435},{"type":"v1","addr":"172.21.15.117:6821","nonce":3946720435}]},"public_addr":"172.21.15.117:6817/3946720435","cluster_addr":"172.21.15.117:6819/3946720435","heartbeat_back_addr":"172.21.15.117:6823/3946720435","heartbeat_front_addr":"172.21.15.117:6821/3946720435","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.801071+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.717056+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.754985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.772544+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.852694+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.736713+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.803068+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:09:25.827144+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6837/1537031567":"2024-03-31T13:12:57.723981+0000","172.21.15.66:6835/1537031567":"2024-03-31T13:12:57.723981+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:13:04.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f916806c530 msgr2=0x7f916806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f916806c530 0x7f916806e9f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9170001700 tx=0x7f9170009040 comp rx=0 tx=0).stop 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 msgr2=0x7f91881579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 0x7f91881579b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f918000cd80 tx=0x7f918000a650 comp rx=0 tx=0).stop 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 shutdown_connections 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f916806c530 0x7f916806e9f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f918815e950 0x7f91881579b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880d4a90 0x7f9188156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 --2- 172.21.15.66:0/2680816627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880d2f00 0x7f91881f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 >> 172.21.15.66:0/2680816627 conn(0x7f91880676e0 msgr2=0x7f91880d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:04.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 shutdown_connections 2024-03-30T13:13:04.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:04.170+0000 7f918ee5b640 1 -- 172.21.15.66:0/2680816627 wait complete. 2024-03-30T13:13:05.186 INFO:tasks.ceph:Scrubbing osd.0 2024-03-30T13:13:05.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:05.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.297+0000 7fa051cc5640 1 Processor -- start 2024-03-30T13:13:05.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.297+0000 7fa051cc5640 1 -- start start 2024-03-30T13:13:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa051cc5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa04c0cd950 0x7fa04c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa051cc5640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa04c0d03e0 0x7fa04c0729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa051cc5640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c0b2fc0 con 0x7fa04c144a80 2024-03-30T13:13:05.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa051cc5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c0d0830 con 0x7fa04c0cd950 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa051cc5640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c0b32a0 con 0x7fa04c0d03e0 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04affd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa04c0cd950 0x7fa04c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04b7fe640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa04c144a80 0x7fa04c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60198/0 (socket says 172.21.15.66:60198) 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04b7fe640 1 -- 172.21.15.66:0/4072107629 learned_addr learned my addr 172.21.15.66:0/4072107629 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04bfff640 1 --2- 172.21.15.66:0/4072107629 >> v2:172.21.15.117:3300/0 conn(0x7fa04c0d03e0 0x7fa04c0729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04a7fc640 1 -- 172.21.15.66:0/4072107629 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123544793 0 0) 0x7fa04c0b2fc0 con 0x7fa04c144a80 2024-03-30T13:13:05.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04a7fc640 1 -- 172.21.15.66:0/4072107629 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa034003520 con 0x7fa04c144a80 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04affd640 1 -- 172.21.15.66:0/4072107629 >> v2:172.21.15.117:3300/0 conn(0x7fa04c0d03e0 msgr2=0x7fa04c0729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04affd640 1 --2- 172.21.15.66:0/4072107629 >> v2:172.21.15.117:3300/0 conn(0x7fa04c0d03e0 0x7fa04c0729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04affd640 1 -- 172.21.15.66:0/4072107629 >> v1:172.21.15.66:6789/0 conn(0x7fa04c144a80 legacy=0x7fa04c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.298+0000 7fa04affd640 1 -- 172.21.15.66:0/4072107629 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0b3580 con 0x7fa04c0cd950 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.299+0000 7fa04affd640 1 --2- 172.21.15.66:0/4072107629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0cd950 0x7fa04c0cfea0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0712b0 tx=0x7fa03c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a661771df8238087 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.299+0000 7fa04a7fc640 1 -- 172.21.15.66:0/4072107629 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c035020 con 0x7fa04c0cd950 2024-03-30T13:13:05.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.299+0000 7fa04a7fc640 1 -- 172.21.15.66:0/4072107629 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03c031830 con 0x7fa04c0cd950 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.299+0000 7fa04a7fc640 1 -- 172.21.15.66:0/4072107629 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c031c00 con 0x7fa04c0cd950 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 -- 172.21.15.66:0/4072107629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0cd950 msgr2=0x7fa04c0cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 --2- 172.21.15.66:0/4072107629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0cd950 0x7fa04c0cfea0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0712b0 tx=0x7fa03c02b8d0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 -- 172.21.15.66:0/4072107629 shutdown_connections 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 --2- 172.21.15.66:0/4072107629 >> v2:172.21.15.117:3300/0 conn(0x7fa04c0d03e0 0x7fa04c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 --2- 172.21.15.66:0/4072107629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0cd950 0x7fa04c0cfea0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 -- 172.21.15.66:0/4072107629 >> 172.21.15.66:0/4072107629 conn(0x7fa04c06f060 msgr2=0x7fa04c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 -- 172.21.15.66:0/4072107629 shutdown_connections 2024-03-30T13:13:05.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.300+0000 7fa051cc5640 1 -- 172.21.15.66:0/4072107629 wait complete. 2024-03-30T13:13:05.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.301+0000 7fa051cc5640 1 Processor -- start 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 -- start start 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa04c0cd950 0x7fa04c1f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0d03e0 0x7fa04c0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa04c144a80 0x7fa04c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c072280 con 0x7fa04c0d03e0 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c0b3b40 con 0x7fa04c0cd950 2024-03-30T13:13:05.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa051cc5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c144e60 con 0x7fa04c144a80 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa04affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0d03e0 0x7fa04c0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa04affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0d03e0 0x7fa04c0b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59784/0 (socket says 172.21.15.66:59784) 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.302+0000 7fa04affd640 1 -- 172.21.15.66:0/3247397269 learned_addr learned my addr 172.21.15.66:0/3247397269 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04bfff640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa04c144a80 0x7fa04c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04b7fe640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa04c0cd950 0x7fa04c1f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04affd640 1 -- 172.21.15.66:0/3247397269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa04c144a80 msgr2=0x7fa04c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04affd640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa04c144a80 0x7fa04c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04affd640 1 -- 172.21.15.66:0/3247397269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa04c0cd950 msgr2=0x7fa04c1f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04affd640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa04c0cd950 0x7fa04c1f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04affd640 1 -- 172.21.15.66:0/3247397269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0b0360 con 0x7fa04c0d03e0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa04affd640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0d03e0 0x7fa04c0b1400 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009890 tx=0x7fa03c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c035040 con 0x7fa04c0d03e0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03c03b920 con 0x7fa04c0d03e0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c0b0640 con 0x7fa04c0d03e0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c03bd10 con 0x7fa04c0d03e0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.303+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c0b0c00 con 0x7fa04c0d03e0 2024-03-30T13:13:05.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.304+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa04c075900 con 0x7fa04c0d03e0 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.305+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa03c038020 con 0x7fa04c0d03e0 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.305+0000 7fa033fff640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa03806c530 0x7fa03806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fa03c0ad0f0 con 0x7fa04c0d03e0 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa04b7fe640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa03806c530 0x7fa03806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa033fff640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7fa0380729d0 0x7fa038074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 --> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa03c001d30 con 0x7fa0380729d0 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa03c0b4050 con 0x7fa04c0d03e0 2024-03-30T13:13:05.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa04bfff640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7fa0380729d0 0x7fa038074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa04b7fe640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa03806c530 0x7fa03806e9f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa040005880 tx=0x7fa040009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.306+0000 7fa04bfff640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7fa0380729d0 0x7fa038074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.307+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== osd.0 v2:172.21.15.66:6816/3495913091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa03c001d30 con 0x7fa0380729d0 2024-03-30T13:13:05.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.320+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 --> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa04c0b4ae0 con 0x7fa0380729d0 2024-03-30T13:13:05.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.320+0000 7fa033fff640 1 -- 172.21.15.66:0/3247397269 <== osd.0 v2:172.21.15.66:6816/3495913091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa04c0b4ae0 con 0x7fa0380729d0 2024-03-30T13:13:05.324 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:05.325 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:05.325 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.320+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7fa0380729d0 msgr2=0x7fa038074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.320+0000 7fa051cc5640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6816/3495913091,v1:172.21.15.66:6817/3495913091] conn(0x7fa0380729d0 0x7fa038074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa03806c530 msgr2=0x7fa03806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa03806c530 0x7fa03806e9f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa040005880 tx=0x7fa040009040 comp rx=0 tx=0).stop 2024-03-30T13:13:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0d03e0 msgr2=0x7fa04c0b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 --2- 172.21.15.66:0/3247397269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa04c0d03e0 0x7fa04c0b1400 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009890 tx=0x7fa03c0046c0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa04bfff640 1 -- 172.21.15.66:0/3247397269 reap_dead start 2024-03-30T13:13:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 shutdown_connections 2024-03-30T13:13:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 >> 172.21.15.66:0/3247397269 conn(0x7fa04c06f060 msgr2=0x7fa04c0cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 shutdown_connections 2024-03-30T13:13:05.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.321+0000 7fa051cc5640 1 -- 172.21.15.66:0/3247397269 wait complete. 2024-03-30T13:13:05.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-30T13:13:05.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.440+0000 7f8b5480d640 1 Processor -- start 2024-03-30T13:13:05.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.440+0000 7f8b5480d640 1 -- start start 2024-03-30T13:13:05.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b5480d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b5015ad70 0x7f8b50071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b5480d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 0x7f8b500cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b5480d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b50073700 con 0x7f8b5015a990 2024-03-30T13:13:05.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b5480d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f8b50072ad0 con 0x7f8b5015ad70 2024-03-30T13:13:05.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b5480d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8b50073000 con 0x7f8b500d0c90 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4e59c640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8b5015a990 0x7f8b500cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60212/0 (socket says 172.21.15.66:60212) 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4e59c640 1 -- 172.21.15.66:0/3349060260 learned_addr learned my addr 172.21.15.66:0/3349060260 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b47fff640 1 --2- 172.21.15.66:0/3349060260 >> v2:172.21.15.117:3300/0 conn(0x7f8b5015ad70 0x7f8b50071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4ed9d640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 0x7f8b500cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404937763 0 0) 0x7f8b50073700 con 0x7f8b5015a990 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b3c004630 con 0x7f8b5015a990 2024-03-30T13:13:05.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3928010511 0 0) 0x7f8b3c004630 con 0x7f8b5015a990 2024-03-30T13:13:05.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 msgr2=0x7f8b500cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 0x7f8b500cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 >> v2:172.21.15.117:3300/0 conn(0x7f8b5015ad70 msgr2=0x7f8b50071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 --2- 172.21.15.66:0/3349060260 >> v2:172.21.15.117:3300/0 conn(0x7f8b5015ad70 0x7f8b50071ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b50072d20 con 0x7f8b5015a990 2024-03-30T13:13:05.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f8b40001670 con 0x7f8b5015a990 2024-03-30T13:13:05.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b47fff640 1 --2- 172.21.15.66:0/3349060260 >> v2:172.21.15.117:3300/0 conn(0x7f8b5015ad70 0x7f8b50071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:13:05.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 >> v1:172.21.15.66:6789/0 conn(0x7f8b5015a990 legacy=0x7f8b500cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 0x7f8b3c004ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005040 0x7f8b3c005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3c005e50 0x7f8b3c0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50072ad0 con 0x7f8b3c005e50 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50073000 con 0x7f8b3c005040 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c00cbb0 con 0x7f8b500d0c90 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.441+0000 7f8b4ed9d640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3c005e50 0x7f8b3c0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b4e59c640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 0x7f8b3c004ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b47fff640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005040 0x7f8b3c005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b4ed9d640 1 -- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 msgr2=0x7f8b3c004ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b4ed9d640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500d0c90 0x7f8b3c004ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b4ed9d640 1 -- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005040 msgr2=0x7f8b3c005910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b4ed9d640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005040 0x7f8b3c005910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.442+0000 7f8b4ed9d640 1 -- 172.21.15.66:0/3349060260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c004630 con 0x7f8b3c005e50 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b4ed9d640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3c005e50 0x7f8b3c0062d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009fc0 tx=0x7f8b3c006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79ef07c4903b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48039070 con 0x7f8b3c005e50 2024-03-30T13:13:05.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4802cc20 con 0x7f8b3c005e50 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b4dd9b640 1 -- 172.21.15.66:0/3349060260 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48009c10 con 0x7f8b3c005e50 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b5480d640 1 -- 172.21.15.66:0/3349060260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3c005e50 msgr2=0x7f8b3c0062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3349060260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3c005e50 0x7f8b3c0062d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009fc0 tx=0x7f8b3c006940 comp rx=0 tx=0).stop 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b4ed9d640 1 -- 172.21.15.66:0/3349060260 reap_dead start 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b5480d640 1 -- 172.21.15.66:0/3349060260 shutdown_connections 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b5480d640 1 -- 172.21.15.66:0/3349060260 >> 172.21.15.66:0/3349060260 conn(0x7f8b5006d610 msgr2=0x7f8b5006da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b5480d640 1 -- 172.21.15.66:0/3349060260 shutdown_connections 2024-03-30T13:13:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.443+0000 7f8b5480d640 1 -- 172.21.15.66:0/3349060260 wait complete. 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.445+0000 7f8b5480d640 1 Processor -- start 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.445+0000 7f8b5480d640 1 -- start start 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b5480d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005e50 0x7f8b500c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b5480d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 0x7f8b500c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b5480d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500ccbb0 0x7f8b500cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b5480d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b500ce330 con 0x7f8b50155300 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b5480d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b500caa20 con 0x7f8b3c005e50 2024-03-30T13:13:05.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b5480d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8b500d0050 con 0x7f8b500ccbb0 2024-03-30T13:13:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 0x7f8b500c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 0x7f8b500c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59804/0 (socket says 172.21.15.66:59804) 2024-03-30T13:13:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 -- 172.21.15.66:0/3689799095 learned_addr learned my addr 172.21.15.66:0/3689799095 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b4ed9d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500ccbb0 0x7f8b500cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b4e59c640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005e50 0x7f8b500c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 -- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500ccbb0 msgr2=0x7f8b500cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500ccbb0 0x7f8b500cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 -- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005e50 msgr2=0x7f8b500c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005e50 0x7f8b500c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 -- 172.21.15.66:0/3689799095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b501471f0 con 0x7f8b50155300 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b4e59c640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005e50 0x7f8b500c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:05.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b47fff640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 0x7f8b500c7120 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8b38011cd0 tx=0x7f8b3800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3800fd20 con 0x7f8b50155300 2024-03-30T13:13:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.446+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b38004030 con 0x7f8b50155300 2024-03-30T13:13:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.447+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38004420 con 0x7f8b50155300 2024-03-30T13:13:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.447+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b50146c70 con 0x7f8b50155300 2024-03-30T13:13:05.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.447+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b50146f10 con 0x7f8b50155300 2024-03-30T13:13:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.447+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b50075900 con 0x7f8b50155300 2024-03-30T13:13:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.448+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8b38012040 con 0x7f8b50155300 2024-03-30T13:13:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.448+0000 7f8b46ffd640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8b3006c530 0x7f8b3006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.448+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f8b3808ba60 con 0x7f8b50155300 2024-03-30T13:13:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.448+0000 7f8b4e59c640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8b3006c530 0x7f8b3006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.449+0000 7f8b4e59c640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8b3006c530 0x7f8b3006e9f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b40007e90 tx=0x7f8b40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.453+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8b38019070 con 0x7f8b50155300 2024-03-30T13:13:05.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.639+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f8b500c33f0 con 0x7f8b3006c530 2024-03-30T13:13:05.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.640+0000 7f8b46ffd640 1 -- 172.21.15.66:0/3689799095 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8b500c33f0 con 0x7f8b3006c530 2024-03-30T13:13:05.644 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 0 to deep-scrub 2024-03-30T13:13:05.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8b3006c530 msgr2=0x7f8b3006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8b3006c530 0x7f8b3006e9f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b40007e90 tx=0x7f8b40009040 comp rx=0 tx=0).stop 2024-03-30T13:13:05.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 msgr2=0x7f8b500c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 0x7f8b500c7120 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8b38011cd0 tx=0x7f8b3800f610 comp rx=0 tx=0).stop 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 shutdown_connections 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8b3006c530 0x7f8b3006e9f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b500ccbb0 0x7f8b500cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b50155300 0x7f8b500c7120 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 --2- 172.21.15.66:0/3689799095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b3c005e50 0x7f8b500c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 >> 172.21.15.66:0/3689799095 conn(0x7f8b5006d610 msgr2=0x7f8b501577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 shutdown_connections 2024-03-30T13:13:05.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.642+0000 7f8b5480d640 1 -- 172.21.15.66:0/3689799095 wait complete. 2024-03-30T13:13:05.656 INFO:tasks.ceph:Scrubbing osd.1 2024-03-30T13:13:05.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:05.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.765+0000 7f74c2d9b640 1 Processor -- start 2024-03-30T13:13:05.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.765+0000 7f74c2d9b640 1 -- start start 2024-03-30T13:13:05.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74c2d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74bc0cef20 0x7f74bc0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74c2d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74c2d9b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc0cf3d0 con 0x7f74bc15e710 2024-03-30T13:13:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74c2d9b640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc06d5a0 con 0x7f74bc0cef20 2024-03-30T13:13:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74c2d9b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0cfc70 con 0x7f74bc0b66f0 2024-03-30T13:13:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74bbfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74bc0cef20 0x7f74bc0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74bbfff640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f74bc0cef20 0x7f74bc0b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59820/0 (socket says 172.21.15.66:59820) 2024-03-30T13:13:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74bbfff640 1 -- 172.21.15.66:0/350988485 learned_addr learned my addr 172.21.15.66:0/350988485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.766+0000 7f74c1338640 1 --2- 172.21.15.66:0/350988485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.767+0000 7f74bbfff640 1 -- 172.21.15.66:0/350988485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 msgr2=0x7f74bc0d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.767+0000 7f74bbfff640 1 --2- 172.21.15.66:0/350988485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.767+0000 7f74bbfff640 1 -- 172.21.15.66:0/350988485 >> v1:172.21.15.117:6789/0 conn(0x7f74bc15e710 legacy=0x7f74bc06ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.767+0000 7f74bbfff640 1 -- 172.21.15.66:0/350988485 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc0cf6b0 con 0x7f74bc0cef20 2024-03-30T13:13:05.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.767+0000 7f74bbfff640 1 --2- 172.21.15.66:0/350988485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc0b61b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009870 tx=0x7f74b002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfebd8af2a45d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74bb7fe640 1 -- 172.21.15.66:0/350988485 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b002f840 con 0x7f74bc0cef20 2024-03-30T13:13:05.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74bb7fe640 1 -- 172.21.15.66:0/350988485 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b002f9e0 con 0x7f74bc0cef20 2024-03-30T13:13:05.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74bb7fe640 1 -- 172.21.15.66:0/350988485 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b002fdb0 con 0x7f74bc0cef20 2024-03-30T13:13:05.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 -- 172.21.15.66:0/350988485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 msgr2=0x7f74bc0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 --2- 172.21.15.66:0/350988485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc0b61b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009870 tx=0x7f74b002ba20 comp rx=0 tx=0).stop 2024-03-30T13:13:05.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 -- 172.21.15.66:0/350988485 shutdown_connections 2024-03-30T13:13:05.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 --2- 172.21.15.66:0/350988485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 --2- 172.21.15.66:0/350988485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc0b61b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 -- 172.21.15.66:0/350988485 >> 172.21.15.66:0/350988485 conn(0x7f74bc0d1210 msgr2=0x7f74bc0d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 -- 172.21.15.66:0/350988485 shutdown_connections 2024-03-30T13:13:05.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.768+0000 7f74c2d9b640 1 -- 172.21.15.66:0/350988485 wait complete. 2024-03-30T13:13:05.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.769+0000 7f74c2d9b640 1 Processor -- start 2024-03-30T13:13:05.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.769+0000 7f74c2d9b640 1 -- start start 2024-03-30T13:13:05.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c2d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c2d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c2d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74bc15e710 0x7f74bc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c2d9b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc15eaf0 con 0x7f74bc0cef20 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c2d9b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0d2710 con 0x7f74bc15e710 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c2d9b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0d3170 con 0x7f74bc0b66f0 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74bbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc15b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59836/0 (socket says 172.21.15.66:59836) 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74bbfff640 1 -- 172.21.15.66:0/3782877603 learned_addr learned my addr 172.21.15.66:0/3782877603 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c0b37640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.770+0000 7f74c1338640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74bc15e710 0x7f74bc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74bbfff640 1 -- 172.21.15.66:0/3782877603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 msgr2=0x7f74bc0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74bbfff640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74bc0b66f0 0x7f74bc0ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74bbfff640 1 -- 172.21.15.66:0/3782877603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74bc15e710 msgr2=0x7f74bc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74bbfff640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74bc15e710 0x7f74bc15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74bbfff640 1 -- 172.21.15.66:0/3782877603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc0cc3a0 con 0x7f74bc0cef20 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74bbfff640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc15b160 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74b002ba60 tx=0x7f74b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0047070 con 0x7f74bc0cef20 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b0037bc0 con 0x7f74bc0cef20 2024-03-30T13:13:05.782 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc0cb5e0 con 0x7f74bc0cef20 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0042480 con 0x7f74bc0cef20 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.771+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc0cbba0 con 0x7f74bc0cef20 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.772+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f74bc075900 con 0x7f74bc0cef20 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.772+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f74b004a070 con 0x7f74bc0cef20 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74b97fa640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74a806c530 0x7f74a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f74b00ad740 con 0x7f74bc0cef20 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74c0b37640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74a806c530 0x7f74a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74b97fa640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f74a80729d0 0x7f74a8074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.783 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 --> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74a80754e0 con 0x7f74a80729d0 2024-03-30T13:13:05.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f74b00b4050 con 0x7f74bc0cef20 2024-03-30T13:13:05.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74c1338640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f74a80729d0 0x7f74a8074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.773+0000 7f74c0b37640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74a806c530 0x7f74a806e9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f74ac007fd0 tx=0x7f74ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.774+0000 7f74c1338640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f74a80729d0 0x7f74a8074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.774+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== osd.1 v2:172.21.15.66:6808/1885904429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74a80754e0 con 0x7f74a80729d0 2024-03-30T13:13:05.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.787+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 --> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f74bc0b9520 con 0x7f74a80729d0 2024-03-30T13:13:05.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74b97fa640 1 -- 172.21.15.66:0/3782877603 <== osd.1 v2:172.21.15.66:6808/1885904429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f74bc0b9520 con 0x7f74a80729d0 2024-03-30T13:13:05.792 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:05.792 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:05.792 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f74a80729d0 msgr2=0x7f74a8074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6808/1885904429,v1:172.21.15.66:6809/1885904429] conn(0x7f74a80729d0 0x7f74a8074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74a806c530 msgr2=0x7f74a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f74a806c530 0x7f74a806e9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f74ac007fd0 tx=0x7f74ac009040 comp rx=0 tx=0).stop 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 msgr2=0x7f74bc15b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 --2- 172.21.15.66:0/3782877603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc0cef20 0x7f74bc15b160 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74b002ba60 tx=0x7f74b0004730 comp rx=0 tx=0).stop 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c1338640 1 -- 172.21.15.66:0/3782877603 reap_dead start 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 shutdown_connections 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 >> 172.21.15.66:0/3782877603 conn(0x7f74bc0d1210 msgr2=0x7f74bc0b9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 shutdown_connections 2024-03-30T13:13:05.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.788+0000 7f74c2d9b640 1 -- 172.21.15.66:0/3782877603 wait complete. 2024-03-30T13:13:05.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-30T13:13:05.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.907+0000 7f1859315640 1 Processor -- start 2024-03-30T13:13:05.911 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.907+0000 7f1859315640 1 -- start start 2024-03-30T13:13:05.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1859315640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f185406c610 0x7f185406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1859315640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1854141d50 0x7f18541421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1859315640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1854163100 con 0x7f185415ee20 2024-03-30T13:13:05.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1859315640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f185415fbc0 con 0x7f1854141d50 2024-03-30T13:13:05.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1859315640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f185415f990 con 0x7f185406c610 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f18537fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1854141d50 0x7f18541421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f184bfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f185415ee20 0x7f185415f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60220/0 (socket says 172.21.15.66:60220) 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f184bfff640 1 -- 172.21.15.66:0/3454483414 learned_addr learned my addr 172.21.15.66:0/3454483414 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1852ffd640 1 --2- 172.21.15.66:0/3454483414 >> v2:172.21.15.117:3300/0 conn(0x7f185406c610 0x7f185406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f18537fe640 1 -- 172.21.15.66:0/3454483414 >> v2:172.21.15.117:3300/0 conn(0x7f185406c610 msgr2=0x7f185406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f18537fe640 1 --2- 172.21.15.66:0/3454483414 >> v2:172.21.15.117:3300/0 conn(0x7f185406c610 0x7f185406c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f18537fe640 1 -- 172.21.15.66:0/3454483414 >> v1:172.21.15.66:6789/0 conn(0x7f185415ee20 legacy=0x7f185415f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f18537fe640 1 -- 172.21.15.66:0/3454483414 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185406cf30 con 0x7f1854141d50 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f18527fc640 1 -- 172.21.15.66:0/3454483414 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2707567166 0 0) 0x7f1854163100 con 0x7f185415ee20 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.908+0000 7f1852ffd640 1 --2- 172.21.15.66:0/3454483414 >> v2:172.21.15.117:3300/0 conn(0x7f185406c610 0x7f185406c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.909+0000 7f18537fe640 1 --2- 172.21.15.66:0/3454483414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18541421c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f18540b4b40 tx=0x7f184c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6f40e1445cea0d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.909+0000 7f18527fc640 1 -- 172.21.15.66:0/3454483414 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c039070 con 0x7f1854141d50 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.909+0000 7f18527fc640 1 -- 172.21.15.66:0/3454483414 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184c02cc80 con 0x7f1854141d50 2024-03-30T13:13:05.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.909+0000 7f18527fc640 1 -- 172.21.15.66:0/3454483414 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c034660 con 0x7f1854141d50 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 -- 172.21.15.66:0/3454483414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 msgr2=0x7f18541421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 --2- 172.21.15.66:0/3454483414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18541421c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f18540b4b40 tx=0x7f184c02bed0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 -- 172.21.15.66:0/3454483414 shutdown_connections 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 --2- 172.21.15.66:0/3454483414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18541421c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 --2- 172.21.15.66:0/3454483414 >> v2:172.21.15.117:3300/0 conn(0x7f185406c610 0x7f185406c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 -- 172.21.15.66:0/3454483414 >> 172.21.15.66:0/3454483414 conn(0x7f18540b3cb0 msgr2=0x7f18540b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 -- 172.21.15.66:0/3454483414 shutdown_connections 2024-03-30T13:13:05.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.910+0000 7f1859315640 1 -- 172.21.15.66:0/3454483414 wait complete. 2024-03-30T13:13:05.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.911+0000 7f1859315640 1 Processor -- start 2024-03-30T13:13:05.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.911+0000 7f1859315640 1 -- start start 2024-03-30T13:13:05.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1859315640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185406c610 0x7f185415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1859315640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18540c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1859315640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185415ee20 0x7f1854160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1859315640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1854162900 con 0x7f1854141d50 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1859315640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1854162620 con 0x7f185415ee20 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1859315640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1854162be0 con 0x7f185406c610 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18540c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18540c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59856/0 (socket says 172.21.15.66:59856) 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 -- 172.21.15.66:0/1543586611 learned_addr learned my addr 172.21.15.66:0/1543586611 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:05.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f18537fe640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185415ee20 0x7f1854160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 -- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185406c610 msgr2=0x7f185415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1852ffd640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185406c610 0x7f185415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185406c610 0x7f185415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 -- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185415ee20 msgr2=0x7f1854160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185415ee20 0x7f1854160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 -- 172.21.15.66:0/1543586611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185415fd50 con 0x7f1854141d50 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f18537fe640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185415ee20 0x7f1854160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:05.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f1852ffd640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185406c610 0x7f185415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184bfff640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18540c82c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c500 tx=0x7f183c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.912+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c018070 con 0x7f1854141d50 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.913+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c007580 con 0x7f1854141d50 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.913+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185415d2b0 con 0x7f1854141d50 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.913+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c004a20 con 0x7f1854141d50 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.913+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541602f0 con 0x7f1854141d50 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.914+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1854075900 con 0x7f1854141d50 2024-03-30T13:13:05.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.914+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f183c01b070 con 0x7f1854141d50 2024-03-30T13:13:05.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.915+0000 7f184b7fe640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183406c530 0x7f183406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:05.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.915+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f183c08a7e0 con 0x7f1854141d50 2024-03-30T13:13:05.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.915+0000 7f1852ffd640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183406c530 0x7f183406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:05.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.915+0000 7f1852ffd640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183406c530 0x7f183406e9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1844006fd0 tx=0x7f1844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:05.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:05.919+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f183c059000 con 0x7f1854141d50 2024-03-30T13:13:06.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.103+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f1854070420 con 0x7f183406c530 2024-03-30T13:13:06.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.104+0000 7f184b7fe640 1 -- 172.21.15.66:0/1543586611 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1854070420 con 0x7f183406c530 2024-03-30T13:13:06.108 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 1 to deep-scrub 2024-03-30T13:13:06.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183406c530 msgr2=0x7f183406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183406c530 0x7f183406e9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1844006fd0 tx=0x7f1844008040 comp rx=0 tx=0).stop 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 msgr2=0x7f18540c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18540c82c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c500 tx=0x7f183c00c9c0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 shutdown_connections 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183406c530 0x7f183406e9f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185415ee20 0x7f1854160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1854141d50 0x7f18540c82c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 --2- 172.21.15.66:0/1543586611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185406c610 0x7f185415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 >> 172.21.15.66:0/1543586611 conn(0x7f18540b3cb0 msgr2=0x7f18540b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 shutdown_connections 2024-03-30T13:13:06.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.106+0000 7f1859315640 1 -- 172.21.15.66:0/1543586611 wait complete. 2024-03-30T13:13:06.121 INFO:tasks.ceph:Scrubbing osd.2 2024-03-30T13:13:06.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:06.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.228+0000 7fe8e880d640 1 Processor -- start 2024-03-30T13:13:06.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.228+0000 7fe8e880d640 1 -- start start 2024-03-30T13:13:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e880d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e880d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8e406c690 0x7fe8e4162870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e880d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e40cf030 con 0x7fe8e4162db0 2024-03-30T13:13:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e880d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e4162100 con 0x7fe8e406c690 2024-03-30T13:13:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e880d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40cf2d0 con 0x7fe8e415e4d0 2024-03-30T13:13:06.234 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e1d9b640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe8e406c690 0x7fe8e4162870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e2d9d640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fe8e4162db0 0x7fe8e40cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60230/0 (socket says 172.21.15.66:60230) 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.229+0000 7fe8e2d9d640 1 -- 172.21.15.66:0/3416773632 learned_addr learned my addr 172.21.15.66:0/3416773632 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.230+0000 7fe8e259c640 1 --2- 172.21.15.66:0/3416773632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.230+0000 7fe8e1d9b640 1 -- 172.21.15.66:0/3416773632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 msgr2=0x7fe8e40ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.230+0000 7fe8e1d9b640 1 --2- 172.21.15.66:0/3416773632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.230+0000 7fe8e1d9b640 1 -- 172.21.15.66:0/3416773632 >> v1:172.21.15.66:6789/0 conn(0x7fe8e4162db0 legacy=0x7fe8e40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.230+0000 7fe8e159a640 1 -- 172.21.15.66:0/3416773632 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1030603890 0 0) 0x7fe8e40cf030 con 0x7fe8e4162db0 2024-03-30T13:13:06.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.230+0000 7fe8e1d9b640 1 -- 172.21.15.66:0/3416773632 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40cf5b0 con 0x7fe8e406c690 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e1d9b640 1 --2- 172.21.15.66:0/3416773632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e4162870 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097a0 tx=0x7fe8d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72844e8acea417b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e159a640 1 -- 172.21.15.66:0/3416773632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80315a0 con 0x7fe8e406c690 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e159a640 1 -- 172.21.15.66:0/3416773632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8037070 con 0x7fe8e406c690 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e159a640 1 -- 172.21.15.66:0/3416773632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8031960 con 0x7fe8e406c690 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 -- 172.21.15.66:0/3416773632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 msgr2=0x7fe8e4162870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 --2- 172.21.15.66:0/3416773632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e4162870 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80097a0 tx=0x7fe8d802b840 comp rx=0 tx=0).stop 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 -- 172.21.15.66:0/3416773632 shutdown_connections 2024-03-30T13:13:06.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 --2- 172.21.15.66:0/3416773632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e4162870 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 --2- 172.21.15.66:0/3416773632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 -- 172.21.15.66:0/3416773632 >> 172.21.15.66:0/3416773632 conn(0x7fe8e4071c90 msgr2=0x7fe8e40672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 -- 172.21.15.66:0/3416773632 shutdown_connections 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.231+0000 7fe8e880d640 1 -- 172.21.15.66:0/3416773632 wait complete. 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 Processor -- start 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 -- start start 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e40b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e40c97b0 0x7fe8e4150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e415e8b0 con 0x7fe8e406c690 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4162350 con 0x7fe8e40c97b0 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.233+0000 7fe8e880d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40ccf00 con 0x7fe8e415e4d0 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e40b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e40b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59884/0 (socket says 172.21.15.66:59884) 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 -- 172.21.15.66:0/1982007358 learned_addr learned my addr 172.21.15.66:0/1982007358 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e2d9d640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e40c97b0 0x7fe8e4150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e1d9b640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 -- 172.21.15.66:0/1982007358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 msgr2=0x7fe8e40c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e415e4d0 0x7fe8e40c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 -- 172.21.15.66:0/1982007358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e40c97b0 msgr2=0x7fe8e4150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e40c97b0 0x7fe8e4150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 -- 172.21.15.66:0/1982007358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40c9d30 con 0x7fe8e406c690 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e259c640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e40b3910 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400d800 tx=0x7fe8d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400b6c0 con 0x7fe8e406c690 2024-03-30T13:13:06.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4019070 con 0x7fe8e406c690 2024-03-30T13:13:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e40ca010 con 0x7fe8e406c690 2024-03-30T13:13:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400bac0 con 0x7fe8e406c690 2024-03-30T13:13:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.234+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e40ca5d0 con 0x7fe8e406c690 2024-03-30T13:13:06.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.235+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8e4075900 con 0x7fe8e406c690 2024-03-30T13:13:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe8d400bc60 con 0x7fe8e406c690 2024-03-30T13:13:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8d2ffd640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe8c406c560 0x7fe8c406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fe8d408ba00 con 0x7fe8e406c690 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8e1d9b640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe8c406c560 0x7fe8c406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8d2ffd640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7fe8c4072a00 0x7fe8c4074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 --> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8c4075510 con 0x7fe8c4072a00 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe8d408de60 con 0x7fe8e406c690 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.236+0000 7fe8e2d9d640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7fe8c4072a00 0x7fe8c4074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.237+0000 7fe8e1d9b640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe8c406c560 0x7fe8c406ea20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8036000 tx=0x7fe8d803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.237+0000 7fe8e2d9d640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7fe8c4072a00 0x7fe8c4074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.245 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.237+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== osd.2 v2:172.21.15.66:6800/948131126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8c4075510 con 0x7fe8c4072a00 2024-03-30T13:13:06.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.250+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 --> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe8e40cc800 con 0x7fe8c4072a00 2024-03-30T13:13:06.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8d2ffd640 1 -- 172.21.15.66:0/1982007358 <== osd.2 v2:172.21.15.66:6800/948131126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe8e40cc800 con 0x7fe8c4072a00 2024-03-30T13:13:06.255 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:06.255 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:06.255 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:06.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7fe8c4072a00 msgr2=0x7fe8c4074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6800/948131126,v1:172.21.15.66:6801/948131126] conn(0x7fe8c4072a00 0x7fe8c4074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe8c406c560 msgr2=0x7fe8c406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fe8c406c560 0x7fe8c406ea20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8036000 tx=0x7fe8d803d040 comp rx=0 tx=0).stop 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 msgr2=0x7fe8e40b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 --2- 172.21.15.66:0/1982007358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8e406c690 0x7fe8e40b3910 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400d800 tx=0x7fe8d400dcc0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e2d9d640 1 -- 172.21.15.66:0/1982007358 reap_dead start 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 shutdown_connections 2024-03-30T13:13:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 >> 172.21.15.66:0/1982007358 conn(0x7fe8e4071c90 msgr2=0x7fe8e40b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 shutdown_connections 2024-03-30T13:13:06.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.251+0000 7fe8e880d640 1 -- 172.21.15.66:0/1982007358 wait complete. 2024-03-30T13:13:06.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-30T13:13:06.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.369+0000 7f4ae4835640 1 Processor -- start 2024-03-30T13:13:06.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.369+0000 7f4ae4835640 1 -- start start 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4ae4835640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae00cec60 0x7f4ae00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4ae4835640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ae00d3ee0 0x7f4ae00d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4ae4835640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae00d3120 con 0x7f4ae015e7a0 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4ae4835640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae00cfc40 con 0x7f4ae00d3ee0 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4ae4835640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae00d3400 con 0x7f4ae00cec60 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4addd9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae00cec60 0x7f4ae00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.370+0000 7f4aded9d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ae00d3ee0 0x7f4ae00d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4ade59c640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4ae015e7a0 0x7f4ae00cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:47848/0 (socket says 172.21.15.66:47848) 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4addd9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae00cec60 0x7f4ae00d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:47868/0 (socket says 172.21.15.66:47868) 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4ade59c640 1 -- 172.21.15.66:0/1208099876 learned_addr learned my addr 172.21.15.66:0/1208099876 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4aded9d640 1 -- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae00cec60 msgr2=0x7f4ae00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4aded9d640 1 --2- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae00cec60 0x7f4ae00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4aded9d640 1 -- 172.21.15.66:0/1208099876 >> v1:172.21.15.117:6789/0 conn(0x7f4ae015e7a0 legacy=0x7f4ae00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4add59a640 1 -- 172.21.15.66:0/1208099876 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013227189 0 0) 0x7f4ae00d3120 con 0x7f4ae015e7a0 2024-03-30T13:13:06.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.371+0000 7f4aded9d640 1 -- 172.21.15.66:0/1208099876 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae00d36e0 con 0x7f4ae00d3ee0 2024-03-30T13:13:06.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4aded9d640 1 --2- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00d17f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009d90 tx=0x7f4ad002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481d78daed5e50e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4add59a640 1 -- 172.21.15.66:0/1208099876 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0039070 con 0x7f4ae00d3ee0 2024-03-30T13:13:06.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4add59a640 1 -- 172.21.15.66:0/1208099876 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad002cbc0 con 0x7f4ae00d3ee0 2024-03-30T13:13:06.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4add59a640 1 -- 172.21.15.66:0/1208099876 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0034440 con 0x7f4ae00d3ee0 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4ae4835640 1 -- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 msgr2=0x7f4ae00d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00d17f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009d90 tx=0x7f4ad002be10 comp rx=0 tx=0).stop 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4ae4835640 1 -- 172.21.15.66:0/1208099876 shutdown_connections 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00d17f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.372+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1208099876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae00cec60 0x7f4ae00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.373+0000 7f4ae4835640 1 -- 172.21.15.66:0/1208099876 >> 172.21.15.66:0/1208099876 conn(0x7f4ae00676b0 msgr2=0x7f4ae0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.373+0000 7f4ae4835640 1 -- 172.21.15.66:0/1208099876 shutdown_connections 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.373+0000 7f4ae4835640 1 -- 172.21.15.66:0/1208099876 wait complete. 2024-03-30T13:13:06.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.374+0000 7f4ae4835640 1 Processor -- start 2024-03-30T13:13:06.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.374+0000 7f4ae4835640 1 -- start start 2024-03-30T13:13:06.382 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ae4835640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 0x7f4ae01507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ae4835640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ae4835640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae0159b20 0x7f4ae00ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ae4835640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae00b7720 con 0x7f4ae00cec60 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ae4835640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae00b75a0 con 0x7f4ae00d3ee0 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ae4835640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae00b78a0 con 0x7f4ae0159b20 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 0x7f4ae01507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 0x7f4ae01507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59898/0 (socket says 172.21.15.66:59898) 2024-03-30T13:13:06.383 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 -- 172.21.15.66:0/1154377791 learned_addr learned my addr 172.21.15.66:0/1154377791 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4aded9d640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae0159b20 0x7f4ae00ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4addd9b640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 -- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae0159b20 msgr2=0x7f4ae00ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae0159b20 0x7f4ae00ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 -- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 msgr2=0x7f4ae00cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 -- 172.21.15.66:0/1154377791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae00cdc90 con 0x7f4ae00cec60 2024-03-30T13:13:06.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4addd9b640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4aded9d640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae0159b20 0x7f4ae00ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ade59c640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 0x7f4ae01507f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c430 tx=0x7f4ad800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.375+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80074e0 con 0x7f4ae00cec60 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.376+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad80047e0 con 0x7f4ae00cec60 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.376+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae00cdf70 con 0x7f4ae00cec60 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.376+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8004bd0 con 0x7f4ae00cec60 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.376+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae00ce530 con 0x7f4ae00cec60 2024-03-30T13:13:06.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.376+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae0075900 con 0x7f4ae00cec60 2024-03-30T13:13:06.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.377+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4ad8011070 con 0x7f4ae00cec60 2024-03-30T13:13:06.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.377+0000 7f4ad6ffd640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4ac006c530 0x7f4ac006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.377+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f4ad808b330 con 0x7f4ae00cec60 2024-03-30T13:13:06.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.377+0000 7f4addd9b640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4ac006c530 0x7f4ac006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.378+0000 7f4addd9b640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4ac006c530 0x7f4ac006e9f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ca960 tx=0x7f4ac8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.382+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4ad800e280 con 0x7f4ae00cec60 2024-03-30T13:13:06.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.567+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f4ae00ca0f0 con 0x7f4ac006c530 2024-03-30T13:13:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.567+0000 7f4ad6ffd640 1 -- 172.21.15.66:0/1154377791 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4ae00ca0f0 con 0x7f4ac006c530 2024-03-30T13:13:06.572 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 2 to deep-scrub 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4ac006c530 msgr2=0x7f4ac006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4ac006c530 0x7f4ac006e9f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ca960 tx=0x7f4ac8008040 comp rx=0 tx=0).stop 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 msgr2=0x7f4ae01507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 0x7f4ae01507f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c430 tx=0x7f4ad800c8f0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 shutdown_connections 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f4ac006c530 0x7f4ac006e9f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ae0159b20 0x7f4ae00ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ae00d3ee0 0x7f4ae00cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 --2- 172.21.15.66:0/1154377791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ae00cec60 0x7f4ae01507f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 >> 172.21.15.66:0/1154377791 conn(0x7f4ae00676b0 msgr2=0x7f4ae00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 shutdown_connections 2024-03-30T13:13:06.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.569+0000 7f4ae4835640 1 -- 172.21.15.66:0/1154377791 wait complete. 2024-03-30T13:13:06.584 INFO:tasks.ceph:Scrubbing osd.3 2024-03-30T13:13:06.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.691+0000 7f80bfed9640 1 Processor -- start 2024-03-30T13:13:06.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.691+0000 7f80bfed9640 1 -- start start 2024-03-30T13:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bfed9640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80b80afdf0 0x7f80b8161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bfed9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 0x7f80b80b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bfed9640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b8162650 con 0x7f80b80b3d20 2024-03-30T13:13:06.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bfed9640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f80b8162ef0 con 0x7f80b80afdf0 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bfed9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f80b815f8c0 con 0x7f80b815f450 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80be476640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f80b80b3d20 0x7f80b8154af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60236/0 (socket says 172.21.15.66:60236) 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80be476640 1 -- 172.21.15.66:0/1792806288 learned_addr learned my addr 172.21.15.66:0/1792806288 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bdc75640 1 --2- 172.21.15.66:0/1792806288 >> v2:172.21.15.117:3300/0 conn(0x7f80b80afdf0 0x7f80b8161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410975343 0 0) 0x7f80b8162650 con 0x7f80b80b3d20 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.692+0000 7f80bd474640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 0x7f80b80b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a4003520 con 0x7f80b80b3d20 2024-03-30T13:13:06.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3172285017 0 0) 0x7f80a4003520 con 0x7f80b80b3d20 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 msgr2=0x7f80b80b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 0x7f80b80b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 >> v2:172.21.15.117:3300/0 conn(0x7f80b80afdf0 msgr2=0x7f80b8161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 --2- 172.21.15.66:0/1792806288 >> v2:172.21.15.117:3300/0 conn(0x7f80b80afdf0 0x7f80b8161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b8162930 con 0x7f80b80b3d20 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f80b00019f0 con 0x7f80b80b3d20 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 >> v1:172.21.15.66:6789/0 conn(0x7f80b80b3d20 legacy=0x7f80b8154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 0x7f80a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80b815f8c0 con 0x7f80a40039b0 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80a400b9c0 con 0x7f80a4004d40 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f80a400bb40 con 0x7f80b815f450 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bdc75640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bd474640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 0x7f80a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80be476640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bdc75640 1 -- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 msgr2=0x7f80a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bdc75640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b815f450 0x7f80a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bdc75640 1 -- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4004d40 msgr2=0x7f80a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bdc75640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.693+0000 7f80bdc75640 1 -- 172.21.15.66:0/1792806288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a4003520 con 0x7f80a40039b0 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.694+0000 7f80bdc75640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80a4003dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009af0 tx=0x7f80a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba317bf324da2ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.694+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4039070 con 0x7f80a40039b0 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.694+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b402c710 con 0x7f80a40039b0 2024-03-30T13:13:06.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.694+0000 7f80bcc73640 1 -- 172.21.15.66:0/1792806288 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4009690 con 0x7f80a40039b0 2024-03-30T13:13:06.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80bfed9640 1 -- 172.21.15.66:0/1792806288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 msgr2=0x7f80a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80bfed9640 1 --2- 172.21.15.66:0/1792806288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80a4003dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009af0 tx=0x7f80a4005830 comp rx=0 tx=0).stop 2024-03-30T13:13:06.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80be476640 1 -- 172.21.15.66:0/1792806288 reap_dead start 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80bfed9640 1 -- 172.21.15.66:0/1792806288 shutdown_connections 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80bfed9640 1 -- 172.21.15.66:0/1792806288 >> 172.21.15.66:0/1792806288 conn(0x7f80b80d0870 msgr2=0x7f80b80d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80bfed9640 1 -- 172.21.15.66:0/1792806288 shutdown_connections 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.695+0000 7f80bfed9640 1 -- 172.21.15.66:0/1792806288 wait complete. 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.696+0000 7f80bfed9640 1 Processor -- start 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.696+0000 7f80bfed9640 1 -- start start 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bfed9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80b81613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bfed9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81500d0 0x7f80b8150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bfed9640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b8150e40 0x7f80b80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bfed9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80b806b720 con 0x7f80a40039b0 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bfed9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80b80b4150 con 0x7f80b81500d0 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bfed9640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f80b80b01d0 con 0x7f80b8150e40 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80b81613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80b81613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59906/0 (socket says 172.21.15.66:59906) 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 -- 172.21.15.66:0/2751822628 learned_addr learned my addr 172.21.15.66:0/2751822628 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80be476640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b8150e40 0x7f80b80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bd474640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81500d0 0x7f80b8150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 -- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b8150e40 msgr2=0x7f80b80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b8150e40 0x7f80b80b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 -- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81500d0 msgr2=0x7f80b8150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81500d0 0x7f80b8150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 -- 172.21.15.66:0/2751822628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b815a0e0 con 0x7f80a40039b0 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80be476640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80b8150e40 0x7f80b80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:06.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bd474640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81500d0 0x7f80b8150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.697+0000 7f80bdc75640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80b81613c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f80b4000c00 tx=0x7f80b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.698+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4039050 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.698+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b4045070 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.698+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b815a380 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.698+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b40044e0 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.698+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b8160ea0 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.698+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80b8075900 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.699+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f80b4003a50 con 0x7f80a40039b0 2024-03-30T13:13:06.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80ae7fc640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f809c06c560 0x7f809c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f80b40ad970 con 0x7f80a40039b0 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80bd474640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f809c06c560 0x7f809c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80ae7fc640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7f809c072a00 0x7f809c074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 --> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f809c075510 con 0x7f809c072a00 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f80b40ade00 con 0x7f80a40039b0 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80be476640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7f809c072a00 0x7f809c074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80bd474640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f809c06c560 0x7f809c06ea20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f80b8154e60 tx=0x7f80a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.700+0000 7f80be476640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7f809c072a00 0x7f809c074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.701+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== osd.3 v2:172.21.15.66:6824/2124064569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f809c075510 con 0x7f809c072a00 2024-03-30T13:13:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.714+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 --> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f80b80b4580 con 0x7f809c072a00 2024-03-30T13:13:06.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.714+0000 7f80ae7fc640 1 -- 172.21.15.66:0/2751822628 <== osd.3 v2:172.21.15.66:6824/2124064569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f80b80b4580 con 0x7f809c072a00 2024-03-30T13:13:06.719 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:06.719 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:06.719 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7f809c072a00 msgr2=0x7f809c074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6824/2124064569,v1:172.21.15.66:6825/2124064569] conn(0x7f809c072a00 0x7f809c074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f809c06c560 msgr2=0x7f809c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f809c06c560 0x7f809c06ea20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f80b8154e60 tx=0x7f80a8008040 comp rx=0 tx=0).stop 2024-03-30T13:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 msgr2=0x7f80b81613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 --2- 172.21.15.66:0/2751822628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a40039b0 0x7f80b81613c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f80b4000c00 tx=0x7f80b4004730 comp rx=0 tx=0).stop 2024-03-30T13:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80be476640 1 -- 172.21.15.66:0/2751822628 reap_dead start 2024-03-30T13:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 shutdown_connections 2024-03-30T13:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 >> 172.21.15.66:0/2751822628 conn(0x7f80b80d0870 msgr2=0x7f80b815d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 shutdown_connections 2024-03-30T13:13:06.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.715+0000 7f80bfed9640 1 -- 172.21.15.66:0/2751822628 wait complete. 2024-03-30T13:13:06.729 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-30T13:13:06.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.833+0000 7fa6ca822640 1 Processor -- start 2024-03-30T13:13:06.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.833+0000 7fa6ca822640 1 -- start start 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6ca822640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6c4071440 0x7fa6c40cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6ca822640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c40cca60 0x7fa6c40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6ca822640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c4071fe0 con 0x7fa6c415fba0 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6ca822640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c4071a20 con 0x7fa6c4071440 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6ca822640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c4071d00 con 0x7fa6c40cca60 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6c8dbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c40cca60 0x7fa6c40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6c37fe640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6c4071440 0x7fa6c40cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6c8dbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c40cca60 0x7fa6c40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.66:47924/0 (socket says 172.21.15.66:47924) 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6c3fff640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa6c415fba0 0x7fa6c4160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.66:47856/0 (socket says 172.21.15.66:47856) 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.834+0000 7fa6c8dbf640 1 -- 172.21.15.66:0/1020831413 learned_addr learned my addr 172.21.15.66:0/1020831413 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.835+0000 7fa6c37fe640 1 -- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c40cca60 msgr2=0x7fa6c40ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.835+0000 7fa6c37fe640 1 --2- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c40cca60 0x7fa6c40ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.835+0000 7fa6c37fe640 1 -- 172.21.15.66:0/1020831413 >> v1:172.21.15.117:6789/0 conn(0x7fa6c415fba0 legacy=0x7fa6c4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.835+0000 7fa6c37fe640 1 -- 172.21.15.66:0/1020831413 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c40cabf0 con 0x7fa6c4071440 2024-03-30T13:13:06.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.835+0000 7fa6c2ffd640 1 -- 172.21.15.66:0/1020831413 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251395061 0 0) 0x7fa6c4071fe0 con 0x7fa6c415fba0 2024-03-30T13:13:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6c37fe640 1 --2- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c40cb3f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40097a0 tx=0x7fa6b402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7cae3f3134e9aab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6c2ffd640 1 -- 172.21.15.66:0/1020831413 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4035020 con 0x7fa6c4071440 2024-03-30T13:13:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6c2ffd640 1 -- 172.21.15.66:0/1020831413 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b4031830 con 0x7fa6c4071440 2024-03-30T13:13:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6c2ffd640 1 -- 172.21.15.66:0/1020831413 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4031c00 con 0x7fa6c4071440 2024-03-30T13:13:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 -- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 msgr2=0x7fa6c40cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 --2- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c40cb3f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40097a0 tx=0x7fa6b402b8d0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 -- 172.21.15.66:0/1020831413 shutdown_connections 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 --2- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c40cca60 0x7fa6c40ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 --2- 172.21.15.66:0/1020831413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c40cb3f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 -- 172.21.15.66:0/1020831413 >> 172.21.15.66:0/1020831413 conn(0x7fa6c406d610 msgr2=0x7fa6c406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 -- 172.21.15.66:0/1020831413 shutdown_connections 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.836+0000 7fa6ca822640 1 -- 172.21.15.66:0/1020831413 wait complete. 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.838+0000 7fa6ca822640 1 Processor -- start 2024-03-30T13:13:06.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.838+0000 7fa6ca822640 1 -- start start 2024-03-30T13:13:06.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.838+0000 7fa6ca822640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c41f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.838+0000 7fa6ca822640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c40cca60 0x7fa6c41e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.838+0000 7fa6ca822640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415fba0 0x7fa6c41f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6ca822640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c40b2f20 con 0x7fa6c40cca60 2024-03-30T13:13:06.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6ca822640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c40cb930 con 0x7fa6c4071440 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6ca822640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c40b30a0 con 0x7fa6c415fba0 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c41f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c8dbf640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415fba0 0x7fa6c41f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c41f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.66:48440/0 (socket says 172.21.15.66:48440) 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 -- 172.21.15.66:0/363503391 learned_addr learned my addr 172.21.15.66:0/363503391 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c37fe640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c40cca60 0x7fa6c41e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 -- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415fba0 msgr2=0x7fa6c41f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415fba0 0x7fa6c41f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 -- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c40cca60 msgr2=0x7fa6c41e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c40cca60 0x7fa6c41e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 -- 172.21.15.66:0/363503391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c41f8150 con 0x7fa6c4071440 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c37fe640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c40cca60 0x7fa6c41e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c3fff640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c41f7900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c460 tx=0x7fa6ac00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.839+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac018070 con 0x7fa6c4071440 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.840+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac007500 con 0x7fa6c4071440 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.840+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c41f8430 con 0x7fa6c4071440 2024-03-30T13:13:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.840+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004a20 con 0x7fa6c4071440 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.840+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c41f4310 con 0x7fa6c4071440 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.840+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c4075900 con 0x7fa6c4071440 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.841+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa6ac01b070 con 0x7fa6c4071440 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.841+0000 7fa6c0ff9640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa6a406c530 0x7fa6a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.842+0000 7fa6c37fe640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa6a406c530 0x7fa6a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.842+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fa6ac08a770 con 0x7fa6c4071440 2024-03-30T13:13:06.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.842+0000 7fa6c37fe640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa6a406c530 0x7fa6a406e9f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4036000 tx=0x7fa6b4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:06.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:06.847+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa6ac0588b0 con 0x7fa6c4071440 2024-03-30T13:13:07.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.031+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fa6c406f4a0 con 0x7fa6a406c530 2024-03-30T13:13:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.031+0000 7fa6c0ff9640 1 -- 172.21.15.66:0/363503391 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa6c406f4a0 con 0x7fa6a406c530 2024-03-30T13:13:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 3 to deep-scrub 2024-03-30T13:13:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa6a406c530 msgr2=0x7fa6a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa6a406c530 0x7fa6a406e9f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4036000 tx=0x7fa6b4008030 comp rx=0 tx=0).stop 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 msgr2=0x7fa6c41f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c41f7900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c460 tx=0x7fa6ac00c920 comp rx=0 tx=0).stop 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 shutdown_connections 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fa6a406c530 0x7fa6a406e9f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6c415fba0 0x7fa6c41f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c40cca60 0x7fa6c41e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 --2- 172.21.15.66:0/363503391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6c4071440 0x7fa6c41f7900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 >> 172.21.15.66:0/363503391 conn(0x7fa6c406d610 msgr2=0x7fa6c415ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 shutdown_connections 2024-03-30T13:13:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.033+0000 7fa6ca822640 1 -- 172.21.15.66:0/363503391 wait complete. 2024-03-30T13:13:07.048 INFO:tasks.ceph:Scrubbing osd.4 2024-03-30T13:13:07.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.155+0000 7f669a093640 1 Processor -- start 2024-03-30T13:13:07.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.155+0000 7f669a093640 1 -- start start 2024-03-30T13:13:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f669a093640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f669415e440 0x7f66940b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f669a093640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f669406d4a0 0x7f66940d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f669a093640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66940d2910 con 0x7f66940d3ef0 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f669a093640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f66940d3520 con 0x7f669415e440 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f669a093640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f66940d2bf0 con 0x7f669406d4a0 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f66937fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f669415e440 0x7f66940b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f6693fff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f66940d3ef0 0x7f66940cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60242/0 (socket says 172.21.15.66:60242) 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.156+0000 7f6693fff640 1 -- 172.21.15.66:0/2246164874 learned_addr learned my addr 172.21.15.66:0/2246164874 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.157+0000 7f6692ffd640 1 --2- 172.21.15.66:0/2246164874 >> v2:172.21.15.117:3300/0 conn(0x7f669406d4a0 0x7f66940d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.157+0000 7f66937fe640 1 -- 172.21.15.66:0/2246164874 >> v2:172.21.15.117:3300/0 conn(0x7f669406d4a0 msgr2=0x7f66940d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.157+0000 7f66937fe640 1 --2- 172.21.15.66:0/2246164874 >> v2:172.21.15.117:3300/0 conn(0x7f669406d4a0 0x7f66940d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.157+0000 7f66937fe640 1 -- 172.21.15.66:0/2246164874 >> v1:172.21.15.66:6789/0 conn(0x7f66940d3ef0 legacy=0x7f66940cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.157+0000 7f66937fe640 1 -- 172.21.15.66:0/2246164874 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66940d2ed0 con 0x7f669415e440 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.157+0000 7f66927fc640 1 -- 172.21.15.66:0/2246164874 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919589993 0 0) 0x7f66940d2910 con 0x7f66940d3ef0 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f66937fe640 1 --2- 172.21.15.66:0/2246164874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66940b9ce0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f66940d0d80 tx=0x7f668402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175d06e75b6ccf09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f66927fc640 1 -- 172.21.15.66:0/2246164874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684034070 con 0x7f669415e440 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f66927fc640 1 -- 172.21.15.66:0/2246164874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6684030610 con 0x7f669415e440 2024-03-30T13:13:07.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f66927fc640 1 -- 172.21.15.66:0/2246164874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66840309e0 con 0x7f669415e440 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 -- 172.21.15.66:0/2246164874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 msgr2=0x7f66940b9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 --2- 172.21.15.66:0/2246164874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66940b9ce0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f66940d0d80 tx=0x7f668402b960 comp rx=0 tx=0).stop 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 -- 172.21.15.66:0/2246164874 shutdown_connections 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 --2- 172.21.15.66:0/2246164874 >> v2:172.21.15.117:3300/0 conn(0x7f669406d4a0 0x7f66940d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 --2- 172.21.15.66:0/2246164874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66940b9ce0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 -- 172.21.15.66:0/2246164874 >> 172.21.15.66:0/2246164874 conn(0x7f66940676b0 msgr2=0x7f6694067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 -- 172.21.15.66:0/2246164874 shutdown_connections 2024-03-30T13:13:07.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.158+0000 7f669a093640 1 -- 172.21.15.66:0/2246164874 wait complete. 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.159+0000 7f669a093640 1 Processor -- start 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 -- start start 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f669406d4a0 0x7f6694161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66941f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66940cb440 0x7f66941541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f669415e820 con 0x7f669415e440 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66940ba220 con 0x7f66940cb440 2024-03-30T13:13:07.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f669a093640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f669415e9a0 con 0x7f669406d4a0 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f6692ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66941f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f6692ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66941f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59950/0 (socket says 172.21.15.66:59950) 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.160+0000 7f6692ffd640 1 -- 172.21.15.66:0/3792204859 learned_addr learned my addr 172.21.15.66:0/3792204859 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6693fff640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66940cb440 0x7f66941541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f66937fe640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f669406d4a0 0x7f6694161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6692ffd640 1 -- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f669406d4a0 msgr2=0x7f6694161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6692ffd640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f669406d4a0 0x7f6694161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6692ffd640 1 -- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66940cb440 msgr2=0x7f66941541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6692ffd640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66940cb440 0x7f66941541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6692ffd640 1 -- 172.21.15.66:0/3792204859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6694153130 con 0x7f669415e440 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6693fff640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66940cb440 0x7f66941541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f66937fe640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f669406d4a0 0x7f6694161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f6692ffd640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66941f3dc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f668800d830 tx=0x7f668800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800b6c0 con 0x7f669415e440 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6688019070 con 0x7f669415e440 2024-03-30T13:13:07.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6694153410 con 0x7f669415e440 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800ba80 con 0x7f669415e440 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.161+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941539d0 con 0x7f669415e440 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.162+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6694075900 con 0x7f669415e440 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.162+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f668800bc20 con 0x7f669415e440 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f667bfff640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f668006c560 0x7f668006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f668808ba10 con 0x7f669415e440 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f66937fe640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f668006c560 0x7f668006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f667bfff640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f6680072a00 0x7f6680074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 --> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6680075510 con 0x7f6680072a00 2024-03-30T13:13:07.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f668808e320 con 0x7f669415e440 2024-03-30T13:13:07.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f6693fff640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f6680072a00 0x7f6680074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.163+0000 7f66937fe640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f668006c560 0x7f668006ea20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6684036000 tx=0x7f6684008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.164+0000 7f6693fff640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f6680072a00 0x7f6680074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.164+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== osd.4 v2:172.21.15.117:6824/2408712778 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6680075510 con 0x7f6680072a00 2024-03-30T13:13:07.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.178+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 --> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6694153eb0 con 0x7f6680072a00 2024-03-30T13:13:07.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.178+0000 7f667bfff640 1 -- 172.21.15.66:0/3792204859 <== osd.4 v2:172.21.15.117:6824/2408712778 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6694153eb0 con 0x7f6680072a00 2024-03-30T13:13:07.182 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:07.183 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:07.183 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f6680072a00 msgr2=0x7f6680074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.117:6824/2408712778,v1:172.21.15.117:6825/2408712778] conn(0x7f6680072a00 0x7f6680074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f668006c560 msgr2=0x7f668006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f668006c560 0x7f668006ea20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6684036000 tx=0x7f6684008030 comp rx=0 tx=0).stop 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 msgr2=0x7f66941f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 --2- 172.21.15.66:0/3792204859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f669415e440 0x7f66941f3dc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f668800d830 tx=0x7f668800dcf0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f6693fff640 1 -- 172.21.15.66:0/3792204859 reap_dead start 2024-03-30T13:13:07.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 shutdown_connections 2024-03-30T13:13:07.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 >> 172.21.15.66:0/3792204859 conn(0x7f66940676b0 msgr2=0x7f66940b8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 shutdown_connections 2024-03-30T13:13:07.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.179+0000 7f669a093640 1 -- 172.21.15.66:0/3792204859 wait complete. 2024-03-30T13:13:07.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-30T13:13:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.297+0000 7fb003874640 1 Processor -- start 2024-03-30T13:13:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.297+0000 7fb003874640 1 -- start start 2024-03-30T13:13:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb003874640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faffc0d0560 0x7faffc162310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb003874640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d1f20 0x7faffc0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb003874640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faffc1628f0 con 0x7faffc15e770 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb003874640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7faffc0b5200 con 0x7faffc0d0560 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb003874640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7faffc162bd0 con 0x7faffc0d1f20 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faffc0d0560 0x7faffc162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faffc0d0560 0x7faffc162310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59966/0 (socket says 172.21.15.66:59966) 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 -- 172.21.15.66:0/2310321637 learned_addr learned my addr 172.21.15.66:0/2310321637 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb001e11640 1 --2- 172.21.15.66:0/2310321637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d1f20 0x7faffc0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 -- 172.21.15.66:0/2310321637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d1f20 msgr2=0x7faffc0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:07.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 --2- 172.21.15.66:0/2310321637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d1f20 0x7faffc0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 -- 172.21.15.66:0/2310321637 >> v1:172.21.15.117:6789/0 conn(0x7faffc15e770 legacy=0x7faffc0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.298+0000 7fb000e0f640 1 -- 172.21.15.66:0/2310321637 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc162eb0 con 0x7faffc0d0560 2024-03-30T13:13:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.299+0000 7fb000e0f640 1 --2- 172.21.15.66:0/2310321637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc0d0560 0x7faffc162310 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fafec009850 tx=0x7fafec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a733e7f1e562fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.299+0000 7faff3fff640 1 -- 172.21.15.66:0/2310321637 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec004230 con 0x7faffc0d0560 2024-03-30T13:13:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.299+0000 7faff3fff640 1 -- 172.21.15.66:0/2310321637 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0043d0 con 0x7faffc0d0560 2024-03-30T13:13:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.299+0000 7faff3fff640 1 -- 172.21.15.66:0/2310321637 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec003810 con 0x7faffc0d0560 2024-03-30T13:13:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 -- 172.21.15.66:0/2310321637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc0d0560 msgr2=0x7faffc162310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 --2- 172.21.15.66:0/2310321637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc0d0560 0x7faffc162310 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fafec009850 tx=0x7fafec02b960 comp rx=0 tx=0).stop 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 -- 172.21.15.66:0/2310321637 shutdown_connections 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 --2- 172.21.15.66:0/2310321637 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d1f20 0x7faffc0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 --2- 172.21.15.66:0/2310321637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc0d0560 0x7faffc162310 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 -- 172.21.15.66:0/2310321637 >> 172.21.15.66:0/2310321637 conn(0x7faffc06f8f0 msgr2=0x7faffc06fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 -- 172.21.15.66:0/2310321637 shutdown_connections 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.300+0000 7fb003874640 1 -- 172.21.15.66:0/2310321637 wait complete. 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.301+0000 7fb003874640 1 Processor -- start 2024-03-30T13:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.301+0000 7fb003874640 1 -- start start 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb003874640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d0560 0x7faffc1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb003874640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc0d1f20 0x7faffc1e9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb003874640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 0x7faffc1e9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb003874640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faffc06c790 con 0x7faffc15e770 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb003874640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faffc15eb50 con 0x7faffc0d1f20 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb003874640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7faffc06c910 con 0x7faffc0d0560 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 0x7faffc1e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 0x7faffc1e9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59980/0 (socket says 172.21.15.66:59980) 2024-03-30T13:13:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 -- 172.21.15.66:0/2885119325 learned_addr learned my addr 172.21.15.66:0/2885119325 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb000e0f640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc0d1f20 0x7faffc1e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001610640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d0560 0x7faffc1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 -- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d0560 msgr2=0x7faffc1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d0560 0x7faffc1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 -- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc0d1f20 msgr2=0x7faffc1e9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc0d1f20 0x7faffc1e9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 -- 172.21.15.66:0/2885119325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc0c8860 con 0x7faffc15e770 2024-03-30T13:13:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001610640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d0560 0x7faffc1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.302+0000 7fb001e11640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 0x7faffc1e9680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faff4007e70 tx=0x7faff400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.303+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4014420 con 0x7faffc15e770 2024-03-30T13:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.303+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff4019070 con 0x7faffc15e770 2024-03-30T13:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.303+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc0c8b40 con 0x7faffc15e770 2024-03-30T13:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.303+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4014820 con 0x7faffc15e770 2024-03-30T13:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.303+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc157630 con 0x7faffc15e770 2024-03-30T13:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.304+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc06ded0 con 0x7faffc15e770 2024-03-30T13:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.304+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faff40149c0 con 0x7faffc15e770 2024-03-30T13:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.305+0000 7faff1ffb640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafe006c530 0x7fafe006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.305+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7faff408bf60 con 0x7faffc15e770 2024-03-30T13:13:07.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.305+0000 7fb001610640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafe006c530 0x7fafe006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.305+0000 7fb001610640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafe006c530 0x7fafe006e9f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faff80015e0 tx=0x7faff8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.309+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faffc06ded0 con 0x7faffc15e770 2024-03-30T13:13:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.493+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7faffc0c8d10 con 0x7fafe006c530 2024-03-30T13:13:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.493+0000 7faff1ffb640 1 -- 172.21.15.66:0/2885119325 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faffc0c8d10 con 0x7fafe006c530 2024-03-30T13:13:07.497 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 4 to deep-scrub 2024-03-30T13:13:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafe006c530 msgr2=0x7fafe006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafe006c530 0x7fafe006e9f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faff80015e0 tx=0x7faff8009040 comp rx=0 tx=0).stop 2024-03-30T13:13:07.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 msgr2=0x7faffc1e9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 0x7faffc1e9680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faff4007e70 tx=0x7faff400e4b0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 shutdown_connections 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fafe006c530 0x7fafe006e9f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faffc15e770 0x7faffc1e9680 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faffc0d1f20 0x7faffc1e9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 --2- 172.21.15.66:0/2885119325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faffc0d0560 0x7faffc1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 >> 172.21.15.66:0/2885119325 conn(0x7faffc06f8f0 msgr2=0x7faffc15f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 shutdown_connections 2024-03-30T13:13:07.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.495+0000 7fb003874640 1 -- 172.21.15.66:0/2885119325 wait complete. 2024-03-30T13:13:07.509 INFO:tasks.ceph:Scrubbing osd.5 2024-03-30T13:13:07.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:07.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.618+0000 7fc642022640 1 Processor -- start 2024-03-30T13:13:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.618+0000 7fc642022640 1 -- start start 2024-03-30T13:13:07.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc642022640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc63c15e710 0x7fc63c06ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc642022640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc642022640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63c0cf3d0 con 0x7fc63c0b66f0 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc642022640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc63c0cfc70 con 0x7fc63c15e710 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc642022640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c0cf6b0 con 0x7fc63c0cef20 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc63b7fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc63c15e710 0x7fc63c06ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc63bfff640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fc63c0b66f0 0x7fc63c1622c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60246/0 (socket says 172.21.15.66:60246) 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc63bfff640 1 -- 172.21.15.66:0/1140253958 learned_addr learned my addr 172.21.15.66:0/1140253958 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc63affd640 1 --2- 172.21.15.66:0/1140253958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc63b7fe640 1 -- 172.21.15.66:0/1140253958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 msgr2=0x7fc63c0b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.619+0000 7fc63b7fe640 1 --2- 172.21.15.66:0/1140253958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c0b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63b7fe640 1 -- 172.21.15.66:0/1140253958 >> v1:172.21.15.66:6789/0 conn(0x7fc63c0b66f0 legacy=0x7fc63c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63a7fc640 1 -- 172.21.15.66:0/1140253958 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4256289110 0 0) 0x7fc63c0cf3d0 con 0x7fc63c0b66f0 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63b7fe640 1 -- 172.21.15.66:0/1140253958 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c0cf990 con 0x7fc63c15e710 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63b7fe640 1 --2- 172.21.15.66:0/1140253958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c06ce90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0097a0 tx=0x7fc62c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a68a65d2f8946e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63a7fc640 1 -- 172.21.15.66:0/1140253958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c034070 con 0x7fc63c15e710 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63a7fc640 1 -- 172.21.15.66:0/1140253958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62c030610 con 0x7fc63c15e710 2024-03-30T13:13:07.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.620+0000 7fc63a7fc640 1 -- 172.21.15.66:0/1140253958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0309e0 con 0x7fc63c15e710 2024-03-30T13:13:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 -- 172.21.15.66:0/1140253958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 msgr2=0x7fc63c06ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 --2- 172.21.15.66:0/1140253958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c06ce90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0097a0 tx=0x7fc62c02b960 comp rx=0 tx=0).stop 2024-03-30T13:13:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 -- 172.21.15.66:0/1140253958 shutdown_connections 2024-03-30T13:13:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 --2- 172.21.15.66:0/1140253958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c0b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 --2- 172.21.15.66:0/1140253958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c06ce90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 -- 172.21.15.66:0/1140253958 >> 172.21.15.66:0/1140253958 conn(0x7fc63c0d1210 msgr2=0x7fc63c0d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 -- 172.21.15.66:0/1140253958 shutdown_connections 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.621+0000 7fc642022640 1 -- 172.21.15.66:0/1140253958 wait complete. 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.622+0000 7fc642022640 1 Processor -- start 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.622+0000 7fc642022640 1 -- start start 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc642022640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc63c0b66f0 0x7fc63c1f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc642022640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c1f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc642022640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc642022640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c06d3d0 con 0x7fc63c15e710 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc642022640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c15eaf0 con 0x7fc63c0b66f0 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc642022640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c06d550 con 0x7fc63c0cef20 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc63bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c15ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59994/0 (socket says 172.21.15.66:59994) 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.623+0000 7fc63bfff640 1 -- 172.21.15.66:0/1161198697 learned_addr learned my addr 172.21.15.66:0/1161198697 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63b7fe640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc63c0b66f0 0x7fc63c1f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63affd640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c1f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63bfff640 1 -- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 msgr2=0x7fc63c1f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63bfff640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc63c0cef20 0x7fc63c1f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63bfff640 1 -- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc63c0b66f0 msgr2=0x7fc63c1f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63bfff640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc63c0b66f0 0x7fc63c1f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63bfff640 1 -- 172.21.15.66:0/1161198697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c1f51c0 con 0x7fc63c15e710 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc63bfff640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c15ed00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c730 tx=0x7fc63400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634018070 con 0x7fc63c15e710 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63400ce40 con 0x7fc63c15e710 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c1f4ee0 con 0x7fc63c15e710 2024-03-30T13:13:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63400a880 con 0x7fc63c15e710 2024-03-30T13:13:07.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.624+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c0cc880 con 0x7fc63c15e710 2024-03-30T13:13:07.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.625+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc63c1f75b0 con 0x7fc63c15e710 2024-03-30T13:13:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc63401b070 con 0x7fc63c15e710 2024-03-30T13:13:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc623fff640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc62806c530 0x7fc62806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc63b7fe640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc62806c530 0x7fc62806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fc63408b090 con 0x7fc63c15e710 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc623fff640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7fc6280729d0 0x7fc628074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 --> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6280754e0 con 0x7fc6280729d0 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.626+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc63408b520 con 0x7fc63c15e710 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.627+0000 7fc63b7fe640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc62806c530 0x7fc62806e9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc62c036000 tx=0x7fc62c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.627+0000 7fc63affd640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7fc6280729d0 0x7fc628074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.627+0000 7fc63affd640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7fc6280729d0 0x7fc628074e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.628+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== osd.5 v2:172.21.15.117:6802/1603026034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6280754e0 con 0x7fc6280729d0 2024-03-30T13:13:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.641+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 --> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc63c0d06d0 con 0x7fc6280729d0 2024-03-30T13:13:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.641+0000 7fc623fff640 1 -- 172.21.15.66:0/1161198697 <== osd.5 v2:172.21.15.117:6802/1603026034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc63c0d06d0 con 0x7fc6280729d0 2024-03-30T13:13:07.646 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:07.646 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:07.646 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7fc6280729d0 msgr2=0x7fc628074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.117:6802/1603026034,v1:172.21.15.117:6804/1603026034] conn(0x7fc6280729d0 0x7fc628074e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc62806c530 msgr2=0x7fc62806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fc62806c530 0x7fc62806e9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc62c036000 tx=0x7fc62c008030 comp rx=0 tx=0).stop 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 msgr2=0x7fc63c15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 --2- 172.21.15.66:0/1161198697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc63c15e710 0x7fc63c15ed00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c730 tx=0x7fc63400cbf0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc63bfff640 1 -- 172.21.15.66:0/1161198697 reap_dead start 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 shutdown_connections 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 >> 172.21.15.66:0/1161198697 conn(0x7fc63c0d1210 msgr2=0x7fc63c0b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 shutdown_connections 2024-03-30T13:13:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.642+0000 7fc642022640 1 -- 172.21.15.66:0/1161198697 wait complete. 2024-03-30T13:13:07.656 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-30T13:13:07.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.760+0000 7f3943340640 1 Processor -- start 2024-03-30T13:13:07.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.760+0000 7f3943340640 1 -- start start 2024-03-30T13:13:07.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f3943340640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f393c15c030 0x7f393c162020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f3943340640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c06de00 0x7f393c06e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f3943340640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c15c410 con 0x7f393c0b3bf0 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f3943340640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f393c0d1ee0 con 0x7f393c15c030 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f3943340640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f393c0711a0 con 0x7f393c06de00 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f39418dd640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f393c0b3bf0 0x7f393c15ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60260/0 (socket says 172.21.15.66:60260) 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f39418dd640 1 -- 172.21.15.66:0/2036047436 learned_addr learned my addr 172.21.15.66:0/2036047436 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f39408db640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c06de00 0x7f393c06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.761+0000 7f39410dc640 1 --2- 172.21.15.66:0/2036047436 >> v2:172.21.15.117:3300/0 conn(0x7f393c15c030 0x7f393c162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f3933fff640 1 -- 172.21.15.66:0/2036047436 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321537887 0 0) 0x7f393c15c410 con 0x7f393c0b3bf0 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f3933fff640 1 -- 172.21.15.66:0/2036047436 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3928003520 con 0x7f393c0b3bf0 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f39410dc640 1 -- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c06de00 msgr2=0x7f393c06e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f39410dc640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c06de00 0x7f393c06e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f39410dc640 1 -- 172.21.15.66:0/2036047436 >> v1:172.21.15.66:6789/0 conn(0x7f393c0b3bf0 legacy=0x7f393c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f39410dc640 1 -- 172.21.15.66:0/2036047436 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c071480 con 0x7f393c15c030 2024-03-30T13:13:07.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.762+0000 7f39408db640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c06de00 0x7f393c06e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:13:07.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f39410dc640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c15c030 0x7f393c162020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f39380097a0 tx=0x7f393802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447c0cdd46721bba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3933fff640 1 -- 172.21.15.66:0/2036047436 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938038070 con 0x7f393c15c030 2024-03-30T13:13:07.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3933fff640 1 -- 172.21.15.66:0/2036047436 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3938031700 con 0x7f393c15c030 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3933fff640 1 -- 172.21.15.66:0/2036047436 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938031ab0 con 0x7f393c15c030 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3943340640 1 -- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c15c030 msgr2=0x7f393c162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3943340640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c15c030 0x7f393c162020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f39380097a0 tx=0x7f393802b7a0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3943340640 1 -- 172.21.15.66:0/2036047436 shutdown_connections 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3943340640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c06de00 0x7f393c06e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3943340640 1 --2- 172.21.15.66:0/2036047436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c15c030 0x7f393c162020 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.763+0000 7f3943340640 1 -- 172.21.15.66:0/2036047436 >> 172.21.15.66:0/2036047436 conn(0x7f393c0cebf0 msgr2=0x7f393c0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.764+0000 7f3943340640 1 -- 172.21.15.66:0/2036047436 shutdown_connections 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.764+0000 7f3943340640 1 -- 172.21.15.66:0/2036047436 wait complete. 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 Processor -- start 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 -- start start 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 0x7f393c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c0b3bf0 0x7f393c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c15c030 0x7f393c1e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f393c0b4080 con 0x7f393c06de00 2024-03-30T13:13:07.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f393c071a40 con 0x7f393c0b3bf0 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.765+0000 7f3943340640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f393c0b4200 con 0x7f393c15c030 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 0x7f393c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 0x7f393c0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60000/0 (socket says 172.21.15.66:60000) 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 -- 172.21.15.66:0/2898070762 learned_addr learned my addr 172.21.15.66:0/2898070762 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39408db640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c0b3bf0 0x7f393c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39418dd640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c15c030 0x7f393c1e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 -- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c15c030 msgr2=0x7f393c1e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c15c030 0x7f393c1e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 -- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c0b3bf0 msgr2=0x7f393c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c0b3bf0 0x7f393c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 -- 172.21.15.66:0/2898070762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c0ca2e0 con 0x7f393c06de00 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39408db640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c0b3bf0 0x7f393c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39418dd640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c15c030 0x7f393c1e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f39410dc640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 0x7f393c0c9c50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f393802b7e0 tx=0x7f39380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938042070 con 0x7f393c06de00 2024-03-30T13:13:07.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393803d870 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938038040 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c0ca5c0 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.766+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c0cab80 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.767+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393c0b4df0 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.768+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3938045070 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.768+0000 7f3931ffb640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f392006c530 0x7f392006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.768+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f39380ad030 con 0x7f393c06de00 2024-03-30T13:13:07.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.768+0000 7f39408db640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f392006c530 0x7f392006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:07.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.769+0000 7f39408db640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f392006c530 0x7f392006e9f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f393c1e9110 tx=0x7f392c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:07.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.773+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3938030070 con 0x7f393c06de00 2024-03-30T13:13:07.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.956+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f393c075900 con 0x7f392006c530 2024-03-30T13:13:07.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.957+0000 7f3931ffb640 1 -- 172.21.15.66:0/2898070762 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f393c075900 con 0x7f392006c530 2024-03-30T13:13:07.961 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 5 to deep-scrub 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.958+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f392006c530 msgr2=0x7f392006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.958+0000 7f3943340640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f392006c530 0x7f392006e9f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f393c1e9110 tx=0x7f392c008040 comp rx=0 tx=0).stop 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.958+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 msgr2=0x7f393c0c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.958+0000 7f3943340640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 0x7f393c0c9c50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f393802b7e0 tx=0x7f39380046c0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 shutdown_connections 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f392006c530 0x7f392006e9f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f393c15c030 0x7f393c1e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f393c0b3bf0 0x7f393c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 --2- 172.21.15.66:0/2898070762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f393c06de00 0x7f393c0c9c50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 >> 172.21.15.66:0/2898070762 conn(0x7f393c0cebf0 msgr2=0x7f393c0cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 shutdown_connections 2024-03-30T13:13:07.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:07.959+0000 7f3943340640 1 -- 172.21.15.66:0/2898070762 wait complete. 2024-03-30T13:13:07.973 INFO:tasks.ceph:Scrubbing osd.6 2024-03-30T13:13:07.973 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.080+0000 7f564b8c8640 1 Processor -- start 2024-03-30T13:13:08.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.080+0000 7f564b8c8640 1 -- start start 2024-03-30T13:13:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.081+0000 7f564b8c8640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5644158690 0x7f564406d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.081+0000 7f564b8c8640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564415f1c0 con 0x7f564415ec10 2024-03-30T13:13:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.081+0000 7f564b8c8640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5644072ac0 con 0x7f56440735a0 2024-03-30T13:13:08.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.081+0000 7f564b8c8640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f5644072da0 con 0x7f5644158690 2024-03-30T13:13:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.081+0000 7f5649e65640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f564415ec10 0x7f56440b35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60266/0 (socket says 172.21.15.66:60266) 2024-03-30T13:13:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.081+0000 7f5649e65640 1 -- 172.21.15.66:0/1720753980 learned_addr learned my addr 172.21.15.66:0/1720753980 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3700278576 0 0) 0x7f564415f1c0 con 0x7f564415ec10 2024-03-30T13:13:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f562c003520 con 0x7f564415ec10 2024-03-30T13:13:08.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 --2- 172.21.15.66:0/1720753980 >> v2:172.21.15.117:3300/0 conn(0x7f5644158690 0x7f564406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 96799502 0 0) 0x7f562c003520 con 0x7f564415ec10 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 >> v2:172.21.15.117:3300/0 conn(0x7f5644158690 msgr2=0x7f564406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 --2- 172.21.15.66:0/1720753980 >> v2:172.21.15.117:3300/0 conn(0x7f5644158690 0x7f564406d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 >> v1:172.21.15.117:6789/0 conn(0x7f56440735a0 legacy=0x7f564415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5644073080 con 0x7f564415ec10 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7f563c001670 con 0x7f564415ec10 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 >> v1:172.21.15.66:6789/0 conn(0x7f564415ec10 legacy=0x7f56440b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f562c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c004800 0x7f562c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c0051c0 0x7f562c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5644072da0 con 0x7f562c0039b0 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f562c00c160 con 0x7f562c004800 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f562c00c360 con 0x7f562c0051c0 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f562c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649e65640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c0051c0 0x7f562c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5648e63640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c004800 0x7f562c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 -- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c0051c0 msgr2=0x7f562c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c0051c0 0x7f562c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 -- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c004800 msgr2=0x7f562c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c004800 0x7f562c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649664640 1 -- 172.21.15.66:0/1720753980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c003520 con 0x7f562c0039b0 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.082+0000 7f5649e65640 1 -- 172.21.15.66:0/1720753980 reap_dead start 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.083+0000 7f5649664640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f562c003dd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5634010290 tx=0x7f562c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5041f6baf4545bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.083+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563403a070 con 0x7f562c0039b0 2024-03-30T13:13:08.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.083+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563402ddf0 con 0x7f562c0039b0 2024-03-30T13:13:08.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.083+0000 7f5643fff640 1 -- 172.21.15.66:0/1720753980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634009d60 con 0x7f562c0039b0 2024-03-30T13:13:08.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 -- 172.21.15.66:0/1720753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 msgr2=0x7f562c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f562c003dd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5634010290 tx=0x7f562c005830 comp rx=0 tx=0).stop 2024-03-30T13:13:08.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 -- 172.21.15.66:0/1720753980 shutdown_connections 2024-03-30T13:13:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 --2- 172.21.15.66:0/1720753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f562c003dd0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 -- 172.21.15.66:0/1720753980 >> 172.21.15.66:0/1720753980 conn(0x7f564406f060 msgr2=0x7f564406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 -- 172.21.15.66:0/1720753980 shutdown_connections 2024-03-30T13:13:08.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.084+0000 7f564b8c8640 1 -- 172.21.15.66:0/1720753980 wait complete. 2024-03-30T13:13:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.085+0000 7f564b8c8640 1 Processor -- start 2024-03-30T13:13:08.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.085+0000 7f564b8c8640 1 -- start start 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f564b8c8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f56441f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f564b8c8640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f564415d000 0x7f564415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f564b8c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56441f2280 0x7f56441f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f564b8c8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f564415e890 con 0x7f562c0039b0 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f564b8c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f564415e710 con 0x7f56441f2280 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f564b8c8640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f564415ea10 con 0x7f564415d000 2024-03-30T13:13:08.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f56441f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f56441f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60010/0 (socket says 172.21.15.66:60010) 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 -- 172.21.15.66:0/3560158202 learned_addr learned my addr 172.21.15.66:0/3560158202 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5648e63640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f564415d000 0x7f564415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649e65640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56441f2280 0x7f56441f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 -- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f564415d000 msgr2=0x7f564415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f564415d000 0x7f564415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 -- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56441f2280 msgr2=0x7f56441f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56441f2280 0x7f56441f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.086+0000 7f5649664640 1 -- 172.21.15.66:0/3560158202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56441f5810 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.087+0000 7f5649664640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f56441f1910 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5634010290 tx=0x7f5634009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.087+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634044020 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.087+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5634040420 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.087+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56441f5af0 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.087+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563403a040 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.087+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56441f60b0 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.088+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5644075900 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.088+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5634005d10 con 0x7f562c0039b0 2024-03-30T13:13:08.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5641ffb640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f563006c560 0x7f563006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f56340ae600 con 0x7f562c0039b0 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5648e63640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f563006c560 0x7f563006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5641ffb640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5630072a00 0x7f5630074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 --> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5634040e50 con 0x7f5630072a00 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f56340aea90 con 0x7f562c0039b0 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5649e65640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5630072a00 0x7f5630074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.089+0000 7f5648e63640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f563006c560 0x7f563006ea20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5638000c00 tx=0x7f5638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.090+0000 7f5649e65640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5630072a00 0x7f5630074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.091+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== osd.6 v2:172.21.15.117:6800/698765583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5634040e50 con 0x7f5630072a00 2024-03-30T13:13:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.104+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 --> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f564406e060 con 0x7f5630072a00 2024-03-30T13:13:08.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f5641ffb640 1 -- 172.21.15.66:0/3560158202 <== osd.6 v2:172.21.15.117:6800/698765583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f564406e060 con 0x7f5630072a00 2024-03-30T13:13:08.109 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:08.109 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:08.109 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:08.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5630072a00 msgr2=0x7f5630074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.117:6800/698765583,v1:172.21.15.117:6801/698765583] conn(0x7f5630072a00 0x7f5630074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f563006c560 msgr2=0x7f563006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f563006c560 0x7f563006ea20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5638000c00 tx=0x7f5638009040 comp rx=0 tx=0).stop 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 msgr2=0x7f56441f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 --2- 172.21.15.66:0/3560158202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f562c0039b0 0x7f56441f1910 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5634010290 tx=0x7f5634009750 comp rx=0 tx=0).stop 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f5649e65640 1 -- 172.21.15.66:0/3560158202 reap_dead start 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 shutdown_connections 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 >> 172.21.15.66:0/3560158202 conn(0x7f564406f060 msgr2=0x7f5644157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 shutdown_connections 2024-03-30T13:13:08.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.105+0000 7f564b8c8640 1 -- 172.21.15.66:0/3560158202 wait complete. 2024-03-30T13:13:08.119 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-30T13:13:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.224+0000 7f5cebbbd640 1 Processor -- start 2024-03-30T13:13:08.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.224+0000 7f5cebbbd640 1 -- start start 2024-03-30T13:13:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5cebbbd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ce40cf3c0 0x7f5ce4162a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5cebbbd640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5ce40d38d0 0x7f5ce40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5cebbbd640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce40b9d20 con 0x7f5ce415e450 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5cebbbd640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce40cea50 con 0x7f5ce40cf3c0 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5cebbbd640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce40b9fc0 con 0x7f5ce40d38d0 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9158640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ce40cf3c0 0x7f5ce4162a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9959640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f5ce415e450 0x7f5ce40d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:60274/0 (socket says 172.21.15.66:60274) 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9959640 1 -- 172.21.15.66:0/2182199290 learned_addr learned my addr 172.21.15.66:0/2182199290 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:08.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5cea15a640 1 --2- 172.21.15.66:0/2182199290 >> v2:172.21.15.117:3300/0 conn(0x7f5ce40d38d0 0x7f5ce40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9158640 1 -- 172.21.15.66:0/2182199290 >> v2:172.21.15.117:3300/0 conn(0x7f5ce40d38d0 msgr2=0x7f5ce40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9158640 1 --2- 172.21.15.66:0/2182199290 >> v2:172.21.15.117:3300/0 conn(0x7f5ce40d38d0 0x7f5ce40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9158640 1 -- 172.21.15.66:0/2182199290 >> v1:172.21.15.66:6789/0 conn(0x7f5ce415e450 legacy=0x7f5ce40d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.225+0000 7f5ce9158640 1 -- 172.21.15.66:0/2182199290 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce40ba2a0 con 0x7f5ce40cf3c0 2024-03-30T13:13:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.226+0000 7f5ce9158640 1 --2- 172.21.15.66:0/2182199290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce4162a80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40ba4d0 tx=0x7f5cd402b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cf6cd034a79ee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.226+0000 7f5ce8957640 1 -- 172.21.15.66:0/2182199290 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4037070 con 0x7f5ce40cf3c0 2024-03-30T13:13:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.226+0000 7f5ce8957640 1 -- 172.21.15.66:0/2182199290 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd4030710 con 0x7f5ce40cf3c0 2024-03-30T13:13:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.226+0000 7f5ce8957640 1 -- 172.21.15.66:0/2182199290 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4030ae0 con 0x7f5ce40cf3c0 2024-03-30T13:13:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 -- 172.21.15.66:0/2182199290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40cf3c0 msgr2=0x7f5ce4162a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/2182199290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce4162a80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40ba4d0 tx=0x7f5cd402b680 comp rx=0 tx=0).stop 2024-03-30T13:13:08.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 -- 172.21.15.66:0/2182199290 shutdown_connections 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/2182199290 >> v2:172.21.15.117:3300/0 conn(0x7f5ce40d38d0 0x7f5ce40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/2182199290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce4162a80 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 -- 172.21.15.66:0/2182199290 >> 172.21.15.66:0/2182199290 conn(0x7f5ce40678f0 msgr2=0x7f5ce40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 -- 172.21.15.66:0/2182199290 shutdown_connections 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.227+0000 7f5cebbbd640 1 -- 172.21.15.66:0/2182199290 wait complete. 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.228+0000 7f5cebbbd640 1 Processor -- start 2024-03-30T13:13:08.237 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.228+0000 7f5cebbbd640 1 -- start start 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce41f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 0x7f5ce40c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ce415e450 0x7f5ce4160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce415e9b0 con 0x7f5ce40d38d0 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce415e830 con 0x7f5ce40cf3c0 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce40cf9a0 con 0x7f5ce415e450 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 0x7f5ce40c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 0x7f5ce40c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60030/0 (socket says 172.21.15.66:60030) 2024-03-30T13:13:08.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 -- 172.21.15.66:0/1907877596 learned_addr learned my addr 172.21.15.66:0/1907877596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 -- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ce415e450 msgr2=0x7f5ce4160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cea15a640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ce415e450 0x7f5ce4160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9959640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce41f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ce415e450 0x7f5ce4160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 -- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce40cf3c0 msgr2=0x7f5ce41f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce41f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 -- 172.21.15.66:0/1907877596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce41f89f0 con 0x7f5ce40d38d0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cea15a640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ce415e450 0x7f5ce4160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9959640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce41f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5ce9158640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 0x7f5ce40c7e70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009860 tx=0x7f5cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4041020 con 0x7f5ce40d38d0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd4032a40 con 0x7f5ce40d38d0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce41f8150 con 0x7f5ce40d38d0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.229+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4037040 con 0x7f5ce40d38d0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.230+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce41f8710 con 0x7f5ce40d38d0 2024-03-30T13:13:08.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.230+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce4075900 con 0x7f5ce40d38d0 2024-03-30T13:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.231+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5cd4044020 con 0x7f5ce40d38d0 2024-03-30T13:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.231+0000 7f5cda7fc640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5cc806c530 0x7f5cc806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.231+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f5cd40ad060 con 0x7f5ce40d38d0 2024-03-30T13:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.231+0000 7f5ce9959640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5cc806c530 0x7f5cc806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.232+0000 7f5ce9959640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5cc806c530 0x7f5cc806e9f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0000f60 tx=0x7f5ce0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.236+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5cd404b030 con 0x7f5ce40d38d0 2024-03-30T13:13:08.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.419+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5ce40d2000 con 0x7f5cc806c530 2024-03-30T13:13:08.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.419+0000 7f5cda7fc640 1 -- 172.21.15.66:0/1907877596 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5ce40d2000 con 0x7f5cc806c530 2024-03-30T13:13:08.423 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 6 to deep-scrub 2024-03-30T13:13:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5cc806c530 msgr2=0x7f5cc806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5cc806c530 0x7f5cc806e9f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0000f60 tx=0x7f5ce0009040 comp rx=0 tx=0).stop 2024-03-30T13:13:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 msgr2=0x7f5ce40c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 0x7f5ce40c7e70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009860 tx=0x7f5cd40046c0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 shutdown_connections 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f5cc806c530 0x7f5cc806e9f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ce415e450 0x7f5ce4160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ce40d38d0 0x7f5ce40c7e70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 --2- 172.21.15.66:0/1907877596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ce40cf3c0 0x7f5ce41f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 >> 172.21.15.66:0/1907877596 conn(0x7f5ce40678f0 msgr2=0x7f5ce40b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 shutdown_connections 2024-03-30T13:13:08.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.421+0000 7f5cebbbd640 1 -- 172.21.15.66:0/1907877596 wait complete. 2024-03-30T13:13:08.435 INFO:tasks.ceph:Scrubbing osd.7 2024-03-30T13:13:08.436 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:13:08.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.543+0000 7fae4c7ba640 1 Processor -- start 2024-03-30T13:13:08.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.543+0000 7fae4c7ba640 1 -- start start 2024-03-30T13:13:08.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4c7ba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440d10b0 0x7fae440cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4c7ba640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae440cfa20 0x7fae440c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4c7ba640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae440cc3a0 con 0x7fae440ccd90 2024-03-30T13:13:08.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4c7ba640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fae440cfee0 con 0x7fae440cfa20 2024-03-30T13:13:08.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4c7ba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fae440ce0d0 con 0x7fae440d10b0 2024-03-30T13:13:08.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae440cfa20 0x7fae440c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae440cfa20 0x7fae440c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60032/0 (socket says 172.21.15.66:60032) 2024-03-30T13:13:08.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 -- 172.21.15.66:0/3323420982 learned_addr learned my addr 172.21.15.66:0/3323420982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:08.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae49d55640 1 --2- 172.21.15.66:0/3323420982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440d10b0 0x7fae440cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 -- 172.21.15.66:0/3323420982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440d10b0 msgr2=0x7fae440cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:08.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 --2- 172.21.15.66:0/3323420982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440d10b0 0x7fae440cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 -- 172.21.15.66:0/3323420982 >> v1:172.21.15.117:6789/0 conn(0x7fae440ccd90 legacy=0x7fae440cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.550 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.544+0000 7fae4ad57640 1 -- 172.21.15.66:0/3323420982 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae440ce3b0 con 0x7fae440cfa20 2024-03-30T13:13:08.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.545+0000 7fae4ad57640 1 --2- 172.21.15.66:0/3323420982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cfa20 0x7fae440c7450 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009e50 tx=0x7fae3c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226b5243de1ef89d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.545+0000 7fae49554640 1 -- 172.21.15.66:0/3323420982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c02cb00 con 0x7fae440cfa20 2024-03-30T13:13:08.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.545+0000 7fae49554640 1 -- 172.21.15.66:0/3323420982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c02cca0 con 0x7fae440cfa20 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.545+0000 7fae49554640 1 -- 172.21.15.66:0/3323420982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c034610 con 0x7fae440cfa20 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3323420982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cfa20 msgr2=0x7fae440c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 --2- 172.21.15.66:0/3323420982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cfa20 0x7fae440c7450 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009e50 tx=0x7fae3c02bf60 comp rx=0 tx=0).stop 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3323420982 shutdown_connections 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 --2- 172.21.15.66:0/3323420982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cfa20 0x7fae440c7450 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 --2- 172.21.15.66:0/3323420982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440d10b0 0x7fae440cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3323420982 >> 172.21.15.66:0/3323420982 conn(0x7fae440d0380 msgr2=0x7fae440d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3323420982 shutdown_connections 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.546+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3323420982 wait complete. 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.547+0000 7fae4c7ba640 1 Processor -- start 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.547+0000 7fae4c7ba640 1 -- start start 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4c7ba640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440ccd90 0x7fae4415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4c7ba640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440cfa20 0x7fae44160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4c7ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440d10b0 0x7fae4409ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4c7ba640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae440cca00 con 0x7fae440ccd90 2024-03-30T13:13:08.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4c7ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae440cc880 con 0x7fae440d10b0 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4c7ba640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7fae440c7990 con 0x7fae440cfa20 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440ccd90 0x7fae4415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440ccd90 0x7fae4415ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60044/0 (socket says 172.21.15.66:60044) 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 -- 172.21.15.66:0/3172561261 learned_addr learned my addr 172.21.15.66:0/3172561261 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae49d55640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440cfa20 0x7fae44160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4ad57640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440d10b0 0x7fae4409ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 -- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440cfa20 msgr2=0x7fae44160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae440cfa20 0x7fae44160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 -- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440d10b0 msgr2=0x7fae4409ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440d10b0 0x7fae4409ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4a556640 1 -- 172.21.15.66:0/3172561261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4415f1a0 con 0x7fae440ccd90 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.548+0000 7fae4ad57640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae440d10b0 0x7fae4409ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.549+0000 7fae4a556640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440ccd90 0x7fae4415ebc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c400 tx=0x7fae4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.549+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae400052d0 con 0x7fae440ccd90 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.549+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae40005490 con 0x7fae440ccd90 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.549+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40007260 con 0x7fae440ccd90 2024-03-30T13:13:08.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.549+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4415f480 con 0x7fae440ccd90 2024-03-30T13:13:08.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.549+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4415fd20 con 0x7fae440ccd90 2024-03-30T13:13:08.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.550+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae4415f880 con 0x7fae440ccd90 2024-03-30T13:13:08.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.550+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fae40007400 con 0x7fae440ccd90 2024-03-30T13:13:08.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae3affd640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae2806c560 0x7fae2806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fae4008a910 con 0x7fae440ccd90 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae49d55640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae2806c560 0x7fae2806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae3affd640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7fae28072a00 0x7fae28074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 --> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae40009e50 con 0x7fae28072a00 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fae4008f2c0 con 0x7fae440ccd90 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae4ad57640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7fae28072a00 0x7fae28074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.551+0000 7fae49d55640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae2806c560 0x7fae2806ea20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae441610b0 tx=0x7fae34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.552+0000 7fae4ad57640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7fae28072a00 0x7fae28074e40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.552+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== osd.7 v2:172.21.15.117:6816/3946720435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae40009e50 con 0x7fae28072a00 2024-03-30T13:13:08.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.566+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 --> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fae44075900 con 0x7fae28072a00 2024-03-30T13:13:08.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.566+0000 7fae3affd640 1 -- 172.21.15.66:0/3172561261 <== osd.7 v2:172.21.15.117:6816/3946720435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fae44075900 con 0x7fae28072a00 2024-03-30T13:13:08.570 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-30T13:13:08.571 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:13:08.571 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-30T13:13:08.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7fae28072a00 msgr2=0x7fae28074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.117:6816/3946720435,v1:172.21.15.117:6817/3946720435] conn(0x7fae28072a00 0x7fae28074e40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae2806c560 msgr2=0x7fae2806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7fae2806c560 0x7fae2806ea20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae441610b0 tx=0x7fae34008040 comp rx=0 tx=0).stop 2024-03-30T13:13:08.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440ccd90 msgr2=0x7fae4415ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 --2- 172.21.15.66:0/3172561261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440ccd90 0x7fae4415ebc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c400 tx=0x7fae4000c8c0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4ad57640 1 -- 172.21.15.66:0/3172561261 reap_dead start 2024-03-30T13:13:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 shutdown_connections 2024-03-30T13:13:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 >> 172.21.15.66:0/3172561261 conn(0x7fae440d0380 msgr2=0x7fae440b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 shutdown_connections 2024-03-30T13:13:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.567+0000 7fae4c7ba640 1 -- 172.21.15.66:0/3172561261 wait complete. 2024-03-30T13:13:08.581 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-30T13:13:08.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.685+0000 7f8f67c22640 1 Processor -- start 2024-03-30T13:13:08.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.685+0000 7f8f67c22640 1 -- start start 2024-03-30T13:13:08.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f67c22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6015e450 0x7f8f600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f67c22640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f60162a80 0x7f8f600d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f67c22640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f600b9d20 con 0x7f8f600d38d0 2024-03-30T13:13:08.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f67c22640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8f600cea50 con 0x7f8f60162a80 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f67c22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8f600b9fc0 con 0x7f8f6015e450 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f60162a80 0x7f8f600d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f60162a80 0x7f8f600d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60050/0 (socket says 172.21.15.66:60050) 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 -- 172.21.15.66:0/2838487313 learned_addr learned my addr 172.21.15.66:0/2838487313 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f659be640 1 --2- 172.21.15.66:0/2838487313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6015e450 0x7f8f600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 -- 172.21.15.66:0/2838487313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6015e450 msgr2=0x7f8f600cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 --2- 172.21.15.66:0/2838487313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6015e450 0x7f8f600cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 -- 172.21.15.66:0/2838487313 >> v1:172.21.15.117:6789/0 conn(0x7f8f600d38d0 legacy=0x7f8f600b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.686+0000 7f8f651bd640 1 -- 172.21.15.66:0/2838487313 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f60162ee0 con 0x7f8f60162a80 2024-03-30T13:13:08.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.687+0000 7f8f651bd640 1 --2- 172.21.15.66:0/2838487313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60162a80 0x7f8f600d2220 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009870 tx=0x7f8f5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f59a3164e84f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.687+0000 7f8f649bc640 1 -- 172.21.15.66:0/2838487313 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50035020 con 0x7f8f60162a80 2024-03-30T13:13:08.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.687+0000 7f8f649bc640 1 -- 172.21.15.66:0/2838487313 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f50031640 con 0x7f8f60162a80 2024-03-30T13:13:08.693 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.687+0000 7f8f649bc640 1 -- 172.21.15.66:0/2838487313 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5002d040 con 0x7f8f60162a80 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 -- 172.21.15.66:0/2838487313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60162a80 msgr2=0x7f8f600d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 --2- 172.21.15.66:0/2838487313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60162a80 0x7f8f600d2220 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009870 tx=0x7f8f5002b8f0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 -- 172.21.15.66:0/2838487313 shutdown_connections 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 --2- 172.21.15.66:0/2838487313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60162a80 0x7f8f600d2220 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 --2- 172.21.15.66:0/2838487313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6015e450 0x7f8f600cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 -- 172.21.15.66:0/2838487313 >> 172.21.15.66:0/2838487313 conn(0x7f8f600678f0 msgr2=0x7f8f600d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 -- 172.21.15.66:0/2838487313 shutdown_connections 2024-03-30T13:13:08.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.688+0000 7f8f67c22640 1 -- 172.21.15.66:0/2838487313 wait complete. 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.689+0000 7f8f67c22640 1 Processor -- start 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.689+0000 7f8f67c22640 1 -- start start 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f67c22640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f600d38d0 0x7f8f600cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f67c22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 0x7f8f600ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f67c22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f60162a80 0x7f8f601f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f67c22640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6015e830 con 0x7f8f6015e450 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f67c22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f600cef30 con 0x7f8f60162a80 2024-03-30T13:13:08.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f67c22640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6015e9b0 con 0x7f8f600d38d0 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 0x7f8f600ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 0x7f8f600ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60062/0 (socket says 172.21.15.66:60062) 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 -- 172.21.15.66:0/803632596 learned_addr learned my addr 172.21.15.66:0/803632596 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f659be640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f600d38d0 0x7f8f600cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f661bf640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f60162a80 0x7f8f601f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 -- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f600d38d0 msgr2=0x7f8f600cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f600d38d0 0x7f8f600cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 -- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f60162a80 msgr2=0x7f8f601f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f60162a80 0x7f8f601f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 -- 172.21.15.66:0/803632596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f60158760 con 0x7f8f6015e450 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f661bf640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f60162a80 0x7f8f601f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f659be640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f600d38d0 0x7f8f600cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.690+0000 7f8f651bd640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 0x7f8f600ccf60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8f50000c00 tx=0x7f8f500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.691+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50040070 con 0x7f8f6015e450 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.691+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f500399f0 con 0x7f8f6015e450 2024-03-30T13:13:08.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.691+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f60158a40 con 0x7f8f6015e450 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.691+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50035040 con 0x7f8f6015e450 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.691+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f60159000 con 0x7f8f6015e450 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.692+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f60075900 con 0x7f8f6015e450 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.692+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8f5004b060 con 0x7f8f6015e450 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.692+0000 7f8f567fc640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8f4406c530 0x7f8f4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.692+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f8f500acd10 con 0x7f8f6015e450 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.693+0000 7f8f659be640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8f4406c530 0x7f8f4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.693+0000 7f8f659be640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8f4406c530 0x7f8f4406e9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ac0 tx=0x7f8f5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:08.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.697+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8f5007b530 con 0x7f8f6015e450 2024-03-30T13:13:08.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.882+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f8f60154e00 con 0x7f8f4406c530 2024-03-30T13:13:08.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.882+0000 7f8f567fc640 1 -- 172.21.15.66:0/803632596 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8f60154e00 con 0x7f8f4406c530 2024-03-30T13:13:08.887 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 7 to deep-scrub 2024-03-30T13:13:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8f4406c530 msgr2=0x7f8f4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8f4406c530 0x7f8f4406e9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ac0 tx=0x7f8f5c008040 comp rx=0 tx=0).stop 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 msgr2=0x7f8f600ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 0x7f8f600ccf60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8f50000c00 tx=0x7f8f500046c0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 shutdown_connections 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f8f4406c530 0x7f8f4406e9f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f60162a80 0x7f8f601f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6015e450 0x7f8f600ccf60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 --2- 172.21.15.66:0/803632596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f600d38d0 0x7f8f600cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 >> 172.21.15.66:0/803632596 conn(0x7f8f600678f0 msgr2=0x7f8f600b8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 shutdown_connections 2024-03-30T13:13:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:08.884+0000 7f8f67c22640 1 -- 172.21.15.66:0/803632596 wait complete. 2024-03-30T13:13:08.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:13:09.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.002+0000 7f2f36470640 1 Processor -- start 2024-03-30T13:13:09.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.002+0000 7f2f36470640 1 -- start start 2024-03-30T13:13:09.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f36470640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f301451b0 0x7f2f301608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:09.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f36470640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f3006b580 0x7f2f3006ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:09.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f36470640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f300ce810 con 0x7f2f30145d10 2024-03-30T13:13:09.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f36470640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2f30145590 con 0x7f2f3006b580 2024-03-30T13:13:09.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f36470640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2f30160440 con 0x7f2f301451b0 2024-03-30T13:13:09.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f3006b580 0x7f2f3006ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:09.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f3006b580 0x7f2f3006ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60070/0 (socket says 172.21.15.66:60070) 2024-03-30T13:13:09.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 -- 172.21.15.66:0/989756679 learned_addr learned my addr 172.21.15.66:0/989756679 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:09.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f2ffff640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f301451b0 0x7f2f301608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:09.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 -- 172.21.15.66:0/989756679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f301451b0 msgr2=0x7f2f301608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:09.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f301451b0 0x7f2f301608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 -- 172.21.15.66:0/989756679 >> v1:172.21.15.117:6789/0 conn(0x7f2f30145d10 legacy=0x7f2f300cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:09.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f34a0d640 1 -- 172.21.15.66:0/989756679 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f300ce250 con 0x7f2f3006b580 2024-03-30T13:13:09.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.003+0000 7f2f2ffff640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f301451b0 0x7f2f301608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:09.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.004+0000 7f2f34a0d640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f3006b580 0x7f2f3006ba10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009e50 tx=0x7f2f2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58023d48962e04a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:09.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.004+0000 7f2f2effd640 1 -- 172.21.15.66:0/989756679 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2802ca70 con 0x7f2f3006b580 2024-03-30T13:13:09.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.004+0000 7f2f2effd640 1 -- 172.21.15.66:0/989756679 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2802cc10 con 0x7f2f3006b580 2024-03-30T13:13:09.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.004+0000 7f2f2effd640 1 -- 172.21.15.66:0/989756679 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f280343f0 con 0x7f2f3006b580 2024-03-30T13:13:09.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 -- 172.21.15.66:0/989756679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f3006b580 msgr2=0x7f2f3006ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:09.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f3006b580 0x7f2f3006ba10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009e50 tx=0x7f2f2802bed0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 -- 172.21.15.66:0/989756679 shutdown_connections 2024-03-30T13:13:09.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f3006b580 0x7f2f3006ba10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 --2- 172.21.15.66:0/989756679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f301451b0 0x7f2f301608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 -- 172.21.15.66:0/989756679 >> 172.21.15.66:0/989756679 conn(0x7f2f300b2e10 msgr2=0x7f2f300b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:09.014 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 -- 172.21.15.66:0/989756679 shutdown_connections 2024-03-30T13:13:09.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.005+0000 7f2f36470640 1 -- 172.21.15.66:0/989756679 wait complete. 2024-03-30T13:13:09.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.006+0000 7f2f36470640 1 Processor -- start 2024-03-30T13:13:09.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.006+0000 7f2f36470640 1 -- start start 2024-03-30T13:13:09.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f36470640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f3006b580 0x7f2f30155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:09.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f36470640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 0x7f2f300b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:09.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f36470640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30145d10 0x7f2f300b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:09.019 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f36470640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f301600d0 con 0x7f2f301451b0 2024-03-30T13:13:09.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f36470640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f300cdff0 con 0x7f2f30145d10 2024-03-30T13:13:09.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f36470640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f2f30160e10 con 0x7f2f3006b580 2024-03-30T13:13:09.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 0x7f2f300b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:09.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 0x7f2f300b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60074/0 (socket says 172.21.15.66:60074) 2024-03-30T13:13:09.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 -- 172.21.15.66:0/3762405467 learned_addr learned my addr 172.21.15.66:0/3762405467 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:09.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f34a0d640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30145d10 0x7f2f300b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:09.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2ffff640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f3006b580 0x7f2f30155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:09.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 -- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f3006b580 msgr2=0x7f2f30155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:09.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f3006b580 0x7f2f30155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 -- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30145d10 msgr2=0x7f2f300b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:09.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30145d10 0x7f2f300b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 -- 172.21.15.66:0/3762405467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f30156260 con 0x7f2f301451b0 2024-03-30T13:13:09.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.007+0000 7f2f2f7fe640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 0x7f2f300b2170 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c400 tx=0x7f2f2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:09.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.008+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200052d0 con 0x7f2f301451b0 2024-03-30T13:13:09.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.008+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f20005490 con 0x7f2f301451b0 2024-03-30T13:13:09.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.008+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f301588f0 con 0x7f2f301451b0 2024-03-30T13:13:09.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.008+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20007260 con 0x7f2f301451b0 2024-03-30T13:13:09.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.008+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f30158eb0 con 0x7f2f301451b0 2024-03-30T13:13:09.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.009+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f30075a10 con 0x7f2f301451b0 2024-03-30T13:13:09.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.009+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2f20007400 con 0x7f2f301451b0 2024-03-30T13:13:09.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.010+0000 7f2f2cff9640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2f1c06c530 0x7f2f1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:09.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.010+0000 7f2f2ffff640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2f1c06c530 0x7f2f1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:09.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.010+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f2f2008aad0 con 0x7f2f301451b0 2024-03-30T13:13:09.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.010+0000 7f2f2ffff640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2f1c06c530 0x7f2f1c06e9f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f24006fd0 tx=0x7f2f24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:09.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.014+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2f30156260 con 0x7f2f301451b0 2024-03-30T13:13:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.196+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f300729b0 con 0x7f2f1c06c530 2024-03-30T13:13:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.197+0000 7f2f2cff9640 1 -- 172.21.15.66:0/3762405467 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13475 (secure 0 0 0) 0x7f2f300729b0 con 0x7f2f1c06c530 2024-03-30T13:13:09.201 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:13:09.203 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2f1c06c530 msgr2=0x7f2f1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2f1c06c530 0x7f2f1c06e9f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f24006fd0 tx=0x7f2f24008040 comp rx=0 tx=0).stop 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 msgr2=0x7f2f300b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 0x7f2f300b2170 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c400 tx=0x7f2f2000c8c0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 shutdown_connections 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f2f1c06c530 0x7f2f1c06e9f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30145d10 0x7f2f300b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f301451b0 0x7f2f300b2170 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 --2- 172.21.15.66:0/3762405467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f3006b580 0x7f2f30155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 >> 172.21.15.66:0/3762405467 conn(0x7f2f300b2e10 msgr2=0x7f2f300d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 shutdown_connections 2024-03-30T13:13:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:09.199+0000 7f2f36470640 1 -- 172.21.15.66:0/3762405467 wait complete. 2024-03-30T13:13:09.214 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-03-30T13:13:08.833527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238368,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736352,"statfs":{"total":773094113280,"available":772850024448,"internally_reserved":0,"allocated":5677056,"data_stored":3187832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-173890432,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-191,"num_read_kb":-168572,"num_write":-562,"num_write_kb":-172416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.097008"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":222,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-30T13:13:01.151477+0000","last_change":"2024-03-30T13:09:30.623051+0000","last_active":"2024-03-30T13:13:01.151477+0000","last_peered":"2024-03-30T13:13:01.151477+0000","last_clean":"2024-03-30T13:13:01.151477+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:13:01.151477+0000","last_undegraded":"2024-03-30T13:13:01.151477+0000","last_fullsized":"2024-03-30T13:13:01.151477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:09:29.065006+0000","last_clean_scrub_stamp":"2024-03-30T13:09:29.065006+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:52.200858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1155072,"data_stored":841255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1155072,"data_stored":841255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31732,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:13:09.217 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-30T13:09:29.065006+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=30, tm_hour=13, tm_min=9, tm_sec=29, tm_wday=5, tm_yday=90, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=30, tm_hour=13, tm_min=13, tm_sec=4, tm_wday=5, tm_yday=90, tm_isdst=0) 2024-03-30T13:13:09.217 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-30T13:13:12.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:12.623 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:29.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:13:29.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.335+0000 7f18582a5640 1 Processor -- start 2024-03-30T13:13:29.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.335+0000 7f18582a5640 1 -- start start 2024-03-30T13:13:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f18582a5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f185006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f18582a5640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f185006fb10 0x7f185006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f18582a5640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18500b4f90 con 0x7f18501470d0 2024-03-30T13:13:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f18582a5640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f185006bf20 con 0x7f185006fb10 2024-03-30T13:13:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f18582a5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f1850162110 con 0x7f185006b580 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f1856842640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f185006fb10 0x7f185006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f1856842640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f185006fb10 0x7f185006ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45588/0 (socket says 172.21.15.66:45588) 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f1856842640 1 -- 172.21.15.66:0/1392312828 learned_addr learned my addr 172.21.15.66:0/1392312828 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.336+0000 7f1855840640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f185006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.337+0000 7f1856842640 1 -- 172.21.15.66:0/1392312828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 msgr2=0x7f185006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.337+0000 7f1856842640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f185006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.337+0000 7f1856842640 1 -- 172.21.15.66:0/1392312828 >> v1:172.21.15.117:6789/0 conn(0x7f18501470d0 legacy=0x7f18500b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.337+0000 7f1856842640 1 -- 172.21.15.66:0/1392312828 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18501623f0 con 0x7f185006fb10 2024-03-30T13:13:29.343 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.337+0000 7f1855840640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f185006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:29.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.337+0000 7f1856842640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185006fb10 0x7f185006ff60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1848009870 tx=0x7f184802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e751bdf5b1e200f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f185503f640 1 -- 172.21.15.66:0/1392312828 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848034070 con 0x7f185006fb10 2024-03-30T13:13:29.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f185503f640 1 -- 172.21.15.66:0/1392312828 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848030840 con 0x7f185006fb10 2024-03-30T13:13:29.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f185503f640 1 -- 172.21.15.66:0/1392312828 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848030c10 con 0x7f185006fb10 2024-03-30T13:13:29.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 -- 172.21.15.66:0/1392312828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185006fb10 msgr2=0x7f185006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185006fb10 0x7f185006ff60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1848009870 tx=0x7f184802b980 comp rx=0 tx=0).stop 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 -- 172.21.15.66:0/1392312828 shutdown_connections 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185006fb10 0x7f185006ff60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 --2- 172.21.15.66:0/1392312828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f185006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 -- 172.21.15.66:0/1392312828 >> 172.21.15.66:0/1392312828 conn(0x7f18500b2e10 msgr2=0x7f18500b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 -- 172.21.15.66:0/1392312828 shutdown_connections 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.338+0000 7f18582a5640 1 -- 172.21.15.66:0/1392312828 wait complete. 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 Processor -- start 2024-03-30T13:13:29.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 -- start start 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f1850158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185006fb10 0x7f185013ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 0x7f185015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1850073700 con 0x7f18501470d0 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18501629b0 con 0x7f185006fb10 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f18582a5640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f18501474b0 con 0x7f185006b580 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f1856842640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 0x7f185015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.340+0000 7f1856842640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 0x7f185015b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45596/0 (socket says 172.21.15.66:45596) 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 -- 172.21.15.66:0/2666342668 learned_addr learned my addr 172.21.15.66:0/2666342668 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1855840640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185006fb10 0x7f185013ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856041640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f1850158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 -- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 msgr2=0x7f1850158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f1850158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 -- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185006fb10 msgr2=0x7f185013ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185006fb10 0x7f185013ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 -- 172.21.15.66:0/2666342668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185013f120 con 0x7f18501470d0 2024-03-30T13:13:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f1856842640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 0x7f185015b1d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1848009870 tx=0x7f1848004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848048070 con 0x7f18501470d0 2024-03-30T13:13:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848030640 con 0x7f18501470d0 2024-03-30T13:13:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185015ba40 con 0x7f18501470d0 2024-03-30T13:13:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848034040 con 0x7f18501470d0 2024-03-30T13:13:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.341+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185015ef20 con 0x7f18501470d0 2024-03-30T13:13:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.342+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850075a10 con 0x7f18501470d0 2024-03-30T13:13:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.343+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f184804b070 con 0x7f18501470d0 2024-03-30T13:13:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.343+0000 7f184effd640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183c06c530 0x7f183c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.343+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f18480ad3d0 con 0x7f18501470d0 2024-03-30T13:13:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.343+0000 7f1856041640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183c06c530 0x7f183c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.344+0000 7f1856041640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183c06c530 0x7f183c06e9f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1840001e70 tx=0x7f1840009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.348+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f185013f120 con 0x7f18501470d0 2024-03-30T13:13:29.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.529+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 --> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1850072d40 con 0x7f183c06c530 2024-03-30T13:13:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.530+0000 7f184effd640 1 -- 172.21.15.66:0/2666342668 <== mgr.4101 v2:172.21.15.66:6832/1829857110 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7f1850072d40 con 0x7f183c06c530 2024-03-30T13:13:29.534 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:29.535 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-03-30T13:13:29.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183c06c530 msgr2=0x7f183c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183c06c530 0x7f183c06e9f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1840001e70 tx=0x7f1840009040 comp rx=0 tx=0).stop 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 msgr2=0x7f185015b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 0x7f185015b1d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1848009870 tx=0x7f1848004730 comp rx=0 tx=0).stop 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 shutdown_connections 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f183c06c530 0x7f183c06e9f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18501470d0 0x7f185015b1d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f185006fb10 0x7f185013ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 --2- 172.21.15.66:0/2666342668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f185006b580 0x7f1850158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 >> 172.21.15.66:0/2666342668 conn(0x7f18500b2e10 msgr2=0x7f18500b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:29.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 shutdown_connections 2024-03-30T13:13:29.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.532+0000 7f18582a5640 1 -- 172.21.15.66:0/2666342668 wait complete. 2024-03-30T13:13:29.547 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-03-30T13:13:28.837508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238356,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736364,"statfs":{"total":773094113280,"available":772850036736,"internally_reserved":0,"allocated":5677056,"data_stored":3187832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002385"},"pg_stats":[{"pgid":"1.0","version":"48'119","reported_seq":233,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-30T13:13:09.330028+0000","last_change":"2024-03-30T13:13:09.330088+0000","last_active":"2024-03-30T13:13:09.330028+0000","last_peered":"2024-03-30T13:13:09.330028+0000","last_clean":"2024-03-30T13:13:09.330028+0000","last_became_active":"2024-03-30T13:09:30.622156+0000","last_became_peered":"2024-03-30T13:09:30.622156+0000","last_unstale":"2024-03-30T13:13:09.330028+0000","last_undegraded":"2024-03-30T13:13:09.330028+0000","last_fullsized":"2024-03-30T13:13:09.330028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'119","last_scrub_stamp":"2024-03-30T13:13:09.329911+0000","last_deep_scrub":"48'119","last_deep_scrub_stamp":"2024-03-30T13:13:09.329911+0000","last_clean_scrub_stamp":"2024-03-30T13:13:09.329911+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:05:09.718254+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1155072,"data_stored":841255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1155072,"data_stored":841255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31732,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":561152,"data_stored":250887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:13:29.548 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-30T13:13:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.651+0000 7efdf3c7b640 1 Processor -- start 2024-03-30T13:13:29.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.651+0000 7efdf3c7b640 1 -- start start 2024-03-30T13:13:29.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf3c7b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 0x7efdec169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf3c7b640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec06d680 con 0x7efdec15e7c0 2024-03-30T13:13:29.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf3c7b640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec0d1170 con 0x7efdec15eba0 2024-03-30T13:13:29.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf3c7b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efdec0cc840 con 0x7efdec0b4800 2024-03-30T13:13:29.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf1a17640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7efdec15e7c0 0x7efdec06c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59928/0 (socket says 172.21.15.66:59928) 2024-03-30T13:13:29.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf1a17640 1 -- 172.21.15.66:0/551752092 learned_addr learned my addr 172.21.15.66:0/551752092 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf2218640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 0x7efdec169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036395676 0 0) 0x7efdec06d680 con 0x7efdec15e7c0 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd8003520 con 0x7efdec15e7c0 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347340742 0 0) 0x7efdec0d1170 con 0x7efdec15eba0 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdec06d680 con 0x7efdec15eba0 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.0 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3056049678 0 0) 0x7efdd8003520 con 0x7efdec15e7c0 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 msgr2=0x7efdec169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:29.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 0x7efdec169520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 >> v1:172.21.15.117:6789/0 conn(0x7efdec15eba0 legacy=0x7efdec06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec0d1450 con 0x7efdec15e7c0 2024-03-30T13:13:29.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.0 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3598893400 0 0) 0x7efde8001670 con 0x7efdec15e7c0 2024-03-30T13:13:29.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 >> v1:172.21.15.66:6789/0 conn(0x7efdec15e7c0 legacy=0x7efdec06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8003b70 0x7efdd8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.652+0000 7efdf0a15640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 0x7efdd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf0a15640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efdd800ba70 con 0x7efdd8004d40 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdd800bbf0 con 0x7efdd8003b70 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efdd800bda0 con 0x7efdec0b4800 2024-03-30T13:13:29.661 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf2218640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf1a17640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8003b70 0x7efdd8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf1216640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 0x7efdd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf2218640 1 -- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 msgr2=0x7efdd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf2218640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b4800 0x7efdd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf2218640 1 -- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8003b70 msgr2=0x7efdd8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf2218640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8003b70 0x7efdd8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.653+0000 7efdf2218640 1 -- 172.21.15.66:0/551752092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd8003520 con 0x7efdd8004d40 2024-03-30T13:13:29.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf2218640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdd80051c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efdec0d01e0 tx=0x7efdd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6edba2e6700555c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4039070 con 0x7efdd8004d40 2024-03-30T13:13:29.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde402cb90 con 0x7efdd8004d40 2024-03-30T13:13:29.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf0a15640 1 -- 172.21.15.66:0/551752092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4009c10 con 0x7efdd8004d40 2024-03-30T13:13:29.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf3c7b640 1 -- 172.21.15.66:0/551752092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 msgr2=0x7efdd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/551752092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdd80051c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efdec0d01e0 tx=0x7efdd8005830 comp rx=0 tx=0).stop 2024-03-30T13:13:29.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf2218640 1 -- 172.21.15.66:0/551752092 reap_dead start 2024-03-30T13:13:29.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf3c7b640 1 -- 172.21.15.66:0/551752092 shutdown_connections 2024-03-30T13:13:29.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.654+0000 7efdf3c7b640 1 -- 172.21.15.66:0/551752092 >> 172.21.15.66:0/551752092 conn(0x7efdec0b3ce0 msgr2=0x7efdec0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:29.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.655+0000 7efdf3c7b640 1 -- 172.21.15.66:0/551752092 shutdown_connections 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.655+0000 7efdf3c7b640 1 -- 172.21.15.66:0/551752092 wait complete. 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.656+0000 7efdf3c7b640 1 Processor -- start 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.656+0000 7efdf3c7b640 1 -- start start 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.656+0000 7efdf3c7b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdec0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b1740 0x7efdec1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdec0b6ed0 0x7efdec0b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efdec0cdce0 con 0x7efdd8004d40 2024-03-30T13:13:29.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdec0b35f0 con 0x7efdec0b6ed0 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7efdec0cde60 con 0x7efdec0b1740 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdec0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdec0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45614/0 (socket says 172.21.15.66:45614) 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 -- 172.21.15.66:0/4205216147 learned_addr learned my addr 172.21.15.66:0/4205216147 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1216640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b1740 0x7efdec1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf2218640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdec0b6ed0 0x7efdec0b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 -- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b1740 msgr2=0x7efdec1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b1740 0x7efdec1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 -- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdec0b6ed0 msgr2=0x7efdec0b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdec0b6ed0 0x7efdec0b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 -- 172.21.15.66:0/4205216147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec153be0 con 0x7efdd8004d40 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf2218640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdec0b6ed0 0x7efdec0b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf1a17640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdec0aeed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7efde8007290 tx=0x7efde8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde800f9c0 con 0x7efdd8004d40 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde8018070 con 0x7efdd8004d40 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec0b1ed0 con 0x7efdd8004d40 2024-03-30T13:13:29.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde800fd80 con 0x7efdd8004d40 2024-03-30T13:13:29.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.657+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec0b2490 con 0x7efdd8004d40 2024-03-30T13:13:29.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.658+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec075890 con 0x7efdd8004d40 2024-03-30T13:13:29.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.659+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efde801d460 con 0x7efdd8004d40 2024-03-30T13:13:29.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.659+0000 7efde27fc640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efdd006c560 0x7efdd006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.659+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7efde808b5b0 con 0x7efdd8004d40 2024-03-30T13:13:29.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.659+0000 7efdf1216640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efdd006c560 0x7efdd006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.660+0000 7efdf1216640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efdd006c560 0x7efdd006ea20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efdec15cc60 tx=0x7efddc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.664+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efdec153be0 con 0x7efdd8004d40 2024-03-30T13:13:29.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.849+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7efdec075130 con 0x7efdd8004d40 2024-03-30T13:13:29.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.852+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7efde8059cb0 con 0x7efdd8004d40 2024-03-30T13:13:29.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.853+0000 7efde27fc640 1 -- 172.21.15.66:0/4205216147 <== mon.0 v2:172.21.15.66:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efde80595b0 con 0x7efdd8004d40 2024-03-30T13:13:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efdd006c560 msgr2=0x7efdd006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efdd006c560 0x7efdd006ea20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efdec15cc60 tx=0x7efddc009040 comp rx=0 tx=0).stop 2024-03-30T13:13:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 msgr2=0x7efdec0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdec0aeed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7efde8007290 tx=0x7efde8007750 comp rx=0 tx=0).stop 2024-03-30T13:13:29.859 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 shutdown_connections 2024-03-30T13:13:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7efdd006c560 0x7efdd006ea20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdec0b6ed0 0x7efdec0b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdec0b1740 0x7efdec1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 --2- 172.21.15.66:0/4205216147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdd8004d40 0x7efdec0aeed0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 >> 172.21.15.66:0/4205216147 conn(0x7efdec0b3ce0 msgr2=0x7efdec0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:29.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.854+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 shutdown_connections 2024-03-30T13:13:29.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.855+0000 7efdf3c7b640 1 -- 172.21.15.66:0/4205216147 wait complete. 2024-03-30T13:13:29.870 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T13:13:29.870 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:13:29.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.974+0000 7f386d60d640 1 Processor -- start 2024-03-30T13:13:29.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.974+0000 7f386d60d640 1 -- start start 2024-03-30T13:13:29.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f386d60d640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3868158ac0 0x7f38680b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f386d60d640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f386815f1d0 0x7f38680ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f386d60d640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38680d0730 con 0x7f38680b4210 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f386d60d640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f386815fab0 con 0x7f386815f1d0 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f386d60d640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 -- 0x7f386815f7d0 con 0x7f3868158ac0 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38677fe640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f386815f1d0 0x7f38680ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38667fc640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f38680b4210 0x7f386815eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:59932/0 (socket says 172.21.15.66:59932) 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38667fc640 1 -- 172.21.15.66:0/2623282261 learned_addr learned my addr 172.21.15.66:0/2623282261 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f3866ffd640 1 --2- 172.21.15.66:0/2623282261 >> v2:172.21.15.117:3300/0 conn(0x7f3868158ac0 0x7f38680b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38677fe640 1 -- 172.21.15.66:0/2623282261 >> v2:172.21.15.117:3300/0 conn(0x7f3868158ac0 msgr2=0x7f38680b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:29.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38677fe640 1 --2- 172.21.15.66:0/2623282261 >> v2:172.21.15.117:3300/0 conn(0x7f3868158ac0 0x7f38680b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38677fe640 1 -- 172.21.15.66:0/2623282261 >> v1:172.21.15.66:6789/0 conn(0x7f38680b4210 legacy=0x7f386815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f3865ffb640 1 -- 172.21.15.66:0/2623282261 <== mon.0 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504109933 0 0) 0x7f38680d0730 con 0x7f38680b4210 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.975+0000 7f38677fe640 1 -- 172.21.15.66:0/2623282261 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386806c6e0 con 0x7f386815f1d0 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.976+0000 7f38677fe640 1 --2- 172.21.15.66:0/2623282261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38680ce8c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3868070750 tx=0x7f386002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12b443e7a70ca8d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.976+0000 7f3865ffb640 1 -- 172.21.15.66:0/2623282261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860039070 con 0x7f386815f1d0 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.976+0000 7f3865ffb640 1 -- 172.21.15.66:0/2623282261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f386002cbf0 con 0x7f386815f1d0 2024-03-30T13:13:29.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.976+0000 7f3865ffb640 1 -- 172.21.15.66:0/2623282261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860034440 con 0x7f386815f1d0 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 -- 172.21.15.66:0/2623282261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 msgr2=0x7f38680ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 --2- 172.21.15.66:0/2623282261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38680ce8c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3868070750 tx=0x7f386002be40 comp rx=0 tx=0).stop 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 -- 172.21.15.66:0/2623282261 shutdown_connections 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 --2- 172.21.15.66:0/2623282261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38680ce8c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 --2- 172.21.15.66:0/2623282261 >> v2:172.21.15.117:3300/0 conn(0x7f3868158ac0 0x7f38680b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 -- 172.21.15.66:0/2623282261 >> 172.21.15.66:0/2623282261 conn(0x7f386806f820 msgr2=0x7f386806fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 -- 172.21.15.66:0/2623282261 shutdown_connections 2024-03-30T13:13:29.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.977+0000 7f386d60d640 1 -- 172.21.15.66:0/2623282261 wait complete. 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.978+0000 7f386d60d640 1 Processor -- start 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.978+0000 7f386d60d640 1 -- start start 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f386d60d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680b4210 0x7f38680b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f386d60d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868158ac0 0x7f38680b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f386d60d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38681572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f386d60d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3868158ea0 con 0x7f386815f1d0 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f386d60d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f386806cca0 con 0x7f38680b4210 2024-03-30T13:13:29.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f386d60d640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 -- 0x7f38680cee00 con 0x7f3868158ac0 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38681572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38681572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45628/0 (socket says 172.21.15.66:45628) 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 -- 172.21.15.66:0/3264625269 learned_addr learned my addr 172.21.15.66:0/3264625269 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f3866ffd640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680b4210 0x7f38680b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38667fc640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868158ac0 0x7f38680b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 -- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868158ac0 msgr2=0x7f38680b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868158ac0 0x7f38680b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 -- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680b4210 msgr2=0x7f38680b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:29.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680b4210 0x7f38680b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.979+0000 7f38677fe640 1 -- 172.21.15.66:0/3264625269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38680cbc00 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.980+0000 7f38677fe640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38681572a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3860037040 tx=0x7f3860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.980+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860039050 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.980+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3860045070 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.980+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38680cbee0 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.980+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386003e870 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.980+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38680b56c0 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.981+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3868075a10 con 0x7f386815f1d0 2024-03-30T13:13:29.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.981+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f386003ea10 con 0x7f386815f1d0 2024-03-30T13:13:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.982+0000 7f384f7fe640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f385406c530 0x7f385406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.982+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f38600ae230 con 0x7f386815f1d0 2024-03-30T13:13:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.982+0000 7f3866ffd640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f385406c530 0x7f385406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:29.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.982+0000 7f3866ffd640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f385406c530 0x7f385406e9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3858006fd0 tx=0x7f3858008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:29.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:29.986+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f386007c370 con 0x7f386815f1d0 2024-03-30T13:13:30.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.212+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f386815efb0 con 0x7f386815f1d0 2024-03-30T13:13:30.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.213+0000 7f384f7fe640 1 -- 172.21.15.66:0/3264625269 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3872 (secure 0 0 0) 0x7f3860080670 con 0x7f386815f1d0 2024-03-30T13:13:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-30T13:13:30.218 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5311,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6834/1397666157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1397666157},{"type":"v1","addr":"172.21.15.117:6834","nonce":1397666157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/164826659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6833","nonce":164826659},{"type":"v1","addr":"172.21.15.117:6835","nonce":164826659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6836/2214024755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2214024755},{"type":"v1","addr":"172.21.15.66:6836","nonce":2214024755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2233769664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6835","nonce":2233769664},{"type":"v1","addr":"172.21.15.66:6837","nonce":2233769664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-03-30T13:13:30.220 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 22 2024-03-30T13:13:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f385406c530 msgr2=0x7f385406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f385406c530 0x7f385406e9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3858006fd0 tx=0x7f3858008040 comp rx=0 tx=0).stop 2024-03-30T13:13:30.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 msgr2=0x7f38681572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:30.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38681572a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3860037040 tx=0x7f3860004a40 comp rx=0 tx=0).stop 2024-03-30T13:13:30.222 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 shutdown_connections 2024-03-30T13:13:30.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:6832/1829857110,v1:172.21.15.66:6833/1829857110] conn(0x7f385406c530 0x7f385406e9f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:30.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f386815f1d0 0x7f38681572a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:30.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3868158ac0 0x7f38680b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:30.223 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 --2- 172.21.15.66:0/3264625269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38680b4210 0x7f38680b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:30.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 >> 172.21.15.66:0/3264625269 conn(0x7f386806f820 msgr2=0x7f386806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:30.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 shutdown_connections 2024-03-30T13:13:30.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-30T13:13:30.215+0000 7f386d60d640 1 -- 172.21.15.66:0/3264625269 wait complete. 2024-03-30T13:13:30.230 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-30T13:13:30.230 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-30T13:13:30.230 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T13:13:30.231 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.240 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T13:13:30.240 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T13:13:30.240 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.290 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T13:13:30.290 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T13:13:30.290 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.303 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T13:13:30.303 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T13:13:30.303 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.355 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T13:13:30.356 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-30T13:13:30.356 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-30T13:13:30.356 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.383 INFO:tasks.ceph.osd.0:Stopped 2024-03-30T13:13:30.384 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-30T13:13:30.384 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.452 INFO:tasks.ceph.osd.1:Stopped 2024-03-30T13:13:30.452 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-30T13:13:30.452 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.521 INFO:tasks.ceph.osd.2:Stopped 2024-03-30T13:13:30.521 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-30T13:13:30.521 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.598 INFO:tasks.ceph.osd.3:Stopped 2024-03-30T13:13:30.599 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-30T13:13:30.599 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.632 INFO:tasks.ceph.osd.4:Stopped 2024-03-30T13:13:30.632 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-30T13:13:30.632 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.710 INFO:tasks.ceph.osd.5:Stopped 2024-03-30T13:13:30.710 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-30T13:13:30.710 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.787 INFO:tasks.ceph.osd.6:Stopped 2024-03-30T13:13:30.787 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-30T13:13:30.787 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.861 INFO:tasks.ceph.osd.7:Stopped 2024-03-30T13:13:30.861 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-30T13:13:30.862 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-30T13:13:30.862 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.887 INFO:tasks.ceph.mgr.y:Stopped 2024-03-30T13:13:30.888 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-30T13:13:30.888 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.929 INFO:tasks.ceph.mgr.x:Stopped 2024-03-30T13:13:30.929 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-30T13:13:30.930 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-30T13:13:30.930 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.941 INFO:tasks.ceph.mon.a:Stopped 2024-03-30T13:13:30.941 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-30T13:13:30.942 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:30.982 INFO:tasks.ceph.mon.b:Stopped 2024-03-30T13:13:30.983 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-30T13:13:30.983 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:13:31.035 INFO:tasks.ceph.mon.c:Stopped 2024-03-30T13:13:31.035 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-30T13:13:31.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-30T13:13:31.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:13:31.073 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-30T13:13:31.288 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:13:31.288 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-30T13:13:31.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:13:31.381 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-30T13:13:31.427 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:13:31.427 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-30T13:13:31.477 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:13:31.477 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-30T13:13:31.633 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:13:31.633 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-30T13:13:31.712 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:13:31.713 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-30T13:13:31.761 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:13:31.761 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-30T13:13:31.828 INFO:tasks.ceph:Archiving mon data... 2024-03-30T13:13:31.830 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/data/mon.a.tgz 2024-03-30T13:13:31.830 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-03-30T13:13:31.846 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.sn5RkRyLLf 2024-03-30T13:13:31.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sn5RkRyLLf 2024-03-30T13:13:31.983 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.sn5RkRyLLf 2024-03-30T13:13:32.072 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.sn5RkRyLLf is 346KB 2024-03-30T13:13:32.140 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.sn5RkRyLLf 2024-03-30T13:13:32.155 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/data/mon.b.tgz 2024-03-30T13:13:32.156 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-30T13:13:32.173 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.aAQP3p0onb 2024-03-30T13:13:32.174 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aAQP3p0onb 2024-03-30T13:13:32.345 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.aAQP3p0onb 2024-03-30T13:13:32.432 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.aAQP3p0onb is 459KB 2024-03-30T13:13:32.502 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.aAQP3p0onb 2024-03-30T13:13:32.516 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/data/mon.c.tgz 2024-03-30T13:13:32.517 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-30T13:13:32.571 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.K4w0Xxbz7C 2024-03-30T13:13:32.571 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.K4w0Xxbz7C 2024-03-30T13:13:32.725 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.K4w0Xxbz7C 2024-03-30T13:13:32.812 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.K4w0Xxbz7C is 459KB 2024-03-30T13:13:32.909 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.K4w0Xxbz7C 2024-03-30T13:13:32.923 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-30T13:13:32.924 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-30T13:13:32.926 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-30T13:13:33.014 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-30T13:13:33.045 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-30T13:13:33.046 INFO:tasks.ceph:Archiving crash dumps... 2024-03-30T13:13:33.048 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/remote/smithi066/crash 2024-03-30T13:13:33.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-30T13:13:33.087 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/remote/smithi117/crash 2024-03-30T13:13:33.089 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-30T13:13:33.115 INFO:tasks.ceph:Shutting down logrotate 2024-03-30T13:13:33.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:33.132 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:33.206 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:33.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:33.265 INFO:tasks.ceph:Compressing logs... 2024-03-30T13:13:33.266 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-30T13:13:33.281 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-30T13:13:33.309 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59548.log 2024-03-30T13:13:33.310 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2024-03-30T13:13:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-30T13:13:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44125.log 2024-03-30T13:13:33.355 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose --/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.37666.log 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.3.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43728.log 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44125.log.gz 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37586.log 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37666.log.gz 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.1.43728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37746.log 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43728.log.gz 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37826.log 2024-03-30T13:13:33.356 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37746.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37586.log.gz 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37746.log.gz 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37826.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.37626.log 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37826.log.gz 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44721.log 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.4.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37626.log.gz 2024-03-30T13:13:33.357 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37706.log 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.1.44721.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44721.log.gz 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37706.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.7.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37706.log.gz 2024-03-30T13:13:33.358 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37546.log 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37546.log.gz 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37786.log 2024-03-30T13:13:33.359 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mon.c.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-30T13:13:33.360 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.37786.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37786.log.gz 2024-03-30T13:13:33.375 INFO:teuthology.orchestra.run.smithi117.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-30T13:13:33.443 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-30T13:13:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48611.log 2024-03-30T13:13:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47177.log 2024-03-30T13:13:33.460 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59548.log.gz 2024-03-30T13:13:33.460 INFO:teuthology.orchestra.run.smithi066.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2024-03-30T13:13:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48359.log 2024-03-30T13:13:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48359.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.47177.log.gz 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48359.log.gz 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46278.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gz 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2024-03-30T13:13:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mon.a.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gz -5 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54896.log 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51140.log 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55226.log 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46139.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54896.log.gz 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51140.log.gz 2024-03-30T13:13:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58177.log 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54764.log 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55226.log: /var/log/ceph/ceph-client.admin.58177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51056.log 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1%/var/log/ceph/ceph-client.admin.54764.log: -- replaced with /var/log/ceph/ceph-client.admin.58177.log.gz 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48485.log 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9%/var/log/ceph/ceph-client.admin.51056.log: -- replaced with /var/log/ceph/ceph-client.admin.55226.log.gz 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 2024-03-30T13:13:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48485.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51056.log.gz 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54764.log.gz 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43654.log 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53906.log 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48485.log.gz 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44448.log: /var/log/ceph/ceph-client.admin.43654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51098.log 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53906.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43654.log.gz 2024-03-30T13:13:33.464 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54962.log 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55374.log 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51098.log.gz 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58529.log 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53906.log.gz 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58529.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55374.log.gz 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54962.log.gz 2024-03-30T13:13:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51638.log 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58529.log.gz 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53840.log 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55516.log: /var/log/ceph/ceph-client.admin.51638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44926.log 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51638.log.gz 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45606.log 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2024-03-30T13:13:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49241.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44926.log.gz 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51430.log 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53840.log.gz 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45605.log 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45606.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54830.log 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45606.log.gz 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51430.log.gz 2024-03-30T13:13:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45605.log.gz 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44658.log 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.48653.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56694.log 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54830.log.gz 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44658.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44658.log.gz 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2024-03-30T13:13:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56694.log: /var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48989.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56694.log.gz 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57306.log 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gz 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45602.log 2024-03-30T13:13:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43486.log: /var/log/ceph/ceph-client.admin.57306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50759.log 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56232.log 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57306.log.gz 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50759.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45602.log.gz 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50276.log 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56232.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50759.log.gz 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45596.log 2024-03-30T13:13:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52729.log 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45596.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50276.log.gz 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56232.log.gz 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59200.log 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7%/var/log/ceph/ceph-client.admin.52729.log: -- replaced with /var/log/ceph/ceph-client.admin.45596.log.gz 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59590.log 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59200.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52729.log.gz 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50150.log 2024-03-30T13:13:33.471 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59200.log.gz 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49604.log 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50060.log 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59590.log.gz 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49604.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50150.log.gz 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48275.log 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49604.log.gz 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2024-03-30T13:13:33.472 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58968.log 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50060.log.gz 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45942.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48275.log.gz 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56430.log 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58968.log.gz 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37795.log 2024-03-30T13:13:33.473 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45943.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56430.log.gz 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44800.log 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57504.log 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44800.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37795.log.gz 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52243.log 2024-03-30T13:13:33.474 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44800.log.gz 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45936.log 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57504.log: /var/log/ceph/ceph-client.admin.52243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45173.log 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45936.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52243.log.gz 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52917.log 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57504.log.gz 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45936.log.gz 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45599.log 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45173.log.gz 2024-03-30T13:13:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.52917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55292.log 0.0% 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.0.52917.log.gz 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59800.log 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46840.log 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45599.log.gz 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57889.log 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59800.log.gz 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46840.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55292.log.gz 2024-03-30T13:13:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59379.log 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46840.log.gz 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45939.log 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5%/var/log/ceph/ceph-client.admin.59379.log: -- replaced with /var/log/ceph/ceph-client.admin.57889.log.gz 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58739.log 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59379.log.gz 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47051.log 2024-03-30T13:13:33.477 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45939.log.gz 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50318.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58739.log.gz 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54368.log 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49283.log 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47051.log.gz 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54368.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2024-03-30T13:13:33.478 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50845.log 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44574.log 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50845.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49283.log.gz 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47009.log 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42999.log 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44574.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50845.log.gz 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2024-03-30T13:13:33.479 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44574.log.gz 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42999.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47009.log.gz 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59422.log 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43126.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42999.log.gz 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48737.log 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45604.log 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59422.log.gz 2024-03-30T13:13:33.480 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48737.log.gz 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42918.log 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49325.log 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45604.log.gz 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42918.log.gz 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59884.log 2024-03-30T13:13:33.481 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49325.log.gz 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45089.log 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59884.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48905.log 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45089.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59884.log.gz 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52201.log 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45089.log.gz 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2024-03-30T13:13:33.482 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57847.log 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48905.log.gz 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45005.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52201.log.gz 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51806.log 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45941.log 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57847.log.gz 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-03-30T13:13:33.483 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51806.log.gz 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50480.log 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50606.log 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45941.log.gz 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54038.log 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50480.log.gz 2024-03-30T13:13:33.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47219.log 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50606.log.gz 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54566.log 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54566.log: 91.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.54038.log.gz 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54434.log 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47219.log.gz 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54434.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54566.log.gz 2024-03-30T13:13:33.485 INFO:teuthology.orchestra.run.smithi066.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46796.log 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45341.log 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.54434.log.gz 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44257.log 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2%/var/log/ceph/ceph-client.admin.45341.log: -- replaced with /var/log/ceph/ceph-client.admin.46796.log.gz 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-30T13:13:33.486 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2%/var/log/ceph/ceph-client.admin.44257.log: -- replaced with /var/log/ceph/ceph-client.admin.45341.log.gz 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44884.log 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50234.log 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57652.log 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44884.log.gz 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43696.log 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50192.log 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57652.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50234.log.gz 2024-03-30T13:13:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51680.log 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57652.log.gz 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.44257.log.gz 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50192.log: gzip 88.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.43696.log.gz -- 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.46967.log 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51680.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50192.log.gz 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37695.log 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46967.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51680.log.gz 2024-03-30T13:13:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52663.log 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60010.log 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37695.log.gz 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49367.log 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.60010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58571.log 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60010.log.gz 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60070.log 2024-03-30T13:13:33.489 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52663.log.gz 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52453.log 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49367.log.gz 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58571.log: /var/log/ceph/ceph-client.admin.60070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43442.log 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58571.log.gz 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52453.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60070.log.gz 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51182.log 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43442.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52453.log.gz 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-03-30T13:13:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49730.log 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43442.log.gz 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51182.log.gz 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45257.log 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49730.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz /var/log/ceph/ceph-client.2.48141.log 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45257.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49730.log.gz 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58403.log 2024-03-30T13:13:33.491 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45257.log.gz 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.2.48141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56892.log 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.48141.log.gz 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58403.log.gz 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48401.log 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56826.log 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.56892.log.gz 2024-03-30T13:13:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48401.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51014.log 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57174.log 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56826.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48401.log.gz 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43168.log 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57174.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51014.log.gz 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56826.log.gz 2024-03-30T13:13:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54632.log 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57174.log.gz 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43168.log.gz 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43210.log 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50803.log 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44490.log: gzip -5 89.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43210.log.gz /var/log/ceph/ceph-client.admin.50438.log 2024-03-30T13:13:33.494 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.54632.log.gz 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56166.log 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50803.log: /var/log/ceph/ceph-client.admin.50438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59968.log 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50803.log.gz 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58361.log 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50438.log.gz 2024-03-30T13:13:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59158.log 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58361.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59968.log.gz 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.56166.log.gz 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58361.log.gz 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52579.log 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59158.log.gz 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58842.log 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58697.log 2024-03-30T13:13:33.496 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52579.log.gz 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47702.log 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58842.log.gz 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55160.log 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58697.log: /var/log/ceph/ceph-client.0.47702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48569.log 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47702.log.gz 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58697.log.gz 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.55160.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58655.log 2024-03-30T13:13:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54500.log 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48569.log.gz 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55160.log.gz 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58655.log.gz 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49772.log 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54500.log: /var/log/ceph/ceph-client.admin.49115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46747.log 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49772.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-03-30T13:13:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56760.log 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49772.log.gz 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54500.log.gz 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45425.log 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46747.log.gz 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr: 88.4% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56760.log.gz 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45425.log: -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-03-30T13:13:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51224.log 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60110.log 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45425.log.gz 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51224.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gz 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46279.log 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51224.log.gz 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.60110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43612.log 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58613.log 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60110.log.gz 2024-03-30T13:13:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46279.log: /var/log/ceph/ceph-client.admin.43612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46279.log.gz 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43612.log.gz 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53322.log 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58613.log.gz 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50930.log 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43358.log: /var/log/ceph/ceph-client.admin.53322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59842.log 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-03-30T13:13:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58319.log 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50930.log.gz 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59842.log: gzip 91.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.53322.log.gz -- 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.52113.log 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58135.log 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59842.log.gz 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58319.log.gz 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-03-30T13:13:33.502 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53972.log 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58135.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52113.log.gz 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59674.log 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58135.log.gz 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44758.log 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53972.log: /var/log/ceph/ceph-client.admin.59674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59674.log.gz 2024-03-30T13:13:33.503 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53972.log.gz/var/log/ceph/ceph-client.admin.44758.log: 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58799.log 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46274.log 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44758.log.gz 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37439.log 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56100.log 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46274.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58799.log.gz 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-03-30T13:13:33.504 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46274.log.gz 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37439.log.gz 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56100.log: /var/log/ceph/ceph-client.admin.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43400.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52285.log 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56100.log.gz 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57973.log 2024-03-30T13:13:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.3.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48821.log 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52285.log.gz 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44842.log 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43780.log 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48821.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57973.log.gz 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52369.log 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48821.log.gz 2024-03-30T13:13:33.506 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58445.log 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44842.log.gz 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59506.log 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58445.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43780.log.gz 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52369.log.gz 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58926.log 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58445.log.gz 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56628.log 2024-03-30T13:13:33.507 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52159.log 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59506.log.gz 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58926.log.gz 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47135.log 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45047.log 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52159.log: /var/log/ceph/ceph-client.admin.47135.log: gzip -5 --verbose 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52159.log.gz 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56628.log.gz 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.48779.log 2024-03-30T13:13:33.508 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45047.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47135.log.gz 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45047.log.gz 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.49688.log 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49688.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49688.log.gz 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44716.log 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46643.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-03-30T13:13:33.509 INFO:teuthology.orchestra.run.smithi066.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51848.log 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56976.log 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51848.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44716.log.gz 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.48443.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51848.log.gz 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43570.log 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51552.log 2024-03-30T13:13:33.510 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56976.log.gz 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.43570.log: -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48443.log.gz 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.54236.log 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51552.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43570.log.gz 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48317.log 88.8% 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51552.log.gz/var/log/ceph/ceph-client.admin.54236.log: 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47093.log 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48863.log 2024-03-30T13:13:33.511 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54236.log.gz 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51350.log 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48317.log.gz 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52621.log 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47093.log.gz 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58093.log 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48863.log.gz 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46276.log 2024-03-30T13:13:33.512 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52621.log: gzip/var/log/ceph/ceph-client.admin.58093.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56298.log 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52621.log.gz 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46276.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58093.log.gz 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59758.log 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46276.log.gz 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57372.log 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59758.log.gz 2024-03-30T13:13:33.513 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46277.log: gzip -5 --verbose 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56298.log.gz 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.52495.log 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57372.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58884.log 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55666.log 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57372.log.gz 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52071.log 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52495.log.gz 2024-03-30T13:13:33.514 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55666.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58884.log.gz 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46883.log 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mgr.y.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52071.log.gz 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55666.log.gz 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59284.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46883.log.gz 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.515 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54698.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59284.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31422.log 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59284.log.gz 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51890.log 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.tmp-client.admin.31422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57438.log 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31422.log.gz 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53038.log 2024-03-30T13:13:33.516 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54698.log.gz 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47837.log 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51890.log.gz 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57438.log: /var/log/ceph/ceph-client.admin.53038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58015.log 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59464.log 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47837.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53038.log.gz 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58015.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47837.log.gz 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.57438.log.gz 2024-03-30T13:13:33.517 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58015.log.gz 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53406.log 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51764.log 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53406.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59464.log.gz 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54170.log 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51764.log: gzip -5 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53406.log.gz 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45299.log 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51764.log.gz 2024-03-30T13:13:33.518 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49646.log 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45299.log: gzip -5 91.0% --verbose -- /var/log/ceph/ceph-client.admin.59926.log 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54170.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45299.log.gz 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54302.log 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49646.log.gz 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57108.log 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54302.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59926.log.gz 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59242.log 2024-03-30T13:13:33.519 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57108.log: gzip -5 --verbose -- 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54302.log.gz 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.52411.log 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59242.log: gzip -5 90.6% -- replaced with /var/log/ceph/ceph-client.admin.57108.log.gz 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49199.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59242.log.gz 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52020.log 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52411.log.gz 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49199.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-30T13:13:33.520 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.0.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52020.log.gz 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58261.log 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55582.log 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58219.log 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58261.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2024-03-30T13:13:33.521 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55582.log: /var/log/ceph/ceph-client.admin.58219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51976.log 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58219.log.gz 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr: 89.4% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55582.log.gz 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51594.log 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51976.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50564.log 2024-03-30T13:13:33.522 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51594.log: /var/log/ceph/ceph-client.admin.46601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43528.log 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50564.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51594.log.gz 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50564.log.gz 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48695.log 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43528.log.gz 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:33.523 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48527.log 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56364.log 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48695.log.gz 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48527.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48527.log.gz 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43846.log 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56364.log: /var/log/ceph/ceph.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56364.log.gz 2024-03-30T13:13:33.524 INFO:teuthology.orchestra.run.smithi066.stderr: 85.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57042.log 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50709.log 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57042.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51266.log 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50709.log: /var/log/ceph/ceph-client.admin.51266.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57042.log.gz 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50709.log.gz 2024-03-30T13:13:33.525 INFO:teuthology.orchestra.run.smithi066.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51266.log.gz 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45937.log 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57931.log 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49409.log 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45937.log.gz 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57931.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49409.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57931.log.gz 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49409.log.gz 2024-03-30T13:13:33.526 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53114.log 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48947.log: /var/log/ceph/ceph-mds.c.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57240.log 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53114.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53114.log.gz 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50888.log 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57240.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57240.log.gz 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52996.log 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50888.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50888.log.gz 2024-03-30T13:13:33.527 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51722.log 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52996.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52996.log.gz 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59716.log 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51722.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51722.log.gz 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49934.log 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59716.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59716.log.gz 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53256.log 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49934.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz 2024-03-30T13:13:33.528 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53256.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53256.log.gz 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57570.log 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42730.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60152.log 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.57570.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57570.log.gz 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56562.log 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.60152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60152.log.gz 2024-03-30T13:13:33.529 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43296.log 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56562.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56562.log.gz 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49157.log 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43296.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43296.log.gz 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49157.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49157.log.gz 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43738.log 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44532.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-03-30T13:13:33.530 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49562.log 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43738.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43738.log.gz 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56496.log 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49562.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49562.log.gz 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59632.log 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.56496.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56496.log.gz 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59632.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59632.log.gz 2024-03-30T13:13:33.531 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44616.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50648.log 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46517.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58487.log 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50648.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50648.log.gz 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55094.log 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.58487.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58487.log.gz 2024-03-30T13:13:33.532 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51308.log 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.55094.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55094.log.gz 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51510.log 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51308.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51308.log.gz 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52537.log 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51510.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51510.log.gz 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52537.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52537.log.gz 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-30T13:13:33.533 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49976.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59337.log 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mds.a.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50972.log 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.59337.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59337.log.gz 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54104.log 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50972.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-30T13:13:33.534 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50972.log.gz 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50360.log 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45607.log 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50360.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.54104.log.gz 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50360.log.gz 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45938.log 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45607.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45607.log.gz 2024-03-30T13:13:33.535 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45938.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45938.log.gz 2024-03-30T13:13:33.536 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-30T13:13:33.676 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-30T13:13:34.108 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-30T13:13:34.141 INFO:teuthology.orchestra.run.smithi117.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-30T13:13:34.169 INFO:teuthology.orchestra.run.smithi117.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-30T13:13:34.252 INFO:teuthology.orchestra.run.smithi066.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-30T13:13:34.274 INFO:teuthology.orchestra.run.smithi117.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-30T13:13:34.276 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-03-30T13:13:34.276 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m0.957s 2024-03-30T13:13:34.276 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m3.532s 2024-03-30T13:13:34.276 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.157s 2024-03-30T13:13:34.301 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-30T13:13:34.352 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-30T13:13:34.404 INFO:teuthology.orchestra.run.smithi066.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-30T13:13:34.495 INFO:teuthology.orchestra.run.smithi066.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-30T13:13:34.497 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-30T13:13:34.497 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m1.203s 2024-03-30T13:13:34.497 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m5.340s 2024-03-30T13:13:34.498 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.335s 2024-03-30T13:13:34.498 INFO:tasks.ceph:Archiving logs... 2024-03-30T13:13:34.499 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/remote/smithi066/log 2024-03-30T13:13:34.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T13:13:35.432 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/remote/smithi117/log 2024-03-30T13:13:35.433 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T13:13:35.964 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-30T13:13:35.974 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-30T13:13:35.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T13:13:35.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T13:13:36.071 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-30T13:13:36.080 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-30T13:13:36.081 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:13:36.084 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:13:36.096 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-03-30T13:13:36.099 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-03-30T13:13:36.101 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T13:13:36.102 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-03-30T13:13:36.102 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +70us[ +76us] +/- 92ms 2024-03-30T13:13:36.102 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -206us[ -199us] +/- 50ms 2024-03-30T13:13:36.102 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +197us[ +204us] +/- 64ms 2024-03-30T13:13:36.102 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:13:36.104 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T13:13:36.104 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-03-30T13:13:36.105 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +101us[ +101us] +/- 92ms 2024-03-30T13:13:36.105 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -180us[ -183us] +/- 50ms 2024-03-30T13:13:36.105 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +206us[ +206us] +/- 64ms 2024-03-30T13:13:36.105 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T13:13:36.106 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-30T13:13:36.115 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-30T13:13:36.116 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-30T13:13:36.125 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T13:13:36.229 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T13:13:36.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-30T13:13:36.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:13:36.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-30T13:13:36.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:13:36.369 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-30T13:13:36.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-30T13:13:36.389 INFO:teuthology.task.internal:Duration was 781.657334 seconds 2024-03-30T13:13:36.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-30T13:13:36.399 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-30T13:13:36.399 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T13:13:36.402 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T13:13:36.450 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T13:13:36.454 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T13:13:36.730 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-30T13:13:36.730 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-03-30T13:13:36.730 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T13:13:36.751 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-03-30T13:13:36.751 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T13:13:36.777 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-30T13:13:36.777 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T13:13:36.793 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T13:13:36.846 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-30T13:13:36.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T13:13:36.865 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T13:13:36.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-30T13:13:36.997 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-30T13:13:36.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T13:13:37.028 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T13:13:37.055 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-30T13:13:37.066 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:13:37.071 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:13:37.098 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-03-30T13:13:37.120 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-03-30T13:13:37.136 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:13:37.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:13:37.172 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:13:37.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:13:37.194 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-30T13:13:37.205 INFO:teuthology.task.internal:Transferring archived files... 2024-03-30T13:13:37.206 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/remote/smithi066 2024-03-30T13:13:37.206 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T13:13:37.253 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149/remote/smithi117 2024-03-30T13:13:37.254 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T13:13:37.289 INFO:teuthology.task.internal:Removing archive directory... 2024-03-30T13:13:37.289 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T13:13:37.291 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T13:13:37.338 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-30T13:13:37.349 INFO:teuthology.task.internal:Not uploading archives. 2024-03-30T13:13:37.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-30T13:13:37.359 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-30T13:13:37.359 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T13:13:37.362 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T13:13:37.374 INFO:teuthology.orchestra.run.smithi066.stdout: 265594 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 13:13 /home/ubuntu/cephtest 2024-03-30T13:13:37.393 INFO:teuthology.orchestra.run.smithi117.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 13:13 /home/ubuntu/cephtest 2024-03-30T13:13:37.394 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-30T13:13:37.410 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-30T13:13:37.431 INFO:teuthology.nuke:Checking targets against current locks 2024-03-30T13:13:37.460 DEBUG:teuthology.nuke:shortname: smithi066 2024-03-30T13:13:37.460 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T13:13:37.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:01.285193', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFY4upZpo7cB0coVquEDOdrwqJg6OHFD4EoV56xP3Skyi4ezLzOlJj9k+KZRYRwcxBjfPZvKAZSZ0i/etF3yKMU='} 2024-03-30T13:13:37.489 DEBUG:teuthology.nuke:shortname: smithi117 2024-03-30T13:13:37.490 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T13:13:37.509 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629149', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:01.286856', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwDGvp2eo//D/JeOH1AxX2bT9k/mPLPwvuLr2Wx8VimpD9fbDot15X4u0nzdw4lsMuJtCNAREBkkp+FK9HhG0s='} 2024-03-30T13:13:37.541 INFO:teuthology.orchestra.console.smithi066:Power off 2024-03-30T13:13:37.541 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T13:13:37.557 INFO:teuthology.orchestra.console.smithi117:Power off 2024-03-30T13:13:37.558 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T13:13:37.568 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2024-03-30T13:13:37.568 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:13:37.581 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-03-30T13:13:37.581 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:13:37.594 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-30T13:13:37.610 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-30T13:13:41.596 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:13:41.612 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:13:41.707 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-30T13:13:41.722 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-30T13:13:45.709 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:13:45.724 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:13:45.820 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2024-03-30T13:13:45.835 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-03-30T13:13:45.921 INFO:teuthology.orchestra.console.smithi066:Power off completed 2024-03-30T13:13:45.936 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-03-30T13:13:46.041 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} duration: 781.6573340892792 failure_reason: 'Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=79f3f4b5ec5841e8b898c6ef751a59d2 status: fail success: false 2024-03-30T13:13:46.042 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T13:13:46.117 INFO:teuthology.run:FAIL